- Reliability 12/20
- Static review sees only the README and server.json. The manifest declares the PyPI package compartment 4.9.6 with uvx runtime, stdio transport and a `serve` positional argument, which is consistent with the README's `compartment serve`, so a clean handshake looks plausible. The README claims extensive internals (evidence-fusion ranking, windowed embeddings, SIMD/HNSW, a shared embedding daemon, tested kill -9 recovery, 800+ tests, three-OS CI badges) and lists thirteen MCP tools with read-only/destructive annotations, but the provided material contains no test files, no CI workflow, no source excerpts and no execution evidence, and README/manifest are explicitly untrusted. Declared tools therefore cannot be checked against real behavior, error/dependency control (bundled ONNX model, file-lock serialization, auto-lock) is only asserted, and per the static calibration the absence of verifiable execution evidence caps this at 12.
- Security and permissions 16/20
- No red line is crossed: no remote fetch, no embedded token, no real secrets in install examples, stdio-only transport with no open ports, and a runtime offline guard (`--assert-offline`) reportedly run in CI on three platforms. The documented design is strong: XChaCha20-Poly1305 at rest including embedding vectors, Argon2id LUKS-style key slots, per-record keys enabling `forget --shred`, hash-chained audit log, Ed25519-signed manifests and packs verified against a caller-trusted key, per-agent namespace grants (rw/ro/none), and `memory_unlock` disabled by default so the passphrase does not enter model context; unlock credentials are wrapped with a per-boot secret never written to disk. Deductions: every security property is self-reported with no implementation, audit or in-repo threat model (only links out), opt-in keychain persistence and 2FA keyfile-location memory are unverifiable, and prompt-injection mitigation is textual (quarantine marking plus a data-not-instructions notice) while explicitly leaving the host agent responsible. Hence 16, not full marks.
- Maintenance 14/20
- License is clear (Apache-2.0), the repo is not archived, there are 0 open issues, and 586 stars are treated only as a discovery signal per the rules. The README references RELEASING.md, CONTRIBUTING.md, a SECURITY.md reporting channel and Discussions, and version 4.9.6 plus narrative change notes (4.7 ranking-fusion change, 4.9.6 memory-footprint fix) indicate ongoing iteration with semver-style releases. However, no commit history, release cadence, dependency-update record or independently verifiable maintainer identity is present in the supplied material; the only ownership evidence is the repository id/link in server.json. Governance and update paths are pointed to but not evidenced, so 14 rather than full.
- Documentation 18/20
- Documentation depth and layering are unusually good: install paths (pip/pipx/uv/Docker/macOS .pkg), auth and lock model, the full MCP tool list with read-only/destructive annotations, a complete settings table with defaults and meanings, environment variables, a command reference table, cross-links to MEMORY/SECURITY/FORMAT/PACKS/INTEGRATIONS/COMPARISON, disclosed limitations (plaintext export safety, encoder token-window truncation, non-comparable cross-query scores, importance only reordering matches), and reproducible benchmark commands (selftest/bench). Deductions: all performance numbers and the competitor comparison table are self-reported, dated in the future (2026-09-02) and unverifiable, and several referenced documents are not included in the supplied material, leaving unproven assumptions. Strong enough for 18.
- Setup experience 15/20
- The install path is clear and mainstream-client friendly: `pip install compartment && compartment init` or pipx/uv, Docker (vault must be pre-created on the host), plus a no-terminal macOS .pkg; init wires Claude Code/Hermes/OpenClaw and starts the panel. Client config examples are explicit for Claude, Cursor, VS Code (`servers` key with `type: stdio`), Zed (`context_servers`), Codex (TOML), LM Studio/goose/Kiro deeplinks and the generic `{"mcpServers":{...}}` block; `integrate --list/--all` claims 28 clients with byte-exact backups, merge-not-replace atomic writes and a paste-instead-of-touch fallback on unparseable configs. Constraints are stated (Python 3.11+, Windows VC++ runtime, macOS 13+). Because no CI workflow or committed tests are present, the static calibration caps setup at 15: the steps are clear and few, but the automated wiring behavior is unverifiable and the Docker path needs a manual pre-step.