- Reliability 9/20
- Static evidence is limited to README plus server.json; no source code, tests, or CI workflow files were supplied, so the calibration cap of 12 applies. Positive signals: the manifest declares an OCI image at 1.5.0 over stdio, and the README documents six MCP tools (recall, remember, forget, purge_forgotten, move_memory, memory_status) and two resources (memory://spaces, memory://stats), plus module launch commands and named startup failures (ModuleNotFoundError, spaCy E050), which shows the author has thought about init failure paths. Deductions: the manifest requires four env vars but declares no preflight validation of database reachability, the pgvector extension, or the spaCy model; the README's Features section claims 'four tools' while the Tools table lists six, an internal contradiction, and tool-vs-behavior parity cannot be verified statically; no tests or CI evidence; purge_forgotten is irreversible yet no confirmation mechanism is described. Scored 9: the happy path is plausible, but error surfaces and dependency controllability are thinly evidenced.
- Security and permissions 12/20
- No red line is triggered: no static evidence of malware, credential theft, covert exfiltration, or fabricated identity, and every credential in the install examples is a placeholder (memory_vault, mv_your-token-here), not a real secret, so the 0-4 band does not apply. Positive evidence: DB_PASSWORD is flagged isSecret in the manifest; the HTTP/SSE MCP transport is off by default and requires MCP_HTTP_ENABLED=true; the README states API_AUTH_ENABLED=false does not open the MCP transport and unauthenticated calls still get 401; a Host-header allowlist returning HTTP 421 mitigates DNS rebinding; it advises binding to a LAN address and fronting with a reverse proxy/TLS and discloses there is no transport encryption; tokens are revocable and shared between REST and MCP via one table; the diagnose bundle scrubs tokens and passwords. Deductions: API_HOST defaults to 0.0.0.0 and API_CORS_ORIGINS defaults to '*'; the dashboard stores bearer tokens in localStorage, an XSS surface; forget is a soft delete but purge_forgotten is a permanent one with no documented confirmation or dry-run; a single read/write DB user means no least-privilege split; single-instance with no multi-tenant isolation; the Host check, rate limiting, and redaction implementations cannot be inspected. Scored 12: main risks are visible and deliberately narrowed, but confirmation flows, least privilege, and verifiability are incomplete.
- Maintenance 11/20
- Evidence: an explicit MIT license; SECURITY.md provides a private vulnerability channel via email; CONTRIBUTING.md, CODE_OF_CONDUCT.md (Contributor Covenant v2.1), a threat-model doc, and issue/PR templates are all referenced; the README reports M1-M7 shipped, v1.0 released 2026-05-07, with a semver promise and additive-only migrations within v1.x; ACKNOWLEDGMENTS credits 20+ external issues across the v1.0.7-v1.0.10 window, implying a real response loop; the manifest version 1.5.0 aligns with README-mentioned v1.5.0 changes (per-query ef_search, index warming). Deductions: source and CHANGELOG are unreadable, so commit cadence, dependency-update policy, and actual issue-response latency cannot be verified; a self-described single maintainer with no governance or succession plan; 11 open issues with no visible triage state; no evidence of a dependency pinning strategy across sentence-transformers, spaCy, pgvector, FastMCP, and React 19; the PRO/OSS boundary remains a plan. Scored 11: active with a sound license and security channel, but governance and version-management details have gaps.
- Documentation 14/20
- The evidence is layered and substantial: Quick Start with both Docker and No-Docker paths; uv and pip+venv installs; Claude Code configuration at project scope (.mcp.json) and global scope (~/.claude/.mcp.json plus enabledMcpjsonServers) alongside Claude Desktop; the Docker-specific DB_HOST=127.0.0.1 guidance; a full remote HTTP/SSE section covering the flag, URL, Bearer header, MCP_HTTP_ALLOWED_HOSTS semantics including that the list replaces rather than extends the default, and what 421 means; a REST endpoint table; auth and token lifecycle; a config-variable table; performance tuning (maintenance_work_mem); a dedicated Limitations section (English-only NER, no alias merging, no re-extraction on edit, single-instance, LM Studio only); an FAQ; troubleshooting for ModuleNotFoundError, spaCy E050, `claude --debug mcp`, CORS, and blocked localStorage; and a bug workflow built on X-Request-ID plus `memory-vault diagnose`. Deductions: the Features section saying 'four tools' directly contradicts the six-row Tools table, an internal inconsistency; tool parameter signatures (recall/remember fields, limit, space semantics) are absent beyond one-line summaries; Local LLM Chat requires LM Studio with a model blocklist justified only narratively and no version floor; cost/resource figures are approximate; and there are no example tool-call responses. Scored 14: install, auth, limits, and troubleshooting are well covered, but parameter-level reference and internal consistency have gaps.
- Setup experience 10/20
- Evidence: the Docker path is three commands (git clone, cd, docker compose up -d) with automatic migrations, a dashboard at http://localhost:8000, and volume-persisted data; the No-Docker path enumerates Python 3.11+, Postgres 16 + pgvector, and full uv or pip+venv sequences (uv sync, spaCy model download, .env, migrate, status); copy-pasteable JSON config blocks are given per client, distinguishing Claude Code project vs global scope and Claude Desktop; the Docker DB_HOST difference is called out; Windows users are routed to docs/windows.md with a WSL2 recommendation. Deductions: every deployment requires an external Postgres 16 + pgvector instance, a heavy prerequisite, and the manifest marks DB_HOST, DB_PORT, DB_NAME, DB_USER, and DB_PASSWORD all required, so any one missing blocks startup; the spaCy model is a separate download whose omission yields OSError [E050]; global-scope MCP requires hand-editing ~/.claude/settings.json, which is easy to miss; no sample .env body is shown (only a pointer to .env.example); no offline/GPU or image-verification guidance; and no minimal connectivity check beyond memory_status and `claude mcp list`. With the static cap of 15 and evidence drawn only from unverified README/manifest text, scored 10: the path is clear but manual steps plus external-database coupling make first-run connection fragile.