- Reliability 10/20
- The manifest specifies a PyPI package, version, Python runtime, stdio transport, and a default tool preset. The README also describes structured failure envelopes, partial-failure handling, and CI workflows. However, this review includes no server implementation, tests, or actual handshake evidence, so MCP initialization, tool-list consistency, dependency behavior, and edge cases cannot be verified. The score is therefore conservative.
- Security and permissions 13/20
- The README claims local analysis, no API keys, a disclosed network boundary, MCP egress secret scrubbing, mode gating, and HMAC-linked receipts; these are positive design signals. It also discloses first-use grammar downloads and optional network-capable commands. Still, the controls are not supported by source code or tests here, and write confirmation, permission scope, path isolation, and network behavior remain unverifiable. No stated red-line risk is evident.
- Maintenance 16/20
- The project is not archived and provides an Apache-2.0 license, an identifiable repository, version 14.0.0, and multiple recent release notes. The README also describes CI, supply-chain hardening, and only a small number of open issues, indicating an active maintenance path. There is no supplied commit history, dependency-update record, vulnerability-response policy, or verifiable maintainer-response evidence, so points are deducted.
- Documentation 17/20
- Documentation covers installation, Python requirements, MCP client configuration, default and full toolsets, presets, network boundaries, limitations, performance, CI, failure envelopes, and the limits of several measurements. It provides many examples and discloses PATH, indexing, and version-evaluation caveats. The actual tool schemas, implementation behavior, and troubleshooting files are not supplied, and the README excerpt is incomplete at the end, so full marks are not justified.
- Setup experience 13/20
- The setup path is clear: install roam-code[mcp], initialize an index, run roam mcp, and configure clients such as Claude, Cursor, and VS Code. Python 3.10+ and Linux, macOS, and Windows are stated as supported. Deductions apply because the client depends on a roam executable being on PATH, indexing and grammar-cache setup are prerequisites, and no real connection or handshake test evidence is provided.