- Reliability 8/20
- This is a static-only review; no server manifest was cached, and no committed CI workflows or test files were supplied as evidence. The README is largely marketing copy and offers no verifiable proof of a real MCP init/tool-list handshake or that declared capabilities match actual behavior. Notably, this project is better characterized as an inspector/client platform for testing other MCP servers rather than a conventional MCP server exposing tools — the README's sweeping claims (16 client configs, 170+ models) are unverified by any automated test or CI evidence in the provided material. Per the static calibration rule, reliability cannot exceed 12 without real CI plus committed tests covering key paths, so 8 is given, reflecting a plausible happy path with unverified edge-case/error handling.
- Security and permissions 13/20
- The evidence shows some security-conscious design: an OAuth debugger for diagnosing authorization flows; a stated guarantee that local skills read from the filesystem 'never leave your machine'; explicit Docker instructions to bind to 127.0.0.1 (not 0.0.0.0) with a stated security rationale; and the hosted app deliberately excluding STDIO/local file access to limit risk. No red-line issues (malware, hardcoded secrets, fabricated identity, unconfirmed destructive defaults) were found. However, the README does not detail how API keys, chat transcripts, or workspace data are handled/retained in the hosted service, nor does it describe confirmation steps for higher-risk operations (OAuth registration, outbound network calls), so permission scoping and data-flow disclosure remain incomplete.
- Maintenance 11/20
- The repository is not archived, and its star count (2136) and open-issue count (206) indicate meaningful activity and attention, but no close rate, release cadence, or version history was provided as evidence. More significantly, there is a licensing contradiction: the repo metadata reports 'NOASSERTION' (no asserted license), while the README badge and text claim 'Apache License 2.0' and point to a LICENSE file — this mismatch between stated and actual license metadata is treated as a deduction in a static review rather than resolved in the README's favor. No formal security-response channel is evident beyond a general Discord community link.
- Documentation 13/20
- The README is well organized, with layered sections (feature tables, install paths, Docker notes, contribution guide) and links out to docs.mcpjam.com for deeper documentation. Those external docs cannot be verified for accuracy or completeness within this static review. No in-repo tool parameter listing was available (manifest not cached), no troubleshooting section is present, and cost/rate-limit details for the 'free' chat offering are unspecified — leaving usable but incompletely disclosed documentation.
- Setup experience 13/20
- The README provides several clear install paths (hosted app, desktop installer, npx terminal run, Docker build-from-source), states the Node.js version requirement, and calls out a Docker security binding caveat — overall a well-documented setup story. Per the static calibration rule, setup is capped at 15 absent real CI workflows and committed tests demonstrating execution; since no such verifiable execution evidence was supplied, a score of 13 is given, reflecting clearly described but unverified setup steps.