- Reliability 9/20
- The README describes a three-tool gateway pattern (search_help_scout/describe_help_scout/read_help_scout) claiming coverage of 55 read operations, and references npm test/type-check/lint commands, but only the README and server.json were provided here — no actual source code, CI workflow files, or test contents were included. There is no way to confirm the declared behavior matches the real implementation or that errors/dependencies are actually controllable. Per the static calibration rule, without verifiable execution evidence (real CI plus committed tests covering key paths), reliability is capped at 12 and further reduced for the evidence gap.
- Security and permissions 15/20
- Credential handling looks reasonable: OAuth2 App ID/Secret and the Docs API key are all marked isSecret, and config examples use placeholders rather than real tokens. The default install is read-only; write access requires an operator to explicitly set env flags, and the customer-facing high-risk operations (sendReply/publishDraft) require an extra flag plus per-call confirm/confirmOperation/targetId triple confirmation, with deletes and admin-config writes explicitly excluded. The redaction feature is honestly caveated as not a compliance boundary and not removing all identifiers, which is a positive disclosure. However, all of this comes from README/manifest claims (untrusted evidence) — static review cannot verify the confirmation logic is actually enforced in code, so full marks are not warranted, though no red-line issue was found.
- Maintenance 12/20
- The project is MIT licensed, and the manifest shows a clear version (2.1.0) published to both npm and a Docker registry, indicating some release process. However, the material lacks evidence of commit history, release cadence, issue response times, dependency update practices, or a dedicated security disclosure channel — only GitHub Issues/Discussions are listed as support, which doesn't substitute for governance transparency. Star count and open-issue count are discovery signals only and are not credited, and missing evidence is treated as a deduction rather than assumed best case.
- Documentation 17/20
- Documentation is well-layered, covering install steps for multiple clients (Claude Desktop, Claude Code, Cursor/VS Code, Docker), credential acquisition, an operation reference table, a detailed write-gating and confirmation protocol with JSON examples, a configuration table with defaults, a troubleshooting section, and an honest caveat about the limits of message redaction. Points were deducted because several referenced internal guide files (e.g., mcp-tool-contract.md) were not available for review, and rate-limit/cost disclosure is thin, mentioning only automatic retry behavior.
- Setup experience 14/20
- Setup paths cover one-click mcpb install, a Claude Code CLI command, generic MCP client JSON config, and a Docker run example, with clear steps and complete config samples plus explicit multi-platform compatibility notes. However, since no CI workflow or test files were available to verify actual execution, setup is capped at 15 per the static calibration rule.