- Reliability 12/20
- Static review sees only the README and server.json; no CI workflow files, committed test sources, or reproducible execution evidence for key paths are present in the supplied material. The README self-claims 2,437 tests, 253 client-config checks, a verify:all suite and benchmark scripts (bench/compression/proof.mjs, session-replay.mjs) plus very specific figures (119/120 passes, 0.926x median cost), but all of this is untrusted evidence from the artifact itself with no third-party verification, so per the calibration note reliability cannot exceed 12. The README is also internally contradictory: one section states TOKEN_OPTIMIZER_PROXY defaults off (switch table) while another says the proxy is enabled by default; harvest is described both as 'not opt-in'/default-on and as credential-gated. Such contradictions directly weaken confidence that declared tools match real behavior. Deducted for: no verifiable CI/test evidence, contradictory default-value statements, and error/edge-case handling presented only as prose.
- Security and permissions 9/20
- Notable but sub-red-line risk. Positives: the manifest exposes only one optional non-secret env var with no embedded credentials; the README states no telemetry, no hosted service, MIT license, forwards rather than stores the provider key, and mentions npm provenance, CHECKSUMS.sha256, and exact uninstall of only self-written files. Risks: installation writes persistent host changes (shell activation, User PATH, owned .cmd launchers, merging into settings.json), a broad privilege footprint; the local proxy writes elided tool-result content in plaintext to OS temp spill files (0600, HMAC salted names, removed on shutdown) and the README itself concedes this may include sensitive conversation content it does not scan for secrets; optional TOKEN_OPTIMIZER_PROXY_CAPTURE writes plaintext request bodies to a named directory; the knowledge graph and harvest may send content to local/remote model endpoints. These are documented, switchable and scoped, and no credential theft, covert exfiltration, or unconfirmed destructive default is evident, so no red line applies, but the enabled-by-default proxy, plaintext spill/capture, and host mutation cap the score. Deducted for: default local proxy with possible plaintext spill files, install-time PATH/shell/settings persistence, and opt-in plaintext request capture.
- Maintenance 14/20
- Repository metadata supports active maintenance: version 7.0.1, an npm package @ooples/token-optimizer-mcp published in sync, a clear MIT license, 531 stars, only 5 open issues, and not archived. The README references a CI workflow badge, commit hashes (e.g. f712b4ec) and dated evidence directories, indicating sustained iteration, versioned evidence and a defined release path. However the repo description appears inflated/inconsistent with actual function (claiming 'live local knowledge graph across 16 CLI clients' while the core is compression/optimization), the manifest description (60-90% reduction) is in tension with the README's repeated admissions that superiority is not established, the credential-gated harvest and causal evidence remain 'Collecting' (unfinished capability), and unverifiable model names like 'gpt-6-astra' appear. No explicit security-response channel is described. Deducted for: capability/description inconsistencies, partly in-progress features, and no independent governance info, while the release and license path is clear, giving a solid mid-high score.
- Documentation 16/20
- Documentation is unusually thorough and well above average: install steps (Claude Code plugin, Codex plugin/mcp add, one-line npx), verification commands (npx token-optimizer-doctor feeds a synthetic payload and asserts a large read is refused), diagnostics commands, troubleshooting, links to TOKEN_ACCOUNTING and EVIDENCE_PROTOCOL, a client capability tiering (lifecycle/native/rules), cost and cache-multiplier explanations, and extensive self-declared limitations ('superiority not established', 'not claimed that embedding beats BM25', 'replay does not establish reasoning survives compression'). That proactive disclosure is a plus. Deductions: internal contradictions (proxy default on/off, inconsistent harvest gating), some numbers are enormous and unverifiable statically, a few implausible model names, and marketing-style comparison tables mixed through long prose. Scored in the 'layered docs with examples and disclosed limitations but hidden assumptions/contradictions' band.
- Setup experience 13/20
- The install path is generally clear: a one-line npx stdio launch for mainstream clients, a plugin-marketplace install for Claude Code, two Codex routes (plugin and mcp add), and 16 generated client configs under integrations/. However complexity is high: the README states that installing only the MCP server just gives ignorable tools, and real effect requires additional native plugins/hook bundles per client, possibly /reload-plugins and a terminal reopen to load PATH, with npm 11 blocking lifecycle scripts unless allow-scripts is enabled. Several CLI-subcommand installs (codex plugin add, /plugin install) cannot be verified in static source, and Windows adds .cmd shims and PowerShell execution-policy considerations. Per the static calibration cap of 15, and given multi-step native integration and lifecycle dependency, it is scored 13. Deducted for: MCP-only install being insufficient, extra plugin/hook and terminal reload steps, and platform/lifecycle-script friction.