- Reliability 9/20
- Evidence: README shows a complete compose architecture (FastAPI backend, Vue frontend, MCP server under src/mcp-server with 90+ claimed tools, scheduler, Redis/SQLite/Vector); deployment scripts (install.sh, start.sh, quickstart.sh); a testing guide and hygiene skills are mentioned. Deductions: static review cannot execute anything; no CI workflow files or committed test suite paths are provided as verifiable evidence; the MCP tool list is only asserted in the README with no actual tool schemas; 272 open issues and a multi-container dependency set (Docker, Redis, Postgres, Vector, OTel) make clean startup and controllable failure modes uncertain.
- Security and permissions 12/20
- Evidence: README explicitly describes Docker container isolation per agent, encrypted credential storage (.credentials.enc), a 4-tier RBAC hierarchy, append-only audit trail, HMAC webhook signatures, x402 payment confirmation, verified-email allow-lists, configurable guardrails, and an independent UnderDefense pentest (Grade A); SECURITY.md is referenced as a disclosure channel. Deductions: none of these claims are backed by source-level verification in this static review (actual RBAC enforcement, audit immutability, encryption details); install examples use localhost and Bearer placeholders with no real tokens found; dangerous operations (remote deploy, payments, ephemeral SSH) lack explicit confirmation-step documentation in the README.
- Maintenance 15/20
- Evidence: Apache-2.0 license; README indicates an active release cadence (v0.8.5 dated 2026-07-26) with versioned release notes, an automated CLI publishing path to PyPI/Homebrew, CONTRIBUTING.md, SECURITY.md, and GitHub Issues/Discussions channels. Deductions: 430 stars against 272 open issues is a high ratio; commit/release frequency cannot be verified from the untrusted README; no concrete dependency-update or security-advisory process evidence is supplied.
- Documentation 14/20
- Evidence: README is a layered docs hub: quick start, manual install, agent templates, multi-agent YAML examples, MCP client JSON config, 90+ tools grouped by category, CLI reference, deployment guide, PostgreSQL setup, known issues, testing guide, security attestation, and an AGENTS.md router for AI agents. Deductions: MCP tools table lacks parameter/input schemas, error codes, and rate limits; the 90+ tool count is claimed but no actual tool definition files were shown; known-issues doc is linked but its content is not evidenced; auth/install steps rely on README assertions that static review cannot confirm.
- Setup experience 12/20
- Evidence: README offers multiple install paths: one-line curl script, manual Docker steps, interactive quickstart.sh, CLI via pip/brew, Claude Code plugin marketplace, and a ready-to-paste MCP client JSON example (streamable-http at localhost:8080/mcp with Bearer auth); prerequisites are only Docker plus an API key. Deductions: no CI workflow or committed tests are provided as verified execution evidence, so the static calibration caps setup at 15; the multi-container dependency set (Docker Compose, Redis, PostgreSQL, Vector, OTel) and private optional submodules make installation potentially fragile; no Windows/other-platform compatibility notes; end-to-end verification steps after MCP connection are thin.