- Reliability 8/20
- Static, source-only review; nothing was executed. The README claims #1 positions on AppWorld and WebArena, but those are external leaderboard badges that cannot be verified from the supplied material and are not scored. The project is a large enterprise agent harness (demos, supervisor, knowledge, a separate events service, Helm deployment), so the MCP init and tool-list handshake path is not concretely demonstrated. The README enumerates a test matrix (unit, policy integration, SDK, stability with an 87% gate), but no CI workflow contents or committed test coverage for the server's key paths are shown, so this cannot be credited under the static calibration. The cap of 12 applies; deductions for unverifiable key paths, unconfirmed tool-behavior match, and a broad dependency surface (Python 3.12, uv, optional extras, browser extension, container/E2B sandboxes).
- Security and permissions 7/20
- Positive signals exist: a policy framework (Intent Guard, Tool Approval with human-in-the-loop confirmation, Tool Guide, Output Formatter), HITL gates, code sandboxes (local / Docker-Podman / E2B isolation), and tool-call budgets, which support least-privilege and confirmation goals. However, several items widen the boundary: README examples contain credential-shaped assignments (OPENAI_API_KEY=... in .env, though placeholders), E2B usage requires exposing the local registry/service publicly via ngrok, MCP servers are launched as stdio commands via uvx, and final_answer.function imports and calls a dotted-path function with no sandbox (README itself warns to point it only at trusted code). No malware, credential theft, covert exfiltration, or default irreversible delete/pay/deploy signals appear, so the 0-4 red-line band does not apply; the score is held below the midpoint because permission scoping, confirmation completeness, and external exposure are not end-to-end verifiable here.
- Maintenance 12/20
- Evidence is limited. The description shows 876 stars, 177 open issues, and an unarchived repository, and the README shows a live feature matrix and roadmap, indicating activity. But the license field is NOASSERTION, meaning the license is not identified in the supplied material, which is a real governance gap. No release history, commit cadence, dependency-update policy, or security-response channel is verifiable, and the issue count is high with no response-time evidence. Scored at the 'active but governance/versioning gaps' anchor without inflation.
- Documentation 13/20
- The documentation is layered: quick start, multi-provider LLM configuration (OpenAI/WatsonX/Azure/Groq/RITS/OpenRouter/wxO), mode and settings tables, sandbox and knowledge-base sections, supervisor, shortlister, and tool-integration examples, with many pointers to docs.cuga.dev and in-repo paths. Some limitations are disclosed explicitly (run receipts measure tokens not cost; final_answer instructions are ignored in default Lite mode; supervisor-synthesized answers are not shaped). Gaps remain: license is undeclared, the server manifest is not cached, cost/quota and troubleshooting coverage is uneven, and much detail lives on external sites that cannot be reviewed here, so it falls short of the top band.
- Setup experience 9/20
- Clear entry steps are given: git clone, uv venv --python=3.12, uv sync, write .env, then `cuga start demo_crm --read-only`, plus multiple optional paths (sandbox, E2B, supervisor, knowledge, skills, Helm). Complexity is introduced by platform prerequisites and container runtimes (Rancher Desktop/Docker, E2B CLI, ngrok) and by hybrid/browser modes needing Chromium plus a browser extension. No verifiable install output or CI config body is present in the static evidence, so the calibration cap of 15 applies; points are deducted for the multi-branch, multi-dependency, non-single-command fragility.