- Reliability 7/20
- No server manifest was cached and the README contains no tool input/output schemas, real invocation traces, or CI/test evidence, so declared capabilities ('5 built-in MCPs', '54+ lifecycle hooks', etc.) cannot be verified against real behavior. The project references a 'Doctor Command' diagnostic and 'Session Recovery' feature, but neither is backed by reviewable evidence in the provided material. 983 open issues suggest a nontrivial volume of reported failures in a complex system. Per the static calibration rule, scores above 12 require verifiable execution evidence, which is essentially absent here — scored 7.
- Security and permissions 9/20
- Telemetry is enabled by default (opt-out only), sending a SHA256-hashed install identifier to PostHog on a recurring basis — a persistent, disclosed-but-default-on external data flow. Installation modifies global config files (e.g., ~/.codex/config.toml, opencode.json) and writes into locations like ~/.local/bin. The Codex Light installer explicitly offers an 'autonomous full-permissions mode,' which if enabled reduces confirmation friction for agent actions. The README also instructs users to have an LLM agent fetch and execute a remote installation doc 'end to end' without summarizing — a workflow pattern structurally susceptible to prompt injection if that remote document were ever compromised. No evidence of malware, credential theft, or irreversible unconfirmed destructive defaults was found, so this does not fall into the red-line band, but permission scoping and confirmation disclosure remain incomplete — scored 9.
- Maintenance 14/20
- The project shows strong star count, an explicit ROADMAP, active Discord community, multilingual READMEs, and a visible 'building in public' cadence, indicating an identifiable and active maintainer (code-yeongyu / Sisyphus Labs). However, the GitHub license field is NOASSERTION while the README badge shows a non-standard 'SUL-1.0' license, leaving licensing governance ambiguous and potentially restricting downstream rights; 983 open issues also suggest a sizeable maintenance backlog. High activity offset by licensing/governance gaps yields a score of 14.
- Documentation 13/20
- Documentation is well-layered, with dedicated pages for installation, features, configuration, and Team Mode, plus multilingual README variants — broad coverage. However, the tone is heavily promotional, and concrete technical details (tool parameter tables, rate/cost limits, explicit troubleshooting guidance) are thin or absent from the material provided — e.g., no parameter specification for the built-in MCPs or explicit cost/limit disclosures. Scored 13.
- Setup experience 9/20
- An install path exists (bunx/npx one-liners with a guided TUI), but the actual process is multi-step and involves choosing between Ultimate/Light editions, subscription detection, model matching across 11 agents, multiple provider authentications, and editing global config files. The maintainers explicitly recommend having an LLM agent perform the install to avoid human error, which itself signals a fragile manual path. No CI or test evidence was provided to substantiate install reproducibility, so per the static calibration ceiling the score stays below 15; combined with process complexity, scored 9.