- Reliability 4/20
- Evidence: the manifest declares the npm package @coolver/home-assistant-mcp version 3.2.31 with stdio transport and required environment variables; the README describes features such as analysis, automation creation, dashboards, themes and deployment. Deduction: no source code, CI or tests were provided, so the MCP init handshake and the actual tool list cannot be verified; error handling and the external agent dependency are not reproducible. Under the static calibration, reliability cannot exceed 12, so a conservative score of 4 is given.
- Security and permissions 6/20
- Evidence: HA_AGENT_KEY is marked as a secret in the manifest and the README warns not to commit mcp.json; the architecture talks to a local agent over REST on port 8099 and mentions Git versioning and rollback. Deduction: no least-privilege model or explicit confirmation for write/deploy/delete operations is shown; the README describes AI auto-deploying changes to Home Assistant, which creates remote side effects by default; HTTP transport and key-rotation details are unverified. No red-line was triggered because the examples contain no real tokens, but the score remains low at 6.
- Maintenance 8/20
- Evidence: the repository is not archived, the MIT license is declared, the version is 3.2.31 and a CHANGELOG.md is listed in the README. Deduction: no commit history, release frequency, dependency updates, issue-response or security-advisory evidence is available; maintenance activity can only be inferred from current metadata, and governance and versioning practices are unclear. Score: 8.
- Documentation 9/20
- Evidence: the README provides prerequisites, quick start, an environment-variable table, an mcpServers configuration example, troubleshooting, architecture and security notes. Deduction: tool parameters, usage limits and cost information are missing; the related-projects links are inconsistent (cursor-agent vs. vibecode-agent); and the README itself is untrusted evidence that cannot substitute for verification of actual behavior. Score: 9.
- Setup experience 12/20
- Evidence: the README covers Node.js installation, HA Vibecode Agent add-on installation, MCP configuration inside an IDE and a connection test, plus a concrete mcpServers JSON example. Deduction: setup depends on another add-on's Web UI, manual key copying and environment variables; no CI or committed tests prove the end-to-end flow. Per static calibration, setup cannot exceed 15, so a conservative score of 12 is given.