- Reliability 4/20
- The server manifest is marked "not cached," and no tool list, initialization handshake trace, or test files are present in the evidence provided. The README indicates this project is actually an agent SDK/MCP client (it connects out to external MCP servers via mcp("http://localhost:8001/mcp")) rather than an MCP server exposing its own tools, so there is no way to verify declared tools against real behavior or confirm the MCP init/tool-list handshake. A CI badge is shown but no workflow file or test content was supplied to demonstrate coverage of key paths. Per the static calibration rule, missing evidence is a deduction rather than a best-case assumption, so this scores low.
- Security and permissions 8/20
- Example code passes API keys via environment variables (e.g., OPENAI_API_KEY) with no hardcoded secrets or credential-leak evidence found. However, the README describes an LLM "automatically" selecting and invoking MCP tools and a coordinator that "autonomously delegates tasks" until done, with no mention anywhere of scoped permissions, human confirmation for consequential actions, or disclosure of what data flows to which LLM provider or MCP endpoint. These are visible but unaddressed risks — not a red line, since there is no evidence of malware, credential theft, or irreversible delete/pay/deploy defaults — but the absence of least-privilege framing or confirmation steps warrants a clear deduction.
- Maintenance 11/20
- The repo shows a CI badge, a published npm package, 401 stars, only 10 open issues, and is not archived, suggesting some activity. However, GitHub's license field reads "NOASSERTION" while the README claims Apache 2.0 with a LICENSE file — a governance inconsistency that needs clarification. No verifiable evidence of release cadence, recent commit activity, dependency-update practices, or a security-response channel (e.g., SECURITY.md) was provided, so scoring stays conservative given the limited information.
- Documentation 9/20
- The README offers install instructions, a quick-start example, and a feature overview, which is reasonably readable, but the core documentation lives on an external site (volcano.dev) that cannot be statically verified from the repository. Details on MCP tool parameters, authentication flows, rate limits, cost, and troubleshooting are not present in the provided material, leaving hidden assumptions about key usage details, which caps the score.
- Setup experience 6/20
- Installation is simple (npm install plus a few lines of example code), but this project is fundamentally a library for writing agent code that connects to other MCP servers, not an MCP server that can be configured directly into mainstream clients like Claude Desktop or Cursor — so no relevant client-configuration example exists or applies. No committed CI workflow content or tests demonstrating a verified install/run path were supplied, so per the static calibration rule setup cannot exceed 15, and it is marked down further given how little verifiable setup evidence is available.