- Reliability 2/20
- The repository is a Nuxt 3 chatbot platform that acts as an MCP client, not an MCP server: the manifest block shows "[not cached]", the README never claims this project exposes an MCP server endpoint (no stdio/SSE/HTTP server entry, no tool list, no handshake description), and all MCP sections explain how to connect external MCP servers. With no server code, tests, or CI evidence in the supplied material, the init + tool-list handshake cannot be verified at all; the static calibration caps this dimension at 12 even with evidence, and here there is essentially none. The recent SQLite→PostgreSQL migration also indicates churn in core storage. Deducted for: not being an MCP server, no tests/CI evidence, no verifiable handshake.
- Security and permissions 6/20
- The project manages many commercial-model API keys via .env and offers an ACL for MCP management, which is positive. However, the default (and even the "recommended production" line) is ACL_ENABLED=false; per the permission table, unauthenticated and regular users get full MCP management, including adding STDIO servers that execute arbitrary commands — effectively an anonymous, arbitrary-command launch surface on default deployments, a clear overreach. Credential storage details, least-privilege design, and a security-response channel are not documented. No malware, credential theft, or exfiltration evidence was found, so no red-line trigger, but the default open command-configuration surface is a major deduction.
- Maintenance 12/20
- Recent feature announcements (2025-08-25 ACL, 2025-08-19 Deep Agents, 2025-08-14 PostgreSQL migration) indicate an active project with frequent commits; the repository is not archived and the README points to a Discord community. Deductions: the metadata license is NOASSERTION despite the README claiming MIT, and the LICENSE content cannot be verified from the supplied material; 245 open issues show no visible triage or response evidence; no dependency-update policy or security-response channel is documented.
- Documentation 10/20
- The README is extensive for the chatbot platform: quick start, Docker and dev setup, environment variables, feature flags, MCP configuration, MCP permission troubleshooting, and user/role management, with well-organized MCP sections. However, documentation of this project as an MCP server is entirely absent (no tool schemas, parameters, or limits), and there are internal contradictions: the env example still shows DATABASE_URL=file:../../chatollama.sqlite while the notice says PostgreSQL is now primary, and the "Data Storage" section still references SQLite paths after announcing the migration. The docs only describe consuming other MCP servers and never disclose the key limitation that this app itself exposes no MCP endpoint to clients.
- Setup experience 8/20
- The Docker quick start is a single `docker compose up` after downloading docker-compose.yaml, and MCP servers are configured through a documented Settings UI supporting STDIO, SSE, and Streamable HTTP with a uvx example. Deductions: there is no client-connectable endpoint example because this is not an MCP server; development mode requires many external services (Node 18+/pnpm, PostgreSQL, Ollama, ChromaDB/Milvus, Redis); and there is no CI or test evidence that the compose stack actually starts. The static calibration caps this dimension at 15 without execution evidence; the large dependency count and contradictory documentation lower it further.