- Reliability 10/20
- The README documents a coherent design: unified search/fetch tools, hybrid BM25 + semantic retrieval with RRF fusion, offline/online modes, health endpoints (/health, /status), and a list of validation commands (test:integration, test:discovery-center, test:sap-objects, etc.), suggesting key paths are intentional. However, this is a purely static review with nothing executed; although CI/sync workflows and test scripts exist in name, coverage of MCP init, tool-list handshake, and error paths cannot be verified from source. The repo also hosts two variants sharing one codebase while server.json maps to only the sap-docs variant, adding behavioral ambiguity. Consistent with the static cap and missing execution evidence, scored 10.
- Security and permissions 12/20
- The server is a public read-only documentation search endpoint requiring no API key; tools (search/fetch/abap_feature_matrix, etc.) have no destructive side effects, and the Docker docs offer `--network none` plus includeOnline=false for isolated operation, showing reasonable boundary awareness. But online sources (SAP Help/Community/Software Heroes) are enabled by default and the README does not clearly disclose the data flow or privacy implications of sending user queries to third parties; the sync workflow relies on the sensitive ABAP_REPO_SYNC_TOKEN for cross-repo pushes without stating scope or rotation. No red-line issues found (no malware, credential theft, or irreversible defaults), so not 0-4, but deducted to 12 for incomplete data-flow and credential-scope disclosure.
- Maintenance 14/20
- Apache-2.0 license is clear, version 0.3.15 is published via Release Please, indicating a versioning/release process; the README describes a one-way sync, deployment model, and PM2 runtime with reasonably clear ownership, and open issues stand at 0. However, static materials give no recent commit timeline, dependency-update policy, or security-response channel (CVE/disclosure process), so sustained maintenance cannot be substantiated. Scored 14, deducted for undisclosed security-response and dependency-update mechanisms.
- Documentation 15/20
- Documentation is well layered: Install, public hosted endpoint, variant comparison tables, offline/online source tables, tool inventory with parameters (search supports query/k/includeOnline/includeSamples/abapFlavor/sources), Docker, BTP CF, PM2, validation commands, and links to docs/ARCHITECTURE.md and HYBRID-SEARCH.md, with disclosed embedding model (~90MB) and port differences. Deductions: full JSON schemas for tool parameters, error-return formats, and rate/cost notes are missing; and the README mixes two variants' tools, risking readers attributing abap_lint etc. to the sap-docs server mapped by server.json. Scored 15.
- Setup experience 14/20
- Multiple install paths are offered: a one-click Werkbank badge, streamable HTTP client JSON config, local npm ci/setup/build/start flow, Docker build/run examples, and BTP CF deployment guidance, with default ports and health-check commands included, making the main path clear and short. Deductions: local runs require a setup phase pulling git submodules and building the FTS index plus a ~90MB embedding model, a heavier first-time prep with implicit network/disk needs; and the repo spans two variants that users must disambiguate via MCP_VARIANT. Scored 14, within the static (unexecuted) cap of 15.