| FMRS | 54 / 100 · D | 79 / 100 · B |
|---|
| Reliability | 8 / 20 | 12 / 20 |
|---|
| Security and permissions | 7 / 20 | 18 / 20 |
|---|
| Maintenance | 12 / 20 | 16 / 20 |
|---|
| Documentation | 14 / 20 | 18 / 20 |
|---|
| Setup experience | 13 / 20 | 15 / 20 |
|---|
| Best for | - Teams with a debuggable, running Flutter/React Native/Electron/Tauri/mobile/web app who want natural-language E2E testing instead of writing test scripts
- Developers who want to drive exploratory testing directly from an MCP client like Claude Desktop, Cursor, Windsurf, or Copilot
- Scenarios where low-token UI understanding (accessibility-tree snapshots instead of screenshots) matters for cost
| - Engineering teams scaling AI coding agents while controlling spend
- Platform teams that need auditable, rollback-aware agent execution records
- Organizations using multiple agents (Claude, Codex, Gemini) under one governed flow
|
|---|
| Not for | - Teams that require a tool maintained by an official upstream vendor (e.g. Flutter, Google, Anthropic) — this is a third-party project maintained by ai-dashboad
- Restricted environments that cannot integrate a platform SDK into the app or cannot install local npm/CLI tooling
- Situations requiring the production-proven maturity and community scale of established tools like Playwright or Appium
| - Developers who just want a simple code-completion helper without run governance
- Teams unwilling to accept local JSONL run records with HMAC signing
- Environments without a supported adapter (Claude/Codex/Gemini CLIs, direct provider, or verifier-only)
|
|---|
| Required permissions | - Local execution permission to launch and run the `flutter-skill` executable over stdio
- Connection and control permissions over the app/browser instance under test (tap, type, screenshot, read widget tree, etc.)
- Access to runtime data such as app logs, console messages, and network requests
- Network port listening permission when using the HTTP `serve` mode (configurable via `--port`/`--host` or environment variables)
| - Read/write access to the configured repo root (writable scope restrictable via --allow-path/--deny-path)
- Ability to invoke the chosen agent CLI (Claude/Codex/Gemini, etc.) and its underlying model account
- Local disk writes for run records, receipts, and share outputs (share/run-receipt., etc.)
- Execution of user-configured verifier commands (e.g. npm test)
|
|---|
| Risks and side effects | - This is a third-party open-source implementation, not maintained by Flutter, Google, or Anthropic — the source material does not describe any audit, signing, or supply-chain security process
- The tools can perform arbitrary interactions (tap, type, execute JavaScript via `eval`, inject login sessions, etc.); if driven by a malicious or manipulated prompt, they could take unintended or destructive actions against the app under test
- Tools like `eval` execute arbitrary JavaScript in the page context, and `mock_api`/`intercept_requests` can alter network behavior — restrict use to trusted development/test environments
- The manifest version (0.9.36) indicates the project is still pre-1.0 and its interface may change
| - Verifier commands are executed; poorly chosen verifier commands carry security risk despite preflight policy checks
- Cost and token figures may be estimates (labeled with provenance) and should not be treated as settled accounting
- Receipt integrity must be verified before use as trustworthy evidence; missing rollback evidence is flagged as EVIDENCE_BOUNDARY
- Policy defaults come from martin.config.yaml and can be overridden by CLI flags; misconfiguration can loosen safety boundaries
|
|---|
| Supported clients | Claude Desktop, Cursor, Windsurf, VSCode Copilot, Cline, Continue, OpenClaw | Claude Code, Codex, Gemini CLI, Cursor, VS Code, GitHub Copilot, Continue |
|---|
| Tools | 100 | 0 |
|---|