M
MCP Directory CURATED DIRECTORY
911 MCP servers indexed
中文
← Back to directory
A

Anansi MCP Server

Community
Self-healing, conversational scraper for hostile sites
How to install ↓ Compare GitHub source repository ↗
Category
Other #83 of 230
Stars
★ 117 Popular
Transport
stdio (local process) · SSE
Runtime
Python 3.11+
Credentials
No credential needed
License
Apache-2.0
Last commit
Tools
17
59FMRS · C

Anansi is a powerful self-healing scraper with advanced anti-bot features, ideal for highly adversarial sites. Its MCP server enables any LLM to drive crawls, simplifying data collection for AI agents, but users must assume compliance responsibility.

Strongest · Documentation 15/20 Weakest · Reliability 10/20

Reliability
10/20
Security and permissions
12/20
Maintenance
10/20
Documentation
15/20
Setup experience
12/20
Why each score
Reliability 10/20
The source code indicates a FastMCP server exposing 17 tools, and the README claims features like self-healing selectors and browser auto-upgrade, but no verifiable tests or CI results are provided, so static review cannot confirm the actual behavior of these tools. Reliability score is 10 because the happy path seems plausible but edge cases and error handling evidence are thin.
Security and permissions 12/20
No obvious malicious behavior is found in the source code, but TLS fingerprint mimicry and anti-detection features exist, which may circumvent website access controls without permission, posing compliance risks. The README provides operator controls and disclaimers but does not show specific permission limits or confirmation workflows for dangerous operations, so security score is 12.
Maintenance 10/20
The repository has an Apache-2.0 license, recent active commits and releases, but open issues count is 0, and no explicit maintainer or security response channel is evident. Maintenance score is 10 because it is active but governance and versioning gaps exist.
Documentation 15/20
Documentation includes installation, quickstart, MCP server guide, and detailed feature explanations, but ethical and legal risks of anti-detection tools are not fully disclosed, and troubleshooting guides are missing. Documentation quality score is 15 because layered docs are decent but still have gaps.
Setup experience 12/20
The README provides pip install commands, Playwright installation steps, and client configuration examples for MCP, but no Docker or virtual environment instructions, unclear Windows support, and actual post-installation operation is not verified. Setup score is 12.

Static review · not runListed 2026-08-07

Read the FMRS scoring method →

Fit and risk

What it can accessUses the networkControls a browser

Best for

  • Developers needing robust web scraping
  • Researchers scraping data in adversarial environments
  • Teams wanting AI agents to handle scraping tasks autonomously

Not for

  • Developers needing legal, low-risk scraping
  • Users targeting only static sites without anti-bot challenges
  • Scenarios with strict legal constraints on scraping

Required permissions

  • Making network requests and scraping public data
  • Using a browser for rendering via Playwright
  • Using proxies that provide access to a proxy service
  • Writing exported files locally

Risks and side effects

  • May violate a site's Terms of Service or applicable laws (e.g., CFAA, GDPR)
  • Anti-bot features could be interpreted as circumventing access controls
  • Proxies and IPs may get blocked by target sites
  • Overly aggressive crawling may overload target servers

Setup

Before you start

Runtime：Python 3.11+

  1. Install core: pip install "git+https://github.com/mdowis/anansi"
  2. For browser-based fetching (Cloudflare bypass, JS rendering): playwright install chromium
  3. For TLS-fingerprint mimicry (curl-cffi impersonation): pip install "anansi-scraper[tls] @ git+https://github.com/mdowis/anansi"
  4. Register MCP: claude mcp add anansi -- anansi-mcp
.mcp.json 
{
  "mcpServers": {
    "anansi": {
      "command": "anansi-mcp",
      "args": []
    }
  }
}

Shown for Claude Code. Other clients may use a different file or key (VS Code uses "servers") — the configurator below converts it.

.vscode/mcp.json
{
  "servers": {
    "anansi": {
      "command": "anansi-mcp",
      "args": []
    }
  }
}

Goes in your project's .vscode/mcp.json (VS Code uses a "servers" key).

Terminal
claude mcp add anansi -- anansi-mcp

Run it in a terminal; replace any <…> placeholders with your own values first.

Generate and validate client config

Check that it works

After registering anansi-mcp, the client's tool list should show the 17 tools including fetch, crawl, and list_spiders; calling get_status should return crawler status info.

Troubleshooting

  1. CLI not working after install: ensure Python 3.11+ and dependencies installed
  2. Cloudflare bypass failing: run 'playwright install chromium'
  3. TLS fingerprint not active: install the [tls] extra
  4. MCP connection failure: ensure anansi-mcp is in PATH and check client config

Things to try

Once connected, you can ask your AI assistant things like:

  • List all registered spiders with list_spiders
  • Fetch https://shop.example.com/products and extract the titles and prices with extract
  • Run a crawl of example.com capped at 100 pages, then export the data
  • Set the rate limit to 2 requests per second and configure a proxy with set_proxy

Tools 17

fetch read-only
Fetch and process a URL with automatic browser upgrades and anti-bot identity.
extract read-only
Extract structured data from a URL using configured selectors.
crawl read-only
Crawl from one or more start URLs within a spider, following links and page rules.
screenshot read-only
Take a screenshot of a URL or a page during crawl.
run_spider writes
Run a configured spider and output the scraped data.
list_spiders read-only
List all registered spiders with their configurations.
get_status read-only
Get the current status and summary of the running spider.
pause writes
Pause the currently running spider.
Show 9 more tools
resume writes
Resume a paused spider.
set_proxy writes
Set proxy configuration for subsequent crawls.
get_stats read-only
Get crawler statistics such as request counts, cache hits, etc.
export_data writes
Export scraped data to CSV/JSON/database files.
clear_cache destructive
Clear cached entries for unchanged pages.
validate_item read-only
Validate an item against the configured schema.
configure_anti_bot writes
Adjust anti-bot settings such as vendor-level behavior and proxy scoring.
set_rate_limit writes
Set adaptive rate limiting parameters.
get_logs read-only
Retrieve crawler logs for debugging.

Use cases

Scraping sites with aggressive anti-bot measures
Crawling frequently redesigned sites without maintaining selectors
Letting AI agents autonomously perform data collection through conversation
Reducing data collection time with incremental re-crawls

Supported clients

Claude Code
Claude Desktop
Cursor
Windsurf
ChatGPT
LangChain
OpenAI Agents SDK

Listed from the project's documentation, not tested by this site.

Overview

Anansi is a self-healing web scraper for hostile sites. It automatically repairs broken CSS selectors, switches to a stealth browser when needed, and presents a coherent identity with matched TLS fingerprints, personas, and vendor-aware handling (Cloudflare, Akamai, DataDome) to slip past bot detection. Ships with an MCP server so any LLM can drive a full crawl through conversation, with incremental re-crawls and Pydantic-based data validation.

Similar servers

BioMCP 75 · B

One binary. One grammar. Evidence from the biomedical sources you already trust.

★ 654 · Tools 11 Compare with this →
Compare with other MCP servers

Source revision 117fbe27b3d6 Data synced 2026-10-12 Read the FMRS scoring method