# PyreCrawl (pypi · pyrecrawl)

Web scraping + research toolkit for AI agents: scrape, crawl, monitor URLs with anti-bot bypass.

- Trust score: 62/100 (medium)
- Change this week: +3
- Registry status: active
- Liveness: live
- Owner verified: no
- Last scored: 2026-09-20

## Components

- pypi · `pyrecrawl`: 62/100 (this document), [markdown](https://verifymcp.io/servers/sanggonboy-pyrecrawl/pyrecrawl.md), [page](https://verifymcp.io/servers/sanggonboy-pyrecrawl/pyrecrawl)

## Channel facts

- Registry: `pypi`
- Package: `pyrecrawl`
- Version: `0.8.0`
- Transport: `stdio`

## Trust breakdown

How this component scores in each security and reliability category. Every signal is checked automatically from public evidence about the published package, including repeated runs of it in an isolated sandbox, and we only credit what we can confirm. Scores are 0–100 per category. Scoring method: https://verifymcp.io/docs/scoring (what has changed: https://verifymcp.io/docs/scoring/changelog)

Scored 2026-09-20.

- **Supply Chain Security**: 50/100
  - Malware scan not yet available for this package.
  - A known CVE affects nltk, but no fixed version has been published, so there is nothing to upgrade to.
  - Runs hatchling.build at install time, a recognised native-build step with no shell scripting around it.
  - 8 of 114 dependencies flagged as unhealthy.
- **Provenance & Transparency**: 45/100
  - Source repository is publicly reachable at the declared URL.
  - Provenance check failed: no build-provenance attestation is published.
  - Clear OSI-approved license (MIT).
  - Actively maintained (last published 0 days ago).
  - Disclosure check failed: no security disclosure policy was found in the source repository.
- **Schema Quality & AI Usability**: 79/100
  - 100% of prompts and resources have a non-trivial description (not blank, and not just the item's name).
  - AI-judged instruction clarity (excellent).
  - Context-footprint check failed: tool/resource definitions use about 2961 tokens (~185/item across 16 items; 13 tools + 3 resources), over budget; trim descriptions and params.
  - Usage-examples check failed: none of the tools include examples.
- **Stability & Change Management**: 57/100
  - Stability observed for 17 of 30 days with no destabilising changes; credit accrues until the full window elapses.
- **Tool Coverage**: 71/100
  - 100% of tools have a non-trivial description (not blank, and not just the tool's name).
  - 0% of tool parameters carry a description.
  - Structured output schemas are declared (100% of tools); any adoption earns full credit.
- **Tool Safety**: 100/100
  - No prompt-injection markers were found in the server instructions, tool names or descriptions we captured.
  - We read all 13 captured tool definition(s), and no name or description among them implies an irreversible operation.
  - An AI judge read all 15 captured unit(s) of tool text and found none that tries to manipulate the model reading it.
- **Capabilities**: 100/100
  - Implements a supported MCP spec version (2025-11-25); the latest is 2026-07-28.

## Install

### How do I install the PyreCrawl MCP server?

PyreCrawl runs locally as a PyPI package, launched with uvx pyrecrawl. Ready-made configuration for Claude, Cursor, VS Code, Codex and 5 more is on this page, copied from each client's own documentation.

### Claude

```bash
claude mcp add sanggonboy-pyrecrawl -- uvx pyrecrawl
```

### Cursor

```json
{
  "mcpServers": {
    "sanggonboy-pyrecrawl": {
      "command": "uvx",
      "args": [
        "pyrecrawl"
      ]
    }
  }
}
```

### VS Code

```json
{
  "servers": {
    "sanggonboy-pyrecrawl": {
      "command": "uvx",
      "args": [
        "pyrecrawl"
      ]
    }
  }
}
```

### Codex

```bash
codex mcp add sanggonboy-pyrecrawl -- uvx pyrecrawl
```

### opencode

```json
{
  "$schema": "https://opencode.ai/config.json",
  "mcp": {
    "sanggonboy-pyrecrawl": {
      "type": "local",
      "command": [
        "uvx",
        "pyrecrawl"
      ],
      "enabled": true
    }
  }
}
```

### OpenClaw

```bash
openclaw mcp add sanggonboy-pyrecrawl --command uvx --arg pyrecrawl
```

### Hermes

```yaml
mcp_servers:
  sanggonboy-pyrecrawl:
    command: "uvx"
    args: ["pyrecrawl"]
```

### Netclaw

```json
{
  "McpServers": {
    "sanggonboy-pyrecrawl": {
      "Transport": "stdio",
      "Command": "uvx",
      "Arguments": [
        "pyrecrawl"
      ]
    }
  }
}
```

### Vellum

```bash
assistant mcp add sanggonboy-pyrecrawl -t stdio -c uvx -a pyrecrawl
```

### Other

```json
{
  "mcpServers": {
    "sanggonboy-pyrecrawl": {
      "command": "uvx",
      "args": [
        "pyrecrawl"
      ]
    }
  }
}
```

## Changelog

Every change recorded for this component, newest first. Days that predate change tracking, or that we cannot explain, say so: "we were watching and nothing happened" and "we were not watching" are different claims.

### 2026-09-20 (score 62, +1)

No change was recorded against any check on this day. Stability & Change Management went from 53 to 57. That category is still filling its 30-day observation window: 16 days of observed history at the previous scan, 17 at this one. The score rises as the window fills, whether or not the server changes.

### 2026-09-19 (score 61, 0)

- [functional regression] Schema quality: 126 → 185
- [functional] Schema quality: good → excellent
- [functional] Package version: 0.7.5 → 0.8.0

### 2026-09-17 (score 61, +1)

No change was recorded against any check on this day. Stability & Change Management went from 43 to 47. That category is still filling its 30-day observation window: 13 days of observed history at the previous scan, 14 at this one. The score rises as the window fills, whether or not the server changes.

### 2026-09-15 (score 60, +1)

No change was recorded against any check on this day. Stability & Change Management went from 37 to 40. That category is still filling its 30-day observation window: 11 days of observed history at the previous scan, 12 at this one. The score rises as the window fills, whether or not the server changes.

### 2026-09-13 (score 59, +1)

No change was recorded against any check on this day. Stability & Change Management went from 30 to 33. That category is still filling its 30-day observation window: 9 days of observed history at the previous scan, 10 at this one. The score rises as the window fills, whether or not the server changes.

### 2026-09-12 (score 58, −15)

- [security regression] Malware scan: pass → unverified

### 2026-09-11 (score 73, +16)

- [security regression] Stability: 0.23 → unverified
- [security regression] Tool safety: pass → unverified
- [security improvement] Malware scan: unverified → pass
- [functional regression] Capabilities: pass → unverified
- [functional regression] Tool coverage: 100 → unverified
- [functional regression] Schema quality: 100 → unverified
- [functional] Package version: 0.7.3 → 0.7.5
- [functional] Package version: 0.7.3 → 0.7.4

### 2026-09-09 (score 57, −15)

- [security regression] Malware scan: pass → unverified

## MCP tools (13)

### `scrape` (~344 tokens)

Scrape a single URL → LLM-ready markdown.

Use this when the user shares a URL and wants its content (read, analyze,
summarize, extract). Auto-escalates through fast→stealth→llm when blocked.

Args:
    url: Target URL (http/https).
    prefer: "auto" | "fast" | "stealth" | "llm".
            auto = fast first, escalate to stealth on block/short page.
            fast = cheap HTTP only (no JS).
            stealth = real Chromium + Cloudflare solver.
            llm = full Crawl4AI browser + BM25 fit-markdown.
    timeout: per-attempt timeout in seconds.
    include_html: include raw HTML in the response (large; off by default).
    js: (stealth only) JS expression evaluated against the live page
        after it settles. The value comes back in ``meta.js_result``.
        Use for data that lives in DOM *properties* (e.g. an input's
        ``.value``) rather than in serialized HTML.
    wait_for: (stealth only) JS predicate expression polled until truthy
        (bounded by ``timeout``). Use to wait for content that arrives
        asynchronously after ``network_idle``.

Returns:
    {url, final_url, status, markdown, title, method, elapsed_ms, meta}
    or {error, url, method} on failure.

Input parameters:

- `include_html` (boolean)
- `js`
- `prefer` (string)
- `timeout` (integer)
- `url` (string, required)
- `wait_for`

### `extract` (~130 tokens)

Scrape + structured extraction using a CSS-based JSON schema.

Use when the user wants structured data (tables, lists, product info)
extracted from a page. Define a CSS schema to target specific elements.

The schema is a JsonCssExtractionStrategy schema:
  { "name": "PageItems", "baseSelector": "div.item",
    "fields": [{"name": "title", "selector": "h2", "type": "text"}, ...] }

Returns parsed JSON in `data`.

Input parameters:

- `prefer` (string)
- `schema` (object, required)
- `url` (string, required)

### `map_site` (~108 tokens)

Enumerate all internal URLs reachable from `root`.

Use when the user wants to map a site's structure or find all pages
before crawling. Often paired with `crawl` or `batch_scrape`.

Args:
    root: Website root (e.g. "https://example.com/docs").
    include_pattern: Optional regex; only URLs matching are returned.
    limit: Hard cap on returned URLs.

Input parameters:

- `include_pattern`
- `limit` (integer)
- `root` (string, required)

### `crawl` (~312 tokens)

Multi-page crawl: discover URLs on `root`, then scrape each.

Use this when the user wants to crawl an entire site section or docs,
or needs multiple pages scraped in bulk. For single pages use `scrape`;
for research questions use `deep_research`.

Args:
    root: start URL.
    max_pages: hard cap on pages scraped.
    css_selector: scope each page's html/markdown to the matched element
        (non-llm: lxml re-scope of the fetched HTML; llm: native crawl4ai
        css_selector).
    prefer: "auto" | "fast" | "stealth" | "llm" (llm = Crawl4AI BFS deep-crawl).
    include_paths: regex — keep only URLs matching (matched against full URL).
    exclude_paths: regex — drop URLs matching (e.g. `/tag/|/page/\d+`).
    max_depth: 0 = flat harvest from the root page's links (default);
        >0 = true BFS up to that link depth, honoring the filters.

Returns:
    {root, pages: [{url, markdown, title, ...}], count, discovered, elapsed_ms}
    or {error, root} on failure.

Input parameters:

- `css_selector`
- `exclude_paths`
- `include_paths`
- `max_depth` (integer)
- `max_pages` (integer)
- `prefer` (string)
- `root` (string, required)

### `document` (~127 tokens)

Extract text from a PDF/DOCX/PPTX URL → markdown (no browser).

Use when the user shares a link to a document (PDF, Word, PowerPoint)
and wants its text content. Also useful after `search_papers` to get
full text from a paper's pdf_url.

Content-type sniffed and routed to pypdf / python-docx / python-pptx.
Optional deps — install with `pip install 'pyrecrawl[docs]'`.

Input parameters:

- `max_pages` (integer)
- `timeout` (integer)
- `url` (string, required)

### `search` (~143 tokens)

Web search via DuckDuckGo HTML (no API key required).

Use this for targeted searches where you need anti-bot bypass (Cloudflare
protection on DDG). For simple searches, the built-in web_search may suffice.
For research questions, prefer `deep_research` (search + scrape + citations).

Returns [{url, title, snippet}, ...]. The smart ladder bypasses
DDG's bot detection if needed.

Returns:
    {query, results: [{url, title, snippet}], count}
    or {error, query} on failure.

Input parameters:

- `limit` (integer)
- `prefer` (string)
- `query` (string, required)

### `batch_scrape` (~215 tokens)

Scrape MANY URLs in ONE call (parallel, deduped, cache-aware).

Use when the user provides multiple URLs or you have a list of pages
to fetch. More efficient than calling `scrape` N times.

Args:
    urls: Target URLs (deduped automatically; empties dropped).
    prefer: "auto" | "fast" | "stealth" | "llm".
    timeout: per-URL timeout in seconds.
    max_concurrency: parallel workers (default 4).
    include_html: include raw HTML per result (large; off by default).

Returns {requested, unique, succeeded, failed, results[]}.
Per-URL failures are isolated — other URLs still succeed.

Returns:
    {requested, unique, succeeded, failed, results: [{url, markdown, ...}]}

Input parameters:

- `include_html` (boolean)
- `max_concurrency` (integer)
- `prefer` (string)
- `timeout` (integer)
- `urls` (array, required)

### `deep_research` (~334 tokens)

Search the web, then pull the top sources as EVIDENCE (no LLM synthesis).

PRIMARY RESEARCH TOOL — use when the user asks to research, investigate,
deep-dive, fact-check, or learn about a topic. Returns a ``citations``
list with stable [n] numbers and an ``evidence`` list of per-source
markdown — the agent does the synthesis from evidence.

Multi-pass mode: set iterations=2-3 to auto-run additional searches with
refined queries (alternatives, criticism, latest developments) and append
deduplicated evidence. Each pass adds up to ``scrape_top`` new sources.

Args:
    query: search string.
    limit: how many search results to fetch.
    scrape_top: how many of those to actually fetch content from.
    prefer: "auto" | "fast" | "stealth" | "llm".
    iterations: 1 (default, single pass), 2-3 (multi-pass with refined
        queries targeting evidence gaps). Each pass searches from a
        different angle and deduplicates by URL.

Returns:
    {query, iterations_run, queries: [str], hits: [{url, title, snippet}],
     citations: [{url, title}], evidence: [{url, title, markdown}],
     scraped, used_engines, elapsed_ms}
    or {error, query, hint} on failure.

Input parameters:

- `iterations` (integer)
- `limit` (integer)
- `prefer` (string)
- `query` (string, required)
- `scrape_top` (integer)

### `monitor` (~241 tokens)

Track a URL over time and report meaningful content changes.

Use when the user wants to watch a page for updates (price changes,
new blog posts, status updates). Ask me to 'set up monitoring for <url>'
for a guided setup playbook.

Args:
    url: target URL.
    action: "check" | "history" | "forget".
    prefer: ladder preference, same as ``scrape``.
    css_selector: scope the diff to one element (so banner /
        nav changes don't trigger false positives).

Snapshots persist under ``PYRECRAWL_MONITOR_DIR`` (default
\``~/.pyrecrawl/monitors/``). ``check`` returns ``status`` of
\``new`` | ``unchanged`` | ``changed`` | ``error`` and a unified
diff when the page changed.

Returns:
    {url, status: "new"|"unchanged"|"changed"|"error",
     diff?: str, snapshot_chars?: int, elapsed_ms?: int}

Input parameters:

- `action` (string)
- `css_selector`
- `prefer` (string)
- `url` (string, required)

### `search_papers` (~207 tokens)

Search academic papers via arXiv or Crossref — no API keys.

Args:
    query: free-text search, e.g. "transformer attention scaling laws".
    limit: max results (1-25 arXiv / 1-20 crossref).
    source: "arxiv" (CS/physics/math preprints, default) or
        "crossref" (all fields, DOI-backed).
    category: optional arXiv category filter, e.g. "cs.LG", "cs.CV".

Returns papers with id/url/pdf_url/title/authors/summary/published.
Feed pdf_url into the `document` tool to extract full text.

Returns:
    {query, source, papers: [{title, authors, abstract, url, pdf_url?, ...}]}
    or {error, query} on failure.

Input parameters:

- `category`
- `limit` (integer)
- `query` (string, required)
- `source` (string)

### `cache` (~77 tokens)

Inspect the response cache: stats, clear, enable, or disable.

Use this to check cache hit rates before large batch jobs, or to
clear stale cached responses when a site's content has changed.

Args:
    action: "stats" (default) | "clear" | "disable" | "enable".

Input parameters:

- `action` (string)

### `health` (~49 tokens)

Verify PyreCrawl is working: engine versions, dependencies, update status.

Use this at the start of a session or before a large scraping job to
confirm all engines are installed and up to date.

### `session` (~344 tokens)

Drive a persistent browser session — cookies & JS state kept across calls.

Use for login walls and multi-step flows the one-shot ladder can't handle
(e.g. user needs to log in first, then scrape a protected page).
For simple pages use `scrape`; for research use `deep_research`.

One-shot scrape has no session memory; here each action runs
against the same live page.

Args:
    session: named session; reuse the same name to keep state.
    action:
      "open" (url)        — navigate, returns url/title/status
      "click" (selector)  — click an element
      "fill" (selector, text) — type into an input
      "type" (key)        — press a key, e.g. "Enter"
      "eval" (js)         — run a JS expression, returns value
      "wait" (selector?)  — wait for selector or sleep timeout_ms
      "content"           — url/title/visible text of current page
      "screenshot" (full_page?) — returns png_base64
      "cookies"           — list session cookies
      "close"             — destroy the session
      "list"              — show live sessions
Returns {session, action, ...result, elapsed_ms}; errors as {error}.

Input parameters:

- `action` (string)
- `full_page` (boolean)
- `headless` (boolean)
- `js`
- `key`
- `selector`
- `session` (string)
- `text`
- `timeout_ms` (integer)
- `url`

## Diagnostics

Captured diagnostic sections: Provenance, Install scripts, Vulnerabilities, Dependencies. The full working is on the page: https://verifymcp.io/servers/sanggonboy-pyrecrawl/pyrecrawl#diagnostics

## Score history

- 2026-09-20: 62
- 2026-09-19: 61
- 2026-09-18: 61
- 2026-09-17: 61
- 2026-09-16: 60
- 2026-09-15: 60
- 2026-09-14: 59
- 2026-09-13: 59
- 2026-09-12: 58
- 2026-09-11: 73
- 2026-09-10: 57
- 2026-09-09: 57
- 2026-09-08: 72
- 2026-09-07: 55
- 2026-09-06: 70
- 2026-09-05: 69
- 2026-09-04: 54
- 2026-09-03: 53

## Common questions

### What is the PyreCrawl MCP server?

PyreCrawl is an MCP server listed in the public MCP registry as io.github.SanggonBoy/PyreCrawl. Web scraping + research toolkit for AI agents: scrape, crawl, monitor URLs with anti-bot bypass. This page covers its PyPI package (pyrecrawl).

### Is the PyreCrawl MCP server safe to use?

PyreCrawl scores 62 out of 100 on VerifyMCP. We recorded 1 known advisory against it as of 20 September 2026. That is a record of what we were able to check automatically, not an endorsement. The category breakdown on this page shows every signal behind the number, including the ones we could not confirm.

### What tools does the PyreCrawl MCP server expose?

PyreCrawl exposes 13 tools: scrape, extract, map_site, crawl, document, and 8 more. Their descriptions and schemas cost roughly 2,631 tokens of context every time the server is loaded.

### Is the PyreCrawl MCP server still maintained?

PyreCrawl is still listed as active in the MCP registry. We last reached this channel on 20 September 2026. Those dates come from our own scans of the registry and the channel itself, not from anything the publisher announced.

### What licence is the PyreCrawl MCP server under?

PyreCrawl declares the MIT licence, which is OSI-approved. That covers the source only, and says nothing about the cost of any service it calls.

## Links

- PyPI project: https://pypi.org/project/pyrecrawl/
- Socket report: https://socket.dev/pypi/package/pyrecrawl
- Repository: https://github.com/SanggonBoy/PyreCrawl
- Website: https://pypi.org/project/pyrecrawl/
- Changelog RSS feed: https://verifymcp.io/servers/sanggonboy-pyrecrawl/pyrecrawl.xml
- Changelog JSON feed: https://verifymcp.io/servers/sanggonboy-pyrecrawl/pyrecrawl.json
- HTML version of this page: https://verifymcp.io/servers/sanggonboy-pyrecrawl/pyrecrawl