# Commoncrawl (npm · @pipeworx/mcp-commoncrawl)

Common Crawl's public web archive — find every time a URL was crawled since 2008 and read the…

- Trust score: 90/100 (high trust)
- Change this week: +3
- Registry status: active
- Liveness: live
- Owner verified: no
- Last scored: 2026-11-22

## Components

- remote · `gateway.pipeworx.io`: 88/100, [markdown](https://verifymcp.io/servers/pipeworx-io-commoncrawl/commoncrawl-mcp.md), [page](https://verifymcp.io/servers/pipeworx-io-commoncrawl/commoncrawl-mcp)
- npm · `@pipeworx/mcp-commoncrawl`: 90/100 (this document), [markdown](https://verifymcp.io/servers/pipeworx-io-commoncrawl/pipeworx-mcp-commoncrawl.md), [page](https://verifymcp.io/servers/pipeworx-io-commoncrawl/pipeworx-mcp-commoncrawl)

## Channel facts

- Registry: `npm`
- Package: `@pipeworx/mcp-commoncrawl`
- Version: `0.2.1`
- Transport: `stdio`

## Trust breakdown

How this component scores in each security and reliability category. Every signal is checked automatically from public evidence about the published package, including repeated runs of it in an isolated sandbox, and we only credit what we can confirm. Scores are 0–100 per category. Scoring method: https://verifymcp.io/docs/scoring (what has changed: https://verifymcp.io/docs/scoring/changelog)

Scored 2026-11-22.

- **Supply Chain Security**: 100/100
  - No malware found by supply-chain analysis.
  - No known CVEs affecting this package version or its production dependencies.
  - No install/post-install scripts declared.
  - No production dependencies, so there is no dependency health to assess.
- **Provenance & Transparency**: 97/100
  - Source repository is publicly reachable at the declared URL.
  - Cryptographically verified build provenance (signed, bound to pipeworx-io/mcp-commoncrawl).
  - Clear OSI-approved license (MIT).
  - Actively maintained (last published 21 days ago).
  - Disclosure check failed: no security disclosure policy was found in the source repository.
- **Schema Quality & AI Usability**: 64/100
  - AI-judged instruction clarity (excellent).
  - Context-footprint check failed: tool/resource definitions use about 859 tokens (~286/item across 3 items; 3 tools + 0 resources), over budget; trim descriptions and params.
  - Usage-examples check failed: none of the tools include examples.
- **Stability & Change Management**: 80/100
  - Stability observed for 24 of 30 days with no destabilising changes; credit accrues until the full window elapses.
- **Tool Coverage**: 100/100
  - 100% of tools have a non-trivial description (not blank, and not just the tool's name).
  - 100% of tool parameters carry a description.
- **Tool Safety**: 100/100
  - No prompt-injection markers were found in the server instructions, tool names or descriptions we captured.
  - We read all 3 captured tool definition(s), and no name or description among them implies an irreversible operation.
  - An AI judge read all 4 captured unit(s) of tool text and found none that tries to manipulate the model reading it.
- **Capabilities**: 100/100
  - Implements a supported MCP spec version (2025-11-25); the latest is 2026-07-28.

## Install

### How do I install the Commoncrawl MCP server?

Commoncrawl runs locally as an npm package, launched with npx -y @pipeworx/mcp-commoncrawl. Ready-made configuration for Claude, Cursor, VS Code, Codex and 5 more is on this page, copied from each client's own documentation.

### Claude

```bash
claude mcp add pipeworx-io-commoncrawl -- npx -y @pipeworx/mcp-commoncrawl
```

### Cursor

```json
{
  "mcpServers": {
    "pipeworx-io-commoncrawl": {
      "command": "npx",
      "args": [
        "-y",
        "@pipeworx/mcp-commoncrawl"
      ]
    }
  }
}
```

### VS Code

```json
{
  "servers": {
    "pipeworx-io-commoncrawl": {
      "command": "npx",
      "args": [
        "-y",
        "@pipeworx/mcp-commoncrawl"
      ]
    }
  }
}
```

### Codex

```bash
codex mcp add pipeworx-io-commoncrawl -- npx -y @pipeworx/mcp-commoncrawl
```

### opencode

```json
{
  "$schema": "https://opencode.ai/config.json",
  "mcp": {
    "pipeworx-io-commoncrawl": {
      "type": "local",
      "command": [
        "npx",
        "-y",
        "@pipeworx/mcp-commoncrawl"
      ],
      "enabled": true
    }
  }
}
```

### OpenClaw

```bash
openclaw mcp add pipeworx-io-commoncrawl --command npx --arg -y --arg @pipeworx/mcp-commoncrawl
```

### Hermes

```yaml
mcp_servers:
  pipeworx-io-commoncrawl:
    command: "npx"
    args: ["-y", "@pipeworx/mcp-commoncrawl"]
```

### Netclaw

```json
{
  "McpServers": {
    "pipeworx-io-commoncrawl": {
      "Transport": "stdio",
      "Command": "npx",
      "Arguments": [
        "-y",
        "@pipeworx/mcp-commoncrawl"
      ]
    }
  }
}
```

### Vellum

```bash
assistant mcp add pipeworx-io-commoncrawl -t stdio -c npx -a -y @pipeworx/mcp-commoncrawl
```

### Other

```json
{
  "mcpServers": {
    "pipeworx-io-commoncrawl": {
      "command": "npx",
      "args": [
        "-y",
        "@pipeworx/mcp-commoncrawl"
      ]
    }
  }
}
```

## Changelog

Every change recorded for this component, newest first. Days that predate change tracking, or that we cannot explain, say so: "we were watching and nothing happened" and "we were not watching" are different claims.

### 2026-11-21 (score 90, +1)

No change was recorded against any check on this day. Stability & Change Management went from 73 to 77. That category is still filling its 30-day observation window: 22 days of observed history at the previous scan, 23 at this one. The score rises as the window fills, whether or not the server changes.

### 2026-11-19 (score 89, +1)

No change was recorded against any check on this day. Stability & Change Management went from 67 to 70. That category is still filling its 30-day observation window: 20 days of observed history at the previous scan, 21 at this one. The score rises as the window fills, whether or not the server changes.

### 2026-11-17 (score 88, +1)

No change was recorded against any check on this day. Stability & Change Management went from 60 to 63. That category is still filling its 30-day observation window: 18 days of observed history at the previous scan, 19 at this one. The score rises as the window fills, whether or not the server changes.

### 2026-11-15 (score 87, +1)

No change was recorded against any check on this day. Stability & Change Management went from 53 to 57. That category is still filling its 30-day observation window: 16 days of observed history at the previous scan, 17 at this one. The score rises as the window fills, whether or not the server changes.

### 2026-11-13 (score 86, 0)

- [functional regression] Security disclosure: unverified → fail

### 2026-11-12 (score 86, +1)

- [functional regression] Security disclosure: fail → unverified

### 2026-11-10 (score 85, +1)

No change was recorded against any check on this day. Stability & Change Management went from 37 to 40. That category is still filling its 30-day observation window: 11 days of observed history at the previous scan, 12 at this one. The score rises as the window fills, whether or not the server changes.

### 2026-11-08 (score 84, +1)

No change was recorded against any check on this day. Stability & Change Management went from 30 to 33. That category is still filling its 30-day observation window: 9 days of observed history at the previous scan, 10 at this one. The score rises as the window fills, whether or not the server changes.

## MCP tools (3)

### `commoncrawl_crawls` (~157 tokens)

List the Common Crawl monthly crawl collections (crawl ids like "CC-MAIN-2026-34") with the date range each one covers. AUTHORITATIVE for "which Common Crawl snapshot covers <date>" — call this first to pick the crawl id that commoncrawl_index_search needs. Newest first. Keyless.

Input parameters:

- `contains_date` (string): Optional ISO date, e.g. "2025-01-15". Returns only the crawl(s) whose capture window covers that date — the answer to "which snapshot would have seen my page then".
- `limit` (number): How many crawls to return, newest first. Default 20, max 120 (the full history back to 2008).

### `commoncrawl_index_search` (~386 tokens)

Search one Common Crawl collection's CDX index for archived captures of a URL, host, or whole domain. PREFER OVER WEB SEARCH when the question is "what did this page look like in <month>", "did Common Crawl ever see this URL", or "list the URLs crawled under this domain" — it returns the crawl record (timestamp, HTTP status, MIME type, detected language, content digest) plus the WARC filename/offset/length that commoncrawl_fetch_record needs to read the page bytes. Keyless.

Input parameters:

- `crawl` (string): Crawl collection id from commoncrawl_crawls, e.g. "CC-MAIN-2026-34". Defaults to the most recent crawl.
- `filter` (string): Optional CDX field filter, e.g. "=status:200" (only successful captures), "=mime:text/html", "~url:.*blog.*". Repeatable filters are not supported here — pass one.
- `from` (string): Optional lower bound on capture timestamp, as YYYY, YYYYMM, or YYYYMMDD, e.g. "202501".
- `limit` (number): Maximum captures to return. Default 20, max 200.
- `match_type` (string): How to match `url`: "exact" (that URL only), "prefix" (that path and everything under it), "host" (every URL on that exact host), "domain" (that host and all subdomains). Default "exact".
- `page` (number): Zero-based page number for paging through a large domain query. Default 0.
- `to` (string): Optional upper bound on capture timestamp, same format as `from`.
- `url` (string, required): URL, host, or domain to look up, e.g. "example.com", "https://www.nasa.gov/news", "*.python.org".

### `commoncrawl_fetch_record` (~264 tokens)

Read one archived page back out of Common Crawl by byte range — pass the filename, offset and length from a commoncrawl_index_search result and get the WARC headers, the captured HTTP response headers, and the page body as it was crawled. AUTHORITATIVE for "what did this page actually say when it was crawled", including pages that have since changed or gone offline. The body is truncated to a byte cap you control. Keyless.

Input parameters:

- `filename` (string, required): WARC path from a capture row, e.g. "crawl-data/CC-MAIN-2025-05/segments/1736703361969.6/warc/CC-MAIN-20250112194358-20250112224358-00433.warc.gz".
- `length` (number, required): Compressed byte length of the record (the capture row's "length"). Max 10485760 (10 MB).
- `max_body_bytes` (number): Truncate the decoded page body to this many bytes before returning it. Default 20000, max 200000. WARC headers and HTTP headers are always returned in full.
- `offset` (number, required): Byte offset of the record within that WARC file (the capture row's "offset").

## Diagnostics

Captured diagnostic sections: Provenance, Dependencies. The full working is on the page: https://verifymcp.io/servers/pipeworx-io-commoncrawl/pipeworx-mcp-commoncrawl#diagnostics

## Score history

- 2026-11-22: 90
- 2026-11-21: 90
- 2026-11-20: 89
- 2026-11-19: 89
- 2026-11-18: 88
- 2026-11-17: 88
- 2026-11-16: 87
- 2026-11-15: 87
- 2026-11-14: 86
- 2026-11-13: 86
- 2026-11-12: 86
- 2026-11-11: 85
- 2026-11-10: 85
- 2026-11-09: 84
- 2026-11-08: 84
- 2026-11-07: 83
- 2026-11-06: 83
- 2026-11-05: 82
- 2026-11-04: 82
- 2026-11-03: 81
- 2026-11-02: 81
- 2026-11-01: 80
- 2026-10-31: 80
- 2026-10-30: 79
- 2026-10-29: 79

## Common questions

### What is the Commoncrawl MCP server?

Commoncrawl is an MCP server listed in the public MCP registry as io.github.pipeworx-io/commoncrawl. Common Crawl's public web archive, find every time a URL was crawled since 2008 and read the… This page covers its npm package (@pipeworx/mcp-commoncrawl).

### Is the Commoncrawl MCP server safe to use?

Commoncrawl scores 90 out of 100 on VerifyMCP. We found no known CVEs affecting it as of 22 November 2026. It declares no install or post-install scripts. Its build provenance is signed and verified. That is a record of what we were able to check automatically, not an endorsement. The category breakdown on this page shows every signal behind the number, including the ones we could not confirm.

### What tools does the Commoncrawl MCP server expose?

Commoncrawl exposes 3 tools: commoncrawl_crawls, commoncrawl_index_search, commoncrawl_fetch_record. Their descriptions and schemas cost roughly 807 tokens of context every time the server is loaded.

### Is the Commoncrawl MCP server still maintained?

Commoncrawl is still listed as active in the MCP registry. We last reached this channel on 22 November 2026. Those dates come from our own scans of the registry and the channel itself, not from anything the publisher announced.

### What licence is the Commoncrawl MCP server under?

Commoncrawl declares the MIT licence, which is OSI-approved. That covers the source only, and says nothing about the cost of any service it calls.

### What did VerifyMCP test on the Commoncrawl MCP server?

VerifyMCP ran 16 checks on Commoncrawl, covering Malware scan, Known CVEs, Install scripts, Dependency health and 12 more. Endpoint Security and Transport & Reachability do not apply to an npm package, so they were not tested. The score does not measure everything, and the scoring documentation lists what it leaves out.

## Links

- npm package: https://www.npmjs.com/package/@pipeworx/mcp-commoncrawl
- Socket report: https://socket.dev/npm/package/@pipeworx/mcp-commoncrawl
- Repository: https://github.com/pipeworx-io/mcp-commoncrawl
- Website: https://pipeworx.io/packs/commoncrawl
- Changelog RSS feed: https://verifymcp.io/servers/pipeworx-io-commoncrawl/pipeworx-mcp-commoncrawl.xml
- Changelog JSON feed: https://verifymcp.io/servers/pipeworx-io-commoncrawl/pipeworx-mcp-commoncrawl/changelog.json
- Trust report (JSON): https://verifymcp.io/servers/pipeworx-io-commoncrawl/pipeworx-mcp-commoncrawl.json
- HTML version of this page: https://verifymcp.io/servers/pipeworx-io-commoncrawl/pipeworx-mcp-commoncrawl
