# io.github.Grubbomatic/crawl-readiness (npm · crawl-readiness-mcp)

Audit any site for 50+ AI crawlers, generate llms.txt, robots.txt and schema, track AI mentions.

- Trust score: 70/100 (medium)
- Registry status: active
- Liveness: live
- Owner verified: no
- Last scored: 2026-09-25

## Components

- npm · `crawl-readiness-mcp`: 70/100 (this document), [markdown](https://verifymcp.io/servers/grubbomatic-crawl-readiness/crawl-readiness-mcp.md), [page](https://verifymcp.io/servers/grubbomatic-crawl-readiness/crawl-readiness-mcp)

## Channel facts

- Registry: `npm`
- Package: `crawl-readiness-mcp`
- Version: `0.2.3`
- Transport: `stdio`

## Trust breakdown

How this component scores in each security and reliability category. Every signal is checked automatically from public evidence about the published package, including repeated runs of it in an isolated sandbox, and we only credit what we can confirm. Scores are 0–100 per category. Scoring method: https://verifymcp.io/docs/scoring (what has changed: https://verifymcp.io/docs/scoring/changelog)

Scored 2026-09-25.

- **Supply Chain Security**: 98/100
  - No malware found by supply-chain analysis.
  - No known CVEs affecting this package version or its production dependencies.
  - No install/post-install scripts declared.
  - 31 of 96 dependencies flagged as unhealthy.
- **Provenance & Transparency**: 45/100
  - Source repository is publicly reachable at the declared URL.
  - Provenance check failed: no build-provenance attestation is published.
  - Clear OSI-approved license (MIT).
  - Actively maintained (last published 4 days ago).
  - Disclosure check failed: no security disclosure policy was found in the source repository.
- **Schema Quality & AI Usability**: 77/100
  - AI-judged instruction clarity (excellent).
  - Context-footprint check failed: tool/resource definitions use about 1083 tokens (~135/item across 8 items; 8 tools + 0 resources), over budget; trim descriptions and params.
  - Usage-examples check failed: none of the tools include examples.
- **Stability & Change Management**: 0/100
  - Stability not yet verified: not enough scan history yet (needs a 30-day window).
- **Tool Coverage**: 100/100
  - 100% of tools have a non-trivial description (not blank, and not just the tool's name).
  - 100% of tool parameters carry a description.
- **Tool Safety**: 100/100
  - No prompt-injection markers were found in the server instructions, tool names or descriptions we captured.
  - We read all 8 captured tool definition(s), and no name or description among them implies an irreversible operation.
  - An AI judge read all 9 captured unit(s) of tool text and found none that tries to manipulate the model reading it.
- **Capabilities**: 100/100
  - Implements a supported MCP spec version (2025-11-25); the latest is 2026-07-28.

**Unverified: 1 category.** A category scored 0 because we could not verify it: a data source with nothing on this package, evidence we could not reach, or a check we could not run. We only credit what we can confirm.

## Install

### How do I install the io.github.Grubbomatic/crawl-readiness MCP server?

io.github.Grubbomatic/crawl-readiness runs locally as an npm package, launched with npx -y crawl-readiness-mcp. Ready-made configuration for Claude, Cursor, VS Code, Codex and 5 more is on this page, copied from each client's own documentation.

### Claude

```bash
claude mcp add grubbomatic-crawl-readiness -- npx -y crawl-readiness-mcp
```

### Cursor

```json
{
  "mcpServers": {
    "grubbomatic-crawl-readiness": {
      "command": "npx",
      "args": [
        "-y",
        "crawl-readiness-mcp"
      ]
    }
  }
}
```

### VS Code

```json
{
  "servers": {
    "grubbomatic-crawl-readiness": {
      "command": "npx",
      "args": [
        "-y",
        "crawl-readiness-mcp"
      ]
    }
  }
}
```

### Codex

```bash
codex mcp add grubbomatic-crawl-readiness -- npx -y crawl-readiness-mcp
```

### opencode

```json
{
  "$schema": "https://opencode.ai/config.json",
  "mcp": {
    "grubbomatic-crawl-readiness": {
      "type": "local",
      "command": [
        "npx",
        "-y",
        "crawl-readiness-mcp"
      ],
      "enabled": true
    }
  }
}
```

### OpenClaw

```bash
openclaw mcp add grubbomatic-crawl-readiness --command npx --arg -y --arg crawl-readiness-mcp
```

### Hermes

```yaml
mcp_servers:
  grubbomatic-crawl-readiness:
    command: "npx"
    args: ["-y", "crawl-readiness-mcp"]
```

### Netclaw

```json
{
  "McpServers": {
    "grubbomatic-crawl-readiness": {
      "Transport": "stdio",
      "Command": "npx",
      "Arguments": [
        "-y",
        "crawl-readiness-mcp"
      ]
    }
  }
}
```

### Vellum

```bash
assistant mcp add grubbomatic-crawl-readiness -t stdio -c npx -a -y crawl-readiness-mcp
```

### Other

```json
{
  "mcpServers": {
    "grubbomatic-crawl-readiness": {
      "command": "npx",
      "args": [
        "-y",
        "crawl-readiness-mcp"
      ]
    }
  }
}
```

## Changelog

Every change recorded for this component, newest first. Days that predate change tracking, or that we cannot explain, say so: "we were watching and nothing happened" and "we were not watching" are different claims.

### 2026-09-25 (score 70, 0)

- [functional] We updated how we score, so this day's move reflects our rubric, not a change to the server

### 2026-09-20 (score 70)

First indexed and scored.

## MCP tools (8)

### `check_ai_readiness` (~119 tokens)

Check AI Readiness

Check whether AI crawlers (ChatGPT, Claude, Perplexity, Google AI, and 50+ others) can access a website. Returns a 0-100 AI readiness score, per-crawler access status, detected AI-specific files (llms.txt, agents.json), structured data presence, meta signals, and a prioritized fix list. Use this as the first step in any AI SEO audit.

Input parameters:

- `url` (string, required): The website URL to check (e.g. 'example.com' or 'https://example.com/page').

### `validate_schema` (~95 tokens)

Validate JSON-LD Schema

Validate all JSON-LD structured data on a URL. Extracts every <script type="application/ld+json"> block, runs each through a rules engine covering 20+ common types (Article, Organization, Product, LocalBusiness, FAQPage, Recipe, Event, etc.), and reports required-field errors, recommended-field warnings, and type-specific gotchas.

Input parameters:

- `url` (string, required): The URL to validate structured data on.

### `validate_robots` (~122 tokens)

Validate robots.txt

Audit a robots.txt file line-by-line. Detects syntax errors, empty user-agent groups, orphan Allow/Disallow lines, non-slash paths, non-numeric Crawl-delay values, unofficial Noindex usage, and wildcard traps. Also summarizes AI-bot coverage across 50+ known AI crawlers. Provide EITHER url (to fetch and audit) OR text.

Input parameters:

- `text` (string): Raw robots.txt text to audit directly (alternative to url).
- `url` (string): URL of the site whose /robots.txt should be fetched and audited.

### `check_content_parity` (~86 tokens)

Content Parity Check

Compare what human browsers see vs what AI crawlers see. Fetches a page four times in parallel — as Chrome, GPTBot, ClaudeBot, and PerplexityBot — and reports word-overlap %, title/description differences, and warnings about JS-only shells, cloaking, or edge-based bot blocking.

Input parameters:

- `url` (string, required): The URL to check parity on.

### `generate_llms_txt` (~90 tokens)

Generate llms.txt

Generate a properly formatted llms.txt file for a website. Crawls the site's sitemap, groups pages by section, pulls page titles and descriptions, and produces both the llms.txt content and a companion robots.txt snippet. The AI client can then write the returned content to disk in the user's project. Requires an API key.

Input parameters:

- `url` (string, required): The website to generate llms.txt for.

### `generate_robots_txt` (~165 tokens)

Generate robots.txt

Generate a complete, ready-to-save AI-crawler-aware robots.txt for a website. Fetches the existing robots.txt (if any) and returns the finished file in `generated.robotsTxt` — the existing rules with an AI-crawler policy section merged in — plus the per-bot allow/block breakdown. Presets: 'allow-all' (public businesses), 'search-only' (allow AI search, block training), 'recommended' (allow major AI assistants that cite sources, block training-only bots), 'block-all'. Write `generated.robotsTxt` to the site's /robots.txt. Requires an API key.

Input parameters:

- `preset` (string): The AI-crawler policy preset to apply.
- `url` (string, required): The website to generate a robots.txt for.

### `generate_schema` (~103 tokens)

Generate JSON-LD Schema

Generate JSON-LD structured data for a website. Detects the site's name, logo, social profiles, contact info, and article metadata, then produces Organization, WebSite, and (when applicable) Article schemas plus starter templates for BreadcrumbList and FAQPage. Returns each schema as a <script type="application/ld+json"> block ready to paste into the site's <head>. Requires an API key.

Input parameters:

- `url` (string, required): The website to generate schema for.

### `get_monitor_trend` (~149 tokens)

Get LLM Monitor Trend

See whether AI assistants (ChatGPT, Claude, Perplexity, Google AI) actually mention a brand in their answers, and how that share-of-voice is trending versus competitors, week over week. Call with NO argument to list the user's monitored brands with each one's current mention rate and direction; pass a brand name or project id to get that brand's full trend, per-provider breakdown, competitor comparison, average position, and short example answers. Reads data the user's LLM Monitor projects have already collected — it does not trigger new runs. Read-only. Requires an API key.

Input parameters:

- `project` (string): Brand name or project id to detail. Omit to list all monitored brands.

## Diagnostics

Captured diagnostic sections: Provenance, Dependencies. The full working is on the page: https://verifymcp.io/servers/grubbomatic-crawl-readiness/crawl-readiness-mcp#diagnostics

## Score history

- 2026-09-25: 70
- 2026-09-24: 70
- 2026-09-23: 70
- 2026-09-22: 70
- 2026-09-21: 70
- 2026-09-20: 70

## Common questions

### What is the io.github.Grubbomatic/crawl-readiness MCP server?

io.github.Grubbomatic/crawl-readiness is an MCP server listed in the public MCP registry as io.github.Grubbomatic/crawl-readiness. Audit any site for 50+ AI crawlers, generate llms.txt, robots.txt and schema, track AI mentions. This page covers its npm package (crawl-readiness-mcp).

### Is the io.github.Grubbomatic/crawl-readiness MCP server safe to use?

io.github.Grubbomatic/crawl-readiness scores 70 out of 100 on VerifyMCP. We found no known CVEs affecting it as of 25 September 2026. It declares no install or post-install scripts. That is a record of what we were able to check automatically, not an endorsement. The category breakdown on this page shows every signal behind the number, including the ones we could not confirm.

### What tools does the io.github.Grubbomatic/crawl-readiness MCP server expose?

io.github.Grubbomatic/crawl-readiness exposes 8 tools: check_ai_readiness, validate_schema, validate_robots, check_content_parity, generate_llms_txt, and 3 more. Their descriptions and schemas cost roughly 929 tokens of context every time the server is loaded.

### Is the io.github.Grubbomatic/crawl-readiness MCP server still maintained?

io.github.Grubbomatic/crawl-readiness is still listed as active in the MCP registry. We last reached this channel on 25 September 2026. Those dates come from our own scans of the registry and the channel itself, not from anything the publisher announced.

### What licence is the io.github.Grubbomatic/crawl-readiness MCP server under?

io.github.Grubbomatic/crawl-readiness declares the MIT licence, which is OSI-approved. That covers the source only, and says nothing about the cost of any service it calls.

## Links

- npm package: https://www.npmjs.com/package/crawl-readiness-mcp
- Socket report: https://socket.dev/npm/package/crawl-readiness-mcp
- Repository: https://github.com/Grubbomatic/crawl-readiness-mcp
- Website: https://www.crawlreadiness.com/mcp
- Changelog RSS feed: https://verifymcp.io/servers/grubbomatic-crawl-readiness/crawl-readiness-mcp.xml
- Changelog JSON feed: https://verifymcp.io/servers/grubbomatic-crawl-readiness/crawl-readiness-mcp.json
- HTML version of this page: https://verifymcp.io/servers/grubbomatic-crawl-readiness/crawl-readiness-mcp
