io.github.Grubbomatic/crawl-readiness
NPM · CRAWL-READINESS-MCP · SCANNED SEP 25
Audit any site for 50+ AI crawlers, generate llms.txt, robots.txt and schema, track AI mentions.
Available components
How this component scores in each security and reliability category. Every signal is checked automatically from public evidence about the published package, including repeated runs of it in an isolated sandbox, and we only credit what we can confirm. How we score → Why this is hard to score →
Supply Chain Security98
- No malware found by supply-chain analysis.Pass
- No known CVEs affecting this package version or its production dependencies.Pass
- No install/post-install scripts declared.Pass
- 31 of 96 dependencies flagged as unhealthy. View diagnostics → Partial
Provenance & Transparency45
- Source repository is publicly reachable at the declared URL. View diagnostics → Pass
- Provenance check failed: no build-provenance attestation is published. See how to fix → View diagnostics → Fail
- Clear OSI-approved license (MIT).Pass
- Actively maintained (last published 4 days ago).Pass
- Disclosure check failed: no security disclosure policy was found in the source repository. See how to fix → Fail
Schema Quality & AI Usability77
- AI-judged instruction clarity (excellent).Pass
- Context-footprint check failed: tool/resource definitions use about 1083 tokens (~135/item across 8 items; 8 tools + 0 resources), over budget; trim descriptions and params. See how to fix → Fail
- Usage-examples check failed: none of the tools include examples. See how to fix → Fail
Stability & Change Management0
- Stability not yet verified: not enough scan history yet (needs a 30-day window).Unverified
Tool Coverage100
- 100% of tools have a non-trivial description (not blank, and not just the tool's name).Pass
- 100% of tool parameters carry a description.Pass
Tool Safety100
- No prompt-injection markers were found in the server instructions, tool names or descriptions we captured.Pass
- We read all 8 captured tool definition(s), and no name or description among them implies an irreversible operation.Pass
- An AI judge read all 9 captured unit(s) of tool text and found none that tries to manipulate the model reading it.Pass
Capabilities100
- Implements a supported MCP spec version (2025-11-25); the latest is 2026-07-28.Pass
Unverified: 1 category
A category scored 0 because we could not verify it: a data source with nothing on this package, evidence we could not reach, or a check we could not run. We only credit what we can confirm.
How do I install the io.github.Grubbomatic/crawl-readiness MCP server?
io.github.Grubbomatic/crawl-readiness runs locally as an npm package, launched with npx -y crawl-readiness-mcp. Ready-made configuration for Claude, Cursor, VS Code, Codex and 5 more is on this page, copied from each client's own documentation.
npm · crawl-readiness-mcp
claude mcp add grubbomatic-crawl-readiness -- npx -y crawl-readiness-mcp
{
"mcpServers": {
"grubbomatic-crawl-readiness": {
"command": "npx",
"args": [
"-y",
"crawl-readiness-mcp"
]
}
}
} {
"servers": {
"grubbomatic-crawl-readiness": {
"command": "npx",
"args": [
"-y",
"crawl-readiness-mcp"
]
}
}
} codex mcp add grubbomatic-crawl-readiness -- npx -y crawl-readiness-mcp
{
"$schema": "https://opencode.ai/config.json",
"mcp": {
"grubbomatic-crawl-readiness": {
"type": "local",
"command": [
"npx",
"-y",
"crawl-readiness-mcp"
],
"enabled": true
}
}
} openclaw mcp add grubbomatic-crawl-readiness --command npx --arg -y --arg crawl-readiness-mcp
mcp_servers:
grubbomatic-crawl-readiness:
command: "npx"
args: ["-y", "crawl-readiness-mcp"] {
"McpServers": {
"grubbomatic-crawl-readiness": {
"Transport": "stdio",
"Command": "npx",
"Arguments": [
"-y",
"crawl-readiness-mcp"
]
}
}
} assistant mcp add grubbomatic-crawl-readiness -t stdio -c npx -a -y crawl-readiness-mcp
{
"mcpServers": {
"grubbomatic-crawl-readiness": {
"command": "npx",
"args": [
"-y",
"crawl-readiness-mcp"
]
}
}
} Every change we have recorded for this component, newest first. Security-relevant changes are always shown. ▲ marks a change for the better, ▼ a change for the worse; unmarked changes are neutral.
- 25 Sept 26 0
- We updated how we score, so this day's move reflects our rubric, not a change to the server See what changed → functional
- 20 Sept 26 70
First indexed and scored.
Diagnostic detail from the automated scan of this channel: what the scanner observed at each step, so you can see exactly where a check passed or failed. It is informational only and never changes the trust score.
Captured 25 Sept 2026 · Analysed npm/crawl-readiness-mcp@0.2.3
Provenance No attestation
The registry publishes no build provenance for this version, so there is nothing to verify.
| Result | No attestation |
|---|---|
| Ecosystem | npm |
Background: How many MCP packages publish verified provenance →
Dependencies 96 packages
| Packages resolved | 96 |
|---|---|
| Stale | 31 |
| Tree resolution | Complete |
Background: SBOMs and build attestations, explained →
The tools this component advertises to a client, with an estimated token cost for each. Expand a tool to see its parameters and schema. The per-tool counts are indicative and are not scored directly; the schema's total context footprint is one signal in Schema Quality & AI Usability. A tool's description is untrusted text the model reads on every call, which is what makes this list a security surface and not just an inventory: how tool poisoning works →
check_ai_readiness Check AI Readiness ~119
Check whether AI crawlers (ChatGPT, Claude, Perplexity, Google AI, and 50+ others) can access a website. Returns a 0-100 AI readiness score, per-crawler access status, detected AI-specific files (llms.txt, agents.json), structured data presence, meta signals, and a prioritized fix list. Use this as the first step in any AI SEO audit.
| Name | Type | Req | Description |
|---|---|---|---|
| url | string | yes | The website URL to check (e.g. 'example.com' or 'https://example.com/page'). |
No output schema declared.
No examples provided.
check_content_parity Content Parity Check ~86
Compare what human browsers see vs what AI crawlers see. Fetches a page four times in parallel — as Chrome, GPTBot, ClaudeBot, and PerplexityBot — and reports word-overlap %, title/description differences, and warnings about JS-only shells, cloaking, or edge-based bot blocking.
| Name | Type | Req | Description |
|---|---|---|---|
| url | string | yes | The URL to check parity on. |
No output schema declared.
No examples provided.
generate_llms_txt Generate llms.txt ~90
Generate a properly formatted llms.txt file for a website. Crawls the site's sitemap, groups pages by section, pulls page titles and descriptions, and produces both the llms.txt content and a companion robots.txt snippet. The AI client can then write the returned content to disk in the user's project. Requires an API key.
| Name | Type | Req | Description |
|---|---|---|---|
| url | string | yes | The website to generate llms.txt for. |
No output schema declared.
No examples provided.
generate_robots_txt Generate robots.txt ~165
Generate a complete, ready-to-save AI-crawler-aware robots.txt for a website. Fetches the existing robots.txt (if any) and returns the finished file in `generated.robotsTxt` — the existing rules with an AI-crawler policy section merged in — plus the per-bot allow/block breakdown. Presets: 'allow-all' (public businesses), 'search-only' (allow AI search, block training), 'recommended' (allow major AI assistants that cite sources, block training-only bots), 'block-all'. Write `generated.robotsTxt` to the site's /robots.txt. Requires an API key.
| Name | Type | Req | Description |
|---|---|---|---|
| preset | string | – | The AI-crawler policy preset to apply. |
| url | string | yes | The website to generate a robots.txt for. |
No output schema declared.
No examples provided.
generate_schema Generate JSON-LD Schema ~103
Generate JSON-LD structured data for a website. Detects the site's name, logo, social profiles, contact info, and article metadata, then produces Organization, WebSite, and (when applicable) Article schemas plus starter templates for BreadcrumbList and FAQPage. Returns each schema as a <script type="application/ld+json"> block ready to paste into the site's <head>. Requires an API key.
| Name | Type | Req | Description |
|---|---|---|---|
| url | string | yes | The website to generate schema for. |
No output schema declared.
No examples provided.
get_monitor_trend Get LLM Monitor Trend ~149
See whether AI assistants (ChatGPT, Claude, Perplexity, Google AI) actually mention a brand in their answers, and how that share-of-voice is trending versus competitors, week over week. Call with NO argument to list the user's monitored brands with each one's current mention rate and direction; pass a brand name or project id to get that brand's full trend, per-provider breakdown, competitor comparison, average position, and short example answers. Reads data the user's LLM Monitor projects have already collected — it does not trigger new runs. Read-only. Requires an API key.
| Name | Type | Req | Description |
|---|---|---|---|
| project | string | – | Brand name or project id to detail. Omit to list all monitored brands. |
No output schema declared.
No examples provided.
validate_robots Validate robots.txt ~122
Audit a robots.txt file line-by-line. Detects syntax errors, empty user-agent groups, orphan Allow/Disallow lines, non-slash paths, non-numeric Crawl-delay values, unofficial Noindex usage, and wildcard traps. Also summarizes AI-bot coverage across 50+ known AI crawlers. Provide EITHER url (to fetch and audit) OR text.
| Name | Type | Req | Description |
|---|---|---|---|
| text | string | – | Raw robots.txt text to audit directly (alternative to url). |
| url | string | – | URL of the site whose /robots.txt should be fetched and audited. |
No output schema declared.
No examples provided.
validate_schema Validate JSON-LD Schema ~95
Validate all JSON-LD structured data on a URL. Extracts every <script type="application/ld+json"> block, runs each through a rules engine covering 20+ common types (Article, Organization, Product, LocalBusiness, FAQPage, Recipe, Event, etc.), and reports required-field errors, recommended-field warnings, and type-specific gotchas.
| Name | Type | Req | Description |
|---|---|---|---|
| url | string | yes | The URL to validate structured data on. |
No output schema declared.
No examples provided.
What is the io.github.Grubbomatic/crawl-readiness MCP server?
io.github.Grubbomatic/crawl-readiness is an MCP server listed in the public MCP registry as io.github.Grubbomatic/crawl-readiness. Audit any site for 50+ AI crawlers, generate llms.txt, robots.txt and schema, track AI mentions. This page covers its npm package (crawl-readiness-mcp).
Is the io.github.Grubbomatic/crawl-readiness MCP server safe to use?
io.github.Grubbomatic/crawl-readiness scores 70 out of 100 on VerifyMCP. We found no known CVEs affecting it as of 25 September 2026. It declares no install or post-install scripts. That is a record of what we were able to check automatically, not an endorsement. The category breakdown on this page shows every signal behind the number, including the ones we could not confirm.
What tools does the io.github.Grubbomatic/crawl-readiness MCP server expose?
io.github.Grubbomatic/crawl-readiness exposes 8 tools: check_ai_readiness, validate_schema, validate_robots, check_content_parity, generate_llms_txt, and 3 more. Their descriptions and schemas cost roughly 929 tokens of context every time the server is loaded.
Is the io.github.Grubbomatic/crawl-readiness MCP server still maintained?
io.github.Grubbomatic/crawl-readiness is still listed as active in the MCP registry. We last reached this channel on 25 September 2026. Those dates come from our own scans of the registry and the channel itself, not from anything the publisher announced.
What licence is the io.github.Grubbomatic/crawl-readiness MCP server under?
io.github.Grubbomatic/crawl-readiness declares the MIT licence, which is OSI-approved. That covers the source only, and says nothing about the cost of any service it calls.