# Robots.txt Policy Lint (remote · www.pathwren.workers.dev)

{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"whoami","arguments":{}}} free no key

- Trust score: 82/100 (high trust)
- Change this week: +3
- Registry status: active
- Liveness: live
- Owner verified: no
- Last scored: 2026-09-21

## Components

- remote · `www.pathwren.workers.dev`: 82/100 (this document), [markdown](https://verifymcp.io/servers/dev-workers-pathwren-www-robots-policy-lint/c-mcp-registry-official-mcp-robots.md), [page](https://verifymcp.io/servers/dev-workers-pathwren-www-robots-policy-lint/c-mcp-registry-official-mcp-robots)

## Channel facts

- Endpoint: `https://www.pathwren.workers.dev/c/mcp-registry-official/mcp/robots`
- Transports: `streamable-http`
- Auth: `none`
- Version: `1.4.0`

## Trust breakdown

How this component scores in each security and reliability category. Every signal is checked automatically against the live server, and we only credit what we can confirm. Scores are 0–100 per category. Scoring method: https://verifymcp.io/docs/scoring (what has changed: https://verifymcp.io/docs/scoring/changelog)

Scored 2026-09-21.

- **Endpoint Security**: 80/100
  - The endpoint's TLS certificate is valid, in date, and uses a strong key.
  - No authorisation is required to call this server. Every tool declares its destructiveHint and none is destructive, so open access doesn't expose one.
  - HTTPS is enforced; there's no plaintext access path.
  - The HSTS (Strict-Transport-Security) header is present.
  - DNSSEC check failed: this domain isn't protected by DNSSEC.
- **Transport & Reachability**: 100/100
  - Verified streamable-http transport via a live MCP handshake.
- **Schema Quality & AI Usability**: 78/100
  - 100% of prompts and resources have a non-trivial description (not blank, and not just the item's name).
  - AI-judged instruction clarity (good).
  - Context-footprint check failed: tool/resource definitions use about 4291 tokens (~357/item across 12 items; 9 tools + 3 resources), over budget; trim descriptions and params.
  - Tools include usage examples.
- **Stability & Change Management**: 67/100
  - Stability observed for 20 of 30 days with no destabilising changes; credit accrues until the full window elapses.
- **Tool Coverage**: 100/100
  - 100% of tools have a non-trivial description (not blank, and not just the tool's name).
  - 100% of tool parameters carry a description.
  - Structured output schemas are declared (44% of tools); any adoption earns full credit.
- **Tool Safety**: 75/100
  - No prompt-injection markers were found in the server instructions, tool names or descriptions we captured.
  - 0 of 1 tool(s) whose name or description implies an irreversible operation declare an MCP destructiveHint annotation; "merge_policy" implies "merge" and declares readOnlyHint instead, contradicting what its own name says it does.
  - An AI judge read all 11 captured unit(s) of tool text and found none that tries to manipulate the model reading it.
- **Capabilities**: 100/100
  - Implements a current MCP spec version (2026-07-28).

## Install

### How do I install the Robots.txt Policy Lint MCP server?

Robots.txt Policy Lint is a hosted endpoint at https://www.pathwren.workers.dev/c/mcp-registry-official/mcp/robots, so there is nothing to install locally. Ready-made configuration for Claude, Cursor, VS Code, Codex and 5 more is on this page, copied from each client's own documentation.

### Claude

```bash
claude mcp add --transport http dev-workers-pathwren-www-robots-policy-lint 'https://www.pathwren.workers.dev/c/mcp-registry-official/mcp/robots'
```

### Cursor

```json
{
  "mcpServers": {
    "dev-workers-pathwren-www-robots-policy-lint": {
      "url": "https://www.pathwren.workers.dev/c/mcp-registry-official/mcp/robots"
    }
  }
}
```

### VS Code

```json
{
  "servers": {
    "dev-workers-pathwren-www-robots-policy-lint": {
      "type": "http",
      "url": "https://www.pathwren.workers.dev/c/mcp-registry-official/mcp/robots"
    }
  }
}
```

### Codex

```toml
[mcp_servers.dev-workers-pathwren-www-robots-policy-lint]
url = "https://www.pathwren.workers.dev/c/mcp-registry-official/mcp/robots"
```

### opencode

```json
{
  "$schema": "https://opencode.ai/config.json",
  "mcp": {
    "dev-workers-pathwren-www-robots-policy-lint": {
      "type": "remote",
      "url": "https://www.pathwren.workers.dev/c/mcp-registry-official/mcp/robots",
      "enabled": true
    }
  }
}
```

### OpenClaw

```bash
openclaw mcp add dev-workers-pathwren-www-robots-policy-lint --url 'https://www.pathwren.workers.dev/c/mcp-registry-official/mcp/robots' --transport streamable-http
```

### Hermes

```yaml
mcp_servers:
  dev-workers-pathwren-www-robots-policy-lint:
    url: "https://www.pathwren.workers.dev/c/mcp-registry-official/mcp/robots"
```

### Netclaw

```json
{
  "McpServers": {
    "dev-workers-pathwren-www-robots-policy-lint": {
      "Transport": "http",
      "Url": "https://www.pathwren.workers.dev/c/mcp-registry-official/mcp/robots"
    }
  }
}
```

### Vellum

```bash
assistant mcp add dev-workers-pathwren-www-robots-policy-lint -t streamable-http -u 'https://www.pathwren.workers.dev/c/mcp-registry-official/mcp/robots'
```

### Other

```json
{
  "mcpServers": {
    "dev-workers-pathwren-www-robots-policy-lint": {
      "type": "http",
      "url": "https://www.pathwren.workers.dev/c/mcp-registry-official/mcp/robots"
    }
  }
}
```

The mcpServers block is a cross-client convention. Remote transports vary, so check your client's docs.

## Changelog

Every change recorded for this component, newest first. Days that predate change tracking, or that we cannot explain, say so: "we were watching and nothing happened" and "we were not watching" are different claims.

### 2026-09-21 (score 82, +1)

No change was recorded against any check on this day. Stability & Change Management went from 63 to 67. That category is still filling its 30-day observation window: 19 days of observed history at the previous scan, 20 at this one. The score rises as the window fills, whether or not the server changes.

### 2026-09-18 (score 81, +1)

No change was recorded against any check on this day. Stability & Change Management went from 53 to 57. That category is still filling its 30-day observation window: 16 days of observed history at the previous scan, 17 at this one. The score rises as the window fills, whether or not the server changes.

### 2026-09-16 (score 80, +1)

No change was recorded against any check on this day. Stability & Change Management went from 47 to 50. That category is still filling its 30-day observation window: 14 days of observed history at the previous scan, 15 at this one. The score rises as the window fills, whether or not the server changes.

### 2026-09-14 (score 79, +1)

No change was recorded against any check on this day. Stability & Change Management went from 40 to 43. That category is still filling its 30-day observation window: 12 days of observed history at the previous scan, 13 at this one. The score rises as the window fills, whether or not the server changes.

### 2026-09-12 (score 78, +1)

No change was recorded against any check on this day. Stability & Change Management went from 33 to 37. That category is still filling its 30-day observation window: 10 days of observed history at the previous scan, 11 at this one. The score rises as the window fills, whether or not the server changes.

### 2026-09-10 (score 77, +1)

No change was recorded against any check on this day. Stability & Change Management went from 27 to 30. That category is still filling its 30-day observation window: 8 days of observed history at the previous scan, 9 at this one. The score rises as the window fills, whether or not the server changes.

### 2026-09-08 (score 76, +1)

No change was recorded against any check on this day. Stability & Change Management went from 20 to 23. That category is still filling its 30-day observation window: 6 days of observed history at the previous scan, 7 at this one. The score rises as the window fills, whether or not the server changes.

### 2026-09-07 (score 75, −1)

- [security] The server rewrote its instructions, which are the text every model session reads
- [security] Tool “am_i_allowed” rewrote its description, which is the text the model reads
- [security] Tool “audit_ai_access” rewrote its description, which is the text the model reads
- [security] Tool “check_path_allowed” rewrote its description, which is the text the model reads
- [security] Tool “diff_robots_txt” rewrote its description, which is the text the model reads
- [security] Tool “lint_robots_txt” rewrote its description, which is the text the model reads
- [security] Tool “merge_policy” rewrote its description, which is the text the model reads
- [security] Tool “whoami” rewrote its description, which is the text the model reads
- [functional regression] Schema quality: 294 → 357
- [functional improvement] Tool coverage: 38% → 44%
- [functional] New tool “no_arguments_lint_this_hosts_robots_txt”

## MCP tools (9)

### `no_arguments_lint_this_hosts_robots_txt` (~554 tokens)

No arguments: lint this host's own robots.txt and ai.txt

TAKES NO ARGUMENTS. POST {"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"no_arguments_lint_this_hosts_robots_txt","arguments":{}}} to https://www.pathwren.workers.dev/mcp/robots — the answer is the full RFC 9309 lint of this host's OWN robots.txt and ai.txt, run for real on the bytes it serves — every finding with its severity, its line number and the fix, plus an allowed/disallowed verdict per crawler token per path with the exact rule and line that produced it, and the check that the two files still agree with each other. There is nothing to fill in: the input schema is literally empty, `arguments: {}` and no `arguments` key at all both work, and the subject is a file this host already publishes, so the answer does not depend on you at all. No key, no account, no OAuth, no session to open first, read-only, and nothing for you to invent. Nothing is fetched to build it — no request leaves this edge, and none is made to you. The other zero-argument call on this server is am_i_allowed, same empty arguments, which answers what this host's own robots.txt says to YOU — the group that matches your user-agent, the line that decided it, that group verbatim, and whether you are allowed each of three paths, evaluated by the same RFC 9309 engine as check_path_allowed. whoami and example are here too and take nothing either. Every other tool on this server wants a file pasted in; this one wants nothing. The siblings answer one question each under the tool named beside them: /mcp (whoami), /mcp/doctor (no_arguments_check_this_hosts_own_discovery_documents), /mcp/lint (whoami), /mcp/triage (no_arguments_triage_this_hosts_own_crawler_log), /mcp/netcheck (no_arguments_report_the_crawler_ip_ranges_this_host_mirrors), /mcp/markdown (markdown_lane_self_report). Example: the complete call, exactly as written, nothing to fill in — {"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"no_arguments_lint_this_hosts_robots_txt","arguments":{}}} returns both files…

Output parameters:

- `answered_by` (object)
- `caveats` (array)
- `files_linted` (array): One entry per file: findings with line numbers and fixes, a summary, and the verdicts.
- `independent` (boolean)
- `license` (string)
- `reproduce` (string)
- `rules_applied` (string)
- `takes_no_arguments` (boolean)
- `the_two_files_agree` (object): Whether robots.txt and ai.txt return the same verdict for every token and path checked.
- `this_call_touched` (object)
- `to_do_this_for_your_own_file` (string)
- `what_this_is` (string)
- `which_ai_crawlers_this_hosts_robots_txt_stops` (object)

### `am_i_allowed` (~474 tokens)

What does this robots.txt say to me? (no arguments)

{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"am_i_allowed","arguments":{}}} — POST that exact body to https://www.pathwren.workers.dev/mcp/robots and the answer is what this host's own robots.txt says to YOU — the group that matches your user-agent, the line that decided it, that group verbatim, and whether you are allowed each of three paths, evaluated by the same RFC 9309 engine as check_path_allowed. THIS IS THE CALL TO MAKE FIRST on this server: it is the only one whose subject is the request you just sent, so there is nothing to look up and no argument to invent. No key, no account, no OAuth, no session to open first, read-only, and nothing for you to invent. Every fact is computed from your own headers and from files this host already publishes; nothing is fetched to build it, and no URL of yours is touched. Only this server answers what this host's own robots.txt says to YOU — the group that matches your token, the line that decided it, and whether you are allowed the path you are on — its six siblings answer one other question each about you, same empty arguments, each under the tool named beside its path: /mcp (whoami), /mcp/doctor (no_arguments_check_this_hosts_own_discovery_documents), /mcp/lint (whoami), /mcp/triage (no_arguments_triage_this_hosts_own_crawler_log), /mcp/netcheck (no_arguments_report_the_crawler_ip_ranges_this_host_mirrors), /mcp/markdown (markdown_lane_self_report) on https://www.pathwren.workers.dev; whoami is answered by all seven and is never an error, so one identical body still walks the whole family if you prefer that. Example: the complete call, exactly as written, nothing to fill in — {"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"am_i_allowed","arguments":{}}} returns your matched group and the line it was matched on, plus an allowed/disallowed verdict for /, /data/agents.json and /mcp/robots.

Output parameters:

- `answered_by` (object): Which server answered, at which endpoint, with which tool.
- `caveats` (array)
- `independent` (boolean)
- `license` (string)
- `this_call_touched` (object): Exactly which files were read. No third party is contacted.
- `we_book_you_as` (object): The class this host's own instrument records for that user-agent.
- `you` (object): The user-agent you sent and the address you came from.

### `whoami` (~371 tokens)

Who is calling? — the one call that needs nothing from you

{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"whoami","arguments":{}}} — POST that to https://www.pathwren.workers.dev/mcp/robots and the answer is the user-agent you sent, the class this host's own instrument books it as, whether we have logged you here before — and the part only this server can answer: which group in this host's own robots.txt matches YOUR user-agent, the line that decided it, and whether you are allowed the paths you are on, evaluated by the same RFC 9309 engine as check_path_allowed. Takes no arguments. Safe to call. Deterministic. Touches no third party. Nothing is fetched to build it: every fact comes from the headers on your own request or from a file this host already publishes. Only this server answers the last part — its five siblings answer one other question each about you, same empty arguments, each under the tool named beside its path: /mcp (whoami), /mcp/doctor (no_arguments_check_this_hosts_own_discovery_documents), /mcp/lint (whoami), /mcp/triage (no_arguments_triage_this_hosts_own_crawler_log), /mcp/netcheck (no_arguments_report_the_crawler_ip_ranges_this_host_mirrors), /mcp/markdown (markdown_lane_self_report) on https://www.pathwren.workers.dev. Example: arguments={} returns your user-agent, your address, the class we book you as, whether we have seen you here before, and what this host's own robots.txt says to YOU — the group that matches your token, the line that decided it, and whether you are allowed the path you are on.

Output parameters:

- `answered_by` (object): Which server answered, at which endpoint.
- `caveats` (array): What this answer does NOT establish — a user-agent is a claim.
- `independent` (boolean): This host is independent and unaffiliated.
- `license` (string)
- `this_call_touched` (object): Exactly which files were read to answer. No third party is contacted.
- `we_book_you_as` (object): The class this host's own instrument records for that user-agent.
- `we_have_seen_you` (object): Whether this user-agent appears in the published observation window.
- `you` (object): The user-agent you sent and the address you came from.

### `example` (~295 tokens)

Run this server's worked example (no arguments)

{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"example","arguments":{}}} — POST that exact body to https://www.pathwren.workers.dev/mcp/robots and you are done: free, read-only, no API key, no account, no consent prompt, nothing for you to invent, and what comes back is this server's own worked example — lint_robots_txt, run for real on a canned input from this host's own data — so you see the exact shape of an answer before you send anything of yours. Takes no arguments. Safe to call. Deterministic. Touches no third party. Runs this server's own worked example end to end — lint_robots_txt on three groups lifted verbatim from this host's own published /robots/block-ai-training.txt and /robots.txt — our data, not yours, and nothing is fetched — and returns exactly the structuredContent a real call returns, not a mock and not a description of one. Use it to see the shape of an answer before you decide what to send. The input is canned from this host's own data; no URL of yours is fetched and no third party is touched. Example: arguments={} runs lint_robots_txt with {"robots_txt":"# lifted from https://www.pathwren.workers.dev/robots/block-ai-training.txt and returns its real answer.

Output parameters:

- `answer` (object): The real structuredContent of that call, not a mock.
- `answered_by` (object)
- `input_came_from` (string): Where the canned input came from — always this host's own data.
- `license` (string)
- `ran` (object): The tool name and the exact arguments that were run.
- `reproduce` (string): A command that reproduces this answer.
- `this_is_not_a_mock` (string)
- `what_it_shows` (string)

### `lint_robots_txt` (~231 tokens)

Lint a robots.txt

FIRST CALL, needs nothing: {"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"no_arguments_lint_this_hosts_robots_txt","arguments":{}}} — Parse a robots.txt you paste and report every fault that makes it do something other than what it looks like: misspelled directives, a full UA string where a product token belongs, rules before any User-agent line, duplicate groups, noindex (unsupported since 2019), relative Sitemap URLs, BOM. Each finding carries the line number and the fix. Example: robots_txt='User-agent: GPTBot\nDisallow: /\n\nUser-agent: *\nAllow: /\n' — paste the whole file, it is never fetched for you. Also callable without MCP, same implementation: GET https://www.pathwren.workers.dev/tools/robots-lint?robots_txt=<urlencoded>&s=client-dossiers — or POST the file as the raw body to the same URL.

Input parameters:

- `robots_txt` (string, required): The contents of the robots.txt file.

### `check_path_allowed` (~260 tokens)

Would this crawler fetch this path?

FIRST CALL, needs nothing: {"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"no_arguments_lint_this_hosts_robots_txt","arguments":{}}} — Evaluate a pasted robots.txt for one crawler and one or more paths under RFC 9309: longest token match for the group, longest path pattern for the rule, Allow breaking a tie, * and $ supported. Returns allowed/disallowed per path with the exact line that decided it, and flags the cases where a merge-groups parser and a first-group-wins parser would disagree. Example: user_agent='GPTBot', paths=['/', '/blog'], with your robots_txt pasted in. Also callable without MCP, same implementation: GET https://www.pathwren.workers.dev/tools/robots-allowed?robots_txt=<urlencoded>&ua=GPTBot&path=/blog&s=client-dossiers

Input parameters:

- `paths`: Path or paths to test, e.g. "/blog/post". Defaults to "/". Max 200.
- `robots_txt` (string, required): The contents of the robots.txt file.
- `user_agent` (string, required): The crawler's product token, e.g. GPTBot, Googlebot, ClaudeBot.

### `audit_ai_access` (~215 tokens)

Which AI crawlers does this file actually stop?

FIRST CALL, needs nothing: {"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"no_arguments_lint_this_hosts_robots_txt","arguments":{}}} — Evaluate a pasted robots.txt against every AI crawler in this index and return the two lists that matter: blocked and allowed, per operator and category. Also names the tokens in your file that match no known crawler (a typo blocks nothing) and separates the crawlers that document obedience from the ones observed ignoring robots.txt, which need an IP or WAF rule instead. Example: path='/' with your robots_txt pasted in — the verdict is per crawler, at that path. Also callable without MCP, same implementation: GET https://www.pathwren.workers.dev/tools/ai-access?robots_txt=<urlencoded>&s=client-dossiers

Input parameters:

- `path` (string): Path to test the verdict at. Defaults to "/".
- `robots_txt` (string, required): The contents of the robots.txt file.

### `diff_robots_txt` (~175 tokens)

Diff two robots.txt by effect

FIRST CALL, needs nothing: {"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"no_arguments_lint_this_hosts_robots_txt","arguments":{}}} — Compare two versions of a robots.txt and report only the crawlers whose verdict actually changes at a given path — not the text difference. Answers 'did my edit do what I meant, and did it do anything else', including sitemap additions and whether the parse errors went up or down. Example: before='User-agent: *\nAllow: /\n', after=your edited file, path='/'.

Input parameters:

- `after` (string, required): The new robots.txt.
- `before` (string, required): The old robots.txt.
- `path` (string): Path to compare verdicts at. Defaults to "/".

### `merge_policy` (~179 tokens)

Add a ready-made stance to an existing file

FIRST CALL, needs nothing: {"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"no_arguments_lint_this_hosts_robots_txt","arguments":{}}} — Merge one of eight maintained robots.txt stances (block-ai-training, allow-ai-search-only, block-all-ai, block-datasets, block-disputed, block-seo-tools, allow-all, maximum-ai-visibility) into a robots.txt you already have, without touching a single rule you wrote: a token you already name keeps your rules and the stance's version is reported instead of applied. Example: stance='block-ai-training', robots_txt='User-agent: *\nAllow: /\n'.

Input parameters:

- `robots_txt` (string, required): Your current robots.txt.
- `stance` (string, required): One of the eight stance slugs.

## Diagnostics

Captured diagnostic sections: TLS, DNSSEC, Authorisation, Transports. The full working is on the page: https://verifymcp.io/servers/dev-workers-pathwren-www-robots-policy-lint/c-mcp-registry-official-mcp-robots#diagnostics

## Score history

- 2026-09-21: 82
- 2026-09-20: 81
- 2026-09-19: 81
- 2026-09-18: 81
- 2026-09-17: 80
- 2026-09-16: 80
- 2026-09-15: 79
- 2026-09-14: 79
- 2026-09-13: 78
- 2026-09-12: 78
- 2026-09-11: 77
- 2026-09-10: 77
- 2026-09-09: 76
- 2026-09-08: 76
- 2026-09-07: 75
- 2026-09-06: 76
- 2026-09-05: 68
- 2026-09-04: 68
- 2026-09-03: 75
- 2026-09-02: 75
- 2026-09-01: 68

## Common questions

### What is the Robots.txt Policy Lint MCP server?

Robots.txt Policy Lint is an MCP server listed in the public MCP registry as dev.workers.pathwren.www/robots-policy-lint. {"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"whoami","arguments":{}}} free no key. This page covers its hosted endpoint (https://www.pathwren.workers.dev/c/mcp-registry-official/mcp/robots).

### Is the Robots.txt Policy Lint MCP server safe to use?

Robots.txt Policy Lint scores 82 out of 100 on VerifyMCP. That is a record of what we were able to check automatically, not an endorsement. The category breakdown on this page shows every signal behind the number, including the ones we could not confirm.

### What tools does the Robots.txt Policy Lint MCP server expose?

Robots.txt Policy Lint exposes 9 tools: no_arguments_lint_this_hosts_robots_txt, am_i_allowed, whoami, example, lint_robots_txt, and 4 more. Their descriptions and schemas cost roughly 2,754 tokens of context every time the server is loaded.

### Does the Robots.txt Policy Lint MCP server require authentication?

No. We connected to Robots.txt Policy Lint without credentials and it answered, so anything it exposes is reachable by anyone who knows the address.

### Is the Robots.txt Policy Lint MCP server still maintained?

Robots.txt Policy Lint is still listed as active in the MCP registry. We last reached this channel on 21 September 2026. Those dates come from our own scans of the registry and the channel itself, not from anything the publisher announced.

## Links

- Remote endpoint: https://www.pathwren.workers.dev/c/mcp-registry-official/mcp/robots
- Website: https://www.pathwren.workers.dev/c/mcp-registry-official/mcp-robots.html
- Changelog RSS feed: https://verifymcp.io/servers/dev-workers-pathwren-www-robots-policy-lint/c-mcp-registry-official-mcp-robots.xml
- Changelog JSON feed: https://verifymcp.io/servers/dev-workers-pathwren-www-robots-policy-lint/c-mcp-registry-official-mcp-robots.json
- HTML version of this page: https://verifymcp.io/servers/dev-workers-pathwren-www-robots-policy-lint/c-mcp-registry-official-mcp-robots