# io.github.sharozdawa/robotstxt-ai (npm · robotstxt-ai-mcp)

Analyze and generate robots.txt with AI crawler detection for 20+ bots like GPTBot and ClaudeBot.

- Trust score: 69/100 (medium)
- Change this week: +23
- Registry status: active
- Liveness: live
- Owner verified: no
- Last scored: 2026-08-03

## Components

- npm · `robotstxt-ai-mcp`: 69/100 (this document), [markdown](https://verifymcp.io/servers/sharozdawa-robotstxt-ai/robotstxt-ai-mcp.md), [page](https://verifymcp.io/servers/sharozdawa-robotstxt-ai/robotstxt-ai-mcp)

## Channel facts

- Registry: `npm`
- Package: `robotstxt-ai-mcp`
- Version: `1.0.1`
- Transport: `stdio`

## Trust breakdown

How this component scores in each security and reliability category. Every signal is checked automatically from public evidence about the published package, including repeated runs of it in an isolated sandbox, and we only credit what we can confirm. Scores are 0–100 per category. Scoring method: https://verifymcp.io/docs/scoring (what has changed: https://verifymcp.io/docs/scoring/changelog)

Scored 2026-08-03.

- **Supply Chain Security**: 86/100
  - No malware found by supply-chain analysis.
  - Only part of the dependency tree could be resolved (94 of 98), so this covers what we could see, not the whole tree.
  - No install/post-install scripts declared.
  - Only part of the dependency tree could be resolved (94 of 98), so this covers what we could see, not the whole tree.
- **Provenance & Transparency**: 45/100
  - Source repository is publicly reachable at the declared URL.
  - Provenance check failed: no build-provenance attestation is published.
  - Clear OSI-approved license (MIT).
  - Actively maintained (last published 134 days ago).
  - Disclosure check failed: no security disclosure policy was found in the source repository.
- **Schema Quality & AI Usability**: 77/100
  - AI-judged instruction clarity (excellent).
  - Tool/resource definitions use about 355 tokens (~71/item across 5 items; 5 tools + 0 resources), lean.
  - Usage-examples check failed: none of the tools include examples.
- **Stability & Change Management**: 23/100
  - Stability observed for 7 of 30 days with no destabilising changes; credit accrues until the full window elapses.
- **Tool Coverage**: 100/100
  - 100% of tools have a non-trivial description (not blank, and not just the tool's name).
  - 100% of tool parameters carry a description.
- **Capabilities**: 100/100
  - Implements a supported MCP spec version (2025-11-25); the latest is 2026-07-28.

## Install

### Claude

```bash
claude mcp add sharozdawa-robotstxt-ai -- npx -y robotstxt-ai-mcp
```

### Codex

```bash
codex mcp add sharozdawa-robotstxt-ai -- npx -y robotstxt-ai-mcp
```

### opencode

```json
{
  "$schema": "https://opencode.ai/config.json",
  "mcp": {
    "sharozdawa-robotstxt-ai": {
      "type": "local",
      "command": [
        "npx",
        "-y",
        "robotstxt-ai-mcp"
      ],
      "enabled": true
    }
  }
}
```

### OpenClaw

```bash
openclaw mcp add sharozdawa-robotstxt-ai --command npx --arg -y --arg robotstxt-ai-mcp
```

### Hermes

```yaml
mcp_servers:
  sharozdawa-robotstxt-ai:
    command: "npx"
    args: ["-y", "robotstxt-ai-mcp"]
```

### Other

```json
{
  "mcpServers": {
    "sharozdawa-robotstxt-ai": {
      "command": "npx",
      "args": [
        "-y",
        "robotstxt-ai-mcp"
      ]
    }
  }
}
```

## Changelog

Every change recorded for this component, newest first. Days that predate change tracking, or that we cannot explain, say so: "we were watching and nothing happened" and "we were not watching" are different claims.

### 2026-08-03 (score 69, +1)

No change was recorded against any check on this day. Stability & Change Management went from 20 to 23. That category is still filling its 30-day observation window: 6 days of observed history at the previous scan, 7 at this one. The score rises as the window fills, whether or not the server changes.

### 2026-08-02 (score 68, +47)

- [security regression] Provenance: unverified → fail
- [security improvement] Known CVEs: unverified → partial
- [security improvement] Install scripts: unverified → pass
- [security improvement] Malware scan: unverified → pass
- [functional improvement] License: unverified → pass
- [functional improvement] Stability: unverified → 0.20
- [functional improvement] MCP protocol: unverified → pass
- [functional improvement] Maintenance: unverified → pass
- [functional improvement] Dependency health: unverified → partial
- [functional improvement] Schema quality: unverified → excellent
- [functional] Licence: MIT

### 2026-07-31 (score 21, −7)

- [functional] We updated how we score, so this day's move reflects our rubric, not a change to the server

### 2026-07-30 (score 28, −40)

- [security regression] Known CVEs: partial → unverified
- [security regression] Malware scan: pass → unverified
- [security regression] Provenance: fail → unverified
- [security regression] Install scripts: pass → unverified
- [functional regression] License: pass → unverified
- [functional regression] Dependency health: partial → unverified
- [functional regression] Maintenance: pass → unverified
- [functional] Licence: MIT

### 2026-07-28 (score 68, +22)

- [security regression] Provenance: unverified → fail
- [security improvement] Known CVEs: unverified → partial
- [security improvement] Install scripts: unverified → pass
- [functional improvement] Maintenance: unverified → pass
- [functional improvement] Dependency health: unverified → partial
- [functional improvement] License: unverified → pass
- [functional] Licence: MIT

### 2026-07-27 (score 46)

First indexed and scored.

## MCP tools (5)

### `fetch_robots` (~52 tokens)

Fetch and analyze a robots.txt file from a URL. Returns which AI bots are blocked or allowed.

Input parameters:

- `url` (string, required): The website URL to fetch robots.txt from (e.g. https://example.com)

### `analyze_robots` (~41 tokens)

Analyze pasted robots.txt content. Returns which AI bots are blocked or allowed based on the rules.

Input parameters:

- `content` (string, required): The robots.txt content to analyze

### `generate_robots` (~142 tokens)

Generate a robots.txt file with specified blocked bots, sitemap URLs, and custom rules.

Input parameters:

- `block_all_ai` (boolean): If true, blocks all known AI crawlers (but not search engines like Googlebot)
- `blocked_bots` (array): Array of bot user-agent strings to block (e.g. ['GPTBot', 'ClaudeBot', 'CCBot']). Use list_ai_bots to see available user-agents.
- `custom_rules` (string): Additional custom robots.txt rules to append (raw robots.txt syntax)
- `sitemap_urls` (array): Array of sitemap URLs to include (e.g. ['https://example.com/sitemap.xml'])

### `list_ai_bots` (~34 tokens)

List all known AI bots with their user-agents, companies, and descriptions. Useful for deciding which bots to block.

### `check_bot_status` (~86 tokens)

Check if a specific bot is blocked or allowed on a given website by fetching and analyzing its robots.txt.

Input parameters:

- `bot_name` (string, required): The bot user-agent string to check (e.g. 'GPTBot', 'ClaudeBot'). Use list_ai_bots to see available names.
- `url` (string, required): The website URL to check (e.g. https://example.com)

## Diagnostics

Captured diagnostic sections: Provenance, Dependencies. The full working is on the page: https://verifymcp.io/servers/sharozdawa-robotstxt-ai/robotstxt-ai-mcp#diagnostics

## Score history

- 2026-08-03: 69
- 2026-08-02: 68
- 2026-08-01: 21
- 2026-07-31: 21
- 2026-07-30: 28
- 2026-07-28: 68
- 2026-07-27: 46

## Links

- npm package: https://www.npmjs.com/package/robotstxt-ai-mcp
- Socket report: https://socket.dev/npm/package/robotstxt-ai-mcp
- Repository: https://github.com/sharozdawa/robotstxt-ai
- Changelog RSS feed: https://verifymcp.io/servers/sharozdawa-robotstxt-ai/robotstxt-ai-mcp/changelog.xml
- Changelog JSON feed: https://verifymcp.io/servers/sharozdawa-robotstxt-ai/robotstxt-ai-mcp/changelog.json
- HTML version of this page: https://verifymcp.io/servers/sharozdawa-robotstxt-ai/robotstxt-ai-mcp