# RobotsGate (remote · robotsgate.mike-tusa.workers.dev)

Generate, validate and check robots.txt rules for AI crawlers. Free; optional Pro key.

- Trust score: 74/100 (medium)
- Change this week: +3
- Registry status: active
- Liveness: live
- Owner verified: no
- Last scored: 2026-11-05

## Components

- remote · `robotsgate.mike-tusa.workers.dev`: 74/100 (this document), [markdown](https://verifymcp.io/servers/dev-workers-mike-tusa-robotsgate-robotsgate/robotsgate.md), [page](https://verifymcp.io/servers/dev-workers-mike-tusa-robotsgate-robotsgate/robotsgate)

## Channel facts

- Endpoint: `https://robotsgate.mike-tusa.workers.dev/mcp`
- Transports: `streamable-http`
- Auth: `none`
- Version: `0.3.1`

## Trust breakdown

How this component scores in each security and reliability category. Every signal is checked automatically against the live server, and we only credit what we can confirm. Scores are 0–100 per category. Scoring method: https://verifymcp.io/docs/scoring (what has changed: https://verifymcp.io/docs/scoring/changelog)

Scored 2026-11-05.

- **Endpoint Security**: 63/100
  - The endpoint's TLS certificate is valid, in date, and uses a strong key.
  - No authorisation is required to call this server. Every tool declares its destructiveHint and none is destructive, so open access doesn't expose one.
  - HTTPS enforcement could not be verified: the plaintext port answered with HTTP 405, which proves neither a plaintext path nor enforcement.
  - HSTS check failed: the Strict-Transport-Security header is absent.
  - DNSSEC check failed: this domain isn't protected by DNSSEC.
- **Transport & Reachability**: 100/100
  - Verified streamable-http transport via a live MCP handshake.
- **Schema Quality & AI Usability**: 63/100
  - AI-judged instruction clarity (good).
  - Context-footprint check failed: tool/resource definitions use about 901 tokens (~225/item across 4 items; 4 tools + 0 resources), over budget; trim descriptions and params.
  - Usage-examples check failed: none of the tools include examples.
- **Stability & Change Management**: 67/100
  - Stability observed for 20 of 30 days with no destabilising changes; credit accrues until the full window elapses.
- **Tool Coverage**: 100/100
  - 100% of tools have a non-trivial description (not blank, and not just the tool's name).
  - 100% of tool parameters carry a description.
  - Structured output schemas are declared (100% of tools); any adoption earns full credit.
- **Tool Safety**: 100/100
  - No prompt-injection markers were found in the server instructions, tool names or descriptions we captured.
  - We read all 4 captured tool definition(s), and no name or description among them implies an irreversible operation.
  - An AI judge read all 5 captured unit(s) of tool text and found none that tries to manipulate the model reading it.
- **Capabilities**: 100/100
  - Implements a current MCP spec version (2026-07-28).

## Install

### How do I install the RobotsGate MCP server?

RobotsGate is a hosted endpoint at https://robotsgate.mike-tusa.workers.dev/mcp, so there is nothing to install locally. Ready-made configuration for Claude, Cursor, VS Code, Codex and 5 more is on this page, copied from each client's own documentation.

### Claude

```bash
claude mcp add --transport http dev-workers-mike-tusa-robotsgate-robotsgate 'https://robotsgate.mike-tusa.workers.dev/mcp'
```

### Cursor

```json
{
  "mcpServers": {
    "dev-workers-mike-tusa-robotsgate-robotsgate": {
      "url": "https://robotsgate.mike-tusa.workers.dev/mcp"
    }
  }
}
```

### VS Code

```json
{
  "servers": {
    "dev-workers-mike-tusa-robotsgate-robotsgate": {
      "type": "http",
      "url": "https://robotsgate.mike-tusa.workers.dev/mcp"
    }
  }
}
```

### Codex

```toml
[mcp_servers.dev-workers-mike-tusa-robotsgate-robotsgate]
url = "https://robotsgate.mike-tusa.workers.dev/mcp"
```

### opencode

```json
{
  "$schema": "https://opencode.ai/config.json",
  "mcp": {
    "dev-workers-mike-tusa-robotsgate-robotsgate": {
      "type": "remote",
      "url": "https://robotsgate.mike-tusa.workers.dev/mcp",
      "enabled": true
    }
  }
}
```

### OpenClaw

```bash
openclaw mcp add dev-workers-mike-tusa-robotsgate-robotsgate --url 'https://robotsgate.mike-tusa.workers.dev/mcp' --transport streamable-http
```

### Hermes

```yaml
mcp_servers:
  dev-workers-mike-tusa-robotsgate-robotsgate:
    url: "https://robotsgate.mike-tusa.workers.dev/mcp"
```

### Netclaw

```json
{
  "McpServers": {
    "dev-workers-mike-tusa-robotsgate-robotsgate": {
      "Transport": "http",
      "Url": "https://robotsgate.mike-tusa.workers.dev/mcp"
    }
  }
}
```

### Vellum

```bash
assistant mcp add dev-workers-mike-tusa-robotsgate-robotsgate -t streamable-http -u 'https://robotsgate.mike-tusa.workers.dev/mcp'
```

### Other

```json
{
  "mcpServers": {
    "dev-workers-mike-tusa-robotsgate-robotsgate": {
      "type": "http",
      "url": "https://robotsgate.mike-tusa.workers.dev/mcp"
    }
  }
}
```

The mcpServers block is a cross-client convention. Remote transports vary, so check your client's docs.

## Changelog

Every change recorded for this component, newest first. Days that predate change tracking, or that we cannot explain, say so: "we were watching and nothing happened" and "we were not watching" are different claims.

### 2026-11-05 (score 74, +1)

No change was recorded against any check on this day. Stability & Change Management went from 63 to 67. That category is still filling its 30-day observation window: 19 days of observed history at the previous scan, 20 at this one. The score rises as the window fills, whether or not the server changes.

### 2026-11-02 (score 73, +1)

No change was recorded against any check on this day. Stability & Change Management went from 53 to 57. That category is still filling its 30-day observation window: 16 days of observed history at the previous scan, 17 at this one. The score rises as the window fills, whether or not the server changes.

### 2026-10-31 (score 72, +1)

No change was recorded against any check on this day. Stability & Change Management went from 47 to 50. That category is still filling its 30-day observation window: 14 days of observed history at the previous scan, 15 at this one. The score rises as the window fills, whether or not the server changes.

### 2026-10-29 (score 71, +1)

No change was recorded against any check on this day. Stability & Change Management went from 40 to 43. That category is still filling its 30-day observation window: 12 days of observed history at the previous scan, 13 at this one. The score rises as the window fills, whether or not the server changes.

### 2026-10-27 (score 70, +1)

No change was recorded against any check on this day. Stability & Change Management went from 33 to 37. That category is still filling its 30-day observation window: 10 days of observed history at the previous scan, 11 at this one. The score rises as the window fills, whether or not the server changes.

### 2026-10-25 (score 69, +1)

No change was recorded against any check on this day. Stability & Change Management went from 27 to 30. That category is still filling its 30-day observation window: 8 days of observed history at the previous scan, 9 at this one. The score rises as the window fills, whether or not the server changes.

### 2026-10-23 (score 68, +1)

No change was recorded against any check on this day. Stability & Change Management went from 20 to 23. That category is still filling its 30-day observation window: 6 days of observed history at the previous scan, 7 at this one. The score rises as the window fills, whether or not the server changes.

### 2026-10-21 (score 67, +1)

No change was recorded against any check on this day. Stability & Change Management went from 13 to 17. That category is still filling its 30-day observation window: 4 days of observed history at the previous scan, 5 at this one. The score rises as the window fills, whether or not the server changes.

## MCP tools (4)

### `list_crawlers` (~97 tokens)

List the AI crawler registry

Return RobotsGate's AI crawler registry: each crawler's robots.txt token, operator, category (training, dataset, search, user_fetch or other), whether the operator says it respects robots.txt, and a link to the operator's documentation, plus unverified entries and notes. Read-only, no network access. Same data as GET /api/crawlers.

Input parameters:

- `category` (string): Optional: only crawlers in this category.

Output parameters:

- `categories` (object)
- `crawlers` (array)
- `generated` (string)
- `schema_version` (integer)
- `unverified` (array)

### `generate_robots` (~324 tokens)

Generate a robots.txt for AI crawlers

Build a robots.txt from per-category choices (allow, block or omit for training, dataset, search, user_fetch or other), optional per-token overrides, a default policy for other bots, paths, sitemaps and custom rules. Returns the file text, a per-crawler summary and notes. Read-only: it only returns text and changes nothing. Same engine as POST /api/generate.

Input parameters:

- `agents` (object): Per-token overrides, e.g. {"GPTBot": "block"}. Tokens: letters, digits, "_", "." and "-", up to 64 characters (not "*"). At most 100.
- `allow_paths` (array): Paths to allow explicitly. Each path starts with "/" or "*" and has no whitespace or "#". At most 100.
- `categories` (object): Action per crawler category. Omitted categories get no group of their own.
- `custom_rules` (string): Extra robots.txt lines appended after validation (max 20000 bytes).
- `date` (string): Date for the header comment (YYYY-MM-DD). Default: today (UTC).
- `default` (string): Policy for "User-agent: *" (default allow).
- `disallow_paths` (array): Paths to disallow for "*" and for allowed AI groups. Each path starts with "/" or "*" and has no whitespace or "#". At most 100.
- `sitemap` (string): A single sitemap URL (alternative to sitemaps).
- `sitemaps` (array): Absolute http(s) sitemap URLs. At most 20.

Output parameters:

- `notes` (array)
- `personal_agents_note` (string)
- `robots_txt` (string)
- `summary` (array)

### `validate_robots` (~128 tokens)

Validate robots.txt text

Validate robots.txt text against RFC 9309 and report errors, warnings and, for one URL path, which registry AI crawlers are allowed or blocked and by which rule. Read-only, no network access. Same engine as POST /api/validate. Up to 50,000 characters over MCP (the whole JSON-RPC request must also fit in 64 KB); for larger files use POST /api/validate or check_site.

Input parameters:

- `path` (string): URL path to test, starting with "/" (default "/").
- `robots_txt` (string, required): The robots.txt file content.

Output parameters:

- `ai_crawlers` (array)
- `diagnostics` (object)
- `errors` (array)
- `info` (array)
- `path` (string)
- `personal_agents` (object)
- `stats` (object)
- `valid` (boolean)
- `warnings` (array)

### `check_site` (~140 tokens)

Check which AI crawlers a live site allows

Fetch a site's /robots.txt (http/https on the default port only; private, loopback and internal addresses are refused; up to 512,000 bytes; results cached for 5 minutes) and report, for one path, which registry AI crawlers are allowed or blocked, plus validation errors and warnings. Read-only: one GET of /robots.txt with a RobotsGate user-agent. Same engine as GET /api/check.

Input parameters:

- `path` (string): URL path to test, starting with "/" (default "/").
- `url` (string, required): Site URL or bare hostname, e.g. example.com. Only the scheme and host are used.

Output parameters:

- `ai_crawlers`: Per-crawler results, or null when robots.txt was unreachable
- `cache` (object)
- `fetched_at` (string)
- `final_url` (string)
- `http_status` (integer)
- `interpretation` (string)
- `path` (string)
- `personal_agents`: Object, or null when robots.txt was unreachable
- `redirects` (integer)
- `robots_found` (boolean)
- `url` (string)

## Diagnostics

Captured diagnostic sections: TLS, DNSSEC, Authorisation, Transports. The full working is on the page: https://verifymcp.io/servers/dev-workers-mike-tusa-robotsgate-robotsgate/robotsgate#diagnostics

## Score history

- 2026-11-05: 74
- 2026-11-04: 73
- 2026-11-03: 73
- 2026-11-02: 73
- 2026-11-01: 72
- 2026-10-31: 72
- 2026-10-30: 71
- 2026-10-29: 71
- 2026-10-28: 70
- 2026-10-27: 70
- 2026-10-26: 69
- 2026-10-25: 69
- 2026-10-24: 68
- 2026-10-23: 68
- 2026-10-22: 67
- 2026-10-21: 67
- 2026-10-20: 66
- 2026-10-19: 66
- 2026-10-18: 66
- 2026-10-17: 65
- 2026-10-16: 65

## Common questions

### What is the RobotsGate MCP server?

RobotsGate is an MCP server listed in the public MCP registry as dev.workers.mike-tusa.robotsgate/robotsgate. Generate, validate and check robots.txt rules for AI crawlers. Free; optional Pro key. This page covers its hosted endpoint (https://robotsgate.mike-tusa.workers.dev/mcp).

### Is the RobotsGate MCP server safe to use?

RobotsGate scores 74 out of 100 on VerifyMCP. That is a record of what we were able to check automatically, not an endorsement. The category breakdown on this page shows every signal behind the number, including the ones we could not confirm.

### What tools does the RobotsGate MCP server expose?

RobotsGate exposes 4 tools: list_crawlers, generate_robots, validate_robots, check_site. Their descriptions and schemas cost roughly 689 tokens of context every time the server is loaded.

### Does the RobotsGate MCP server require authentication?

No. We connected to RobotsGate without credentials and it answered, so anything it exposes is reachable by anyone who knows the address.

### Is the RobotsGate MCP server still maintained?

RobotsGate is still listed as active in the MCP registry. We last reached this channel on 5 November 2026. Those dates come from our own scans of the registry and the channel itself, not from anything the publisher announced.

## Links

- Remote endpoint: https://robotsgate.mike-tusa.workers.dev/mcp
- Website: https://robotsgate.mike-tusa.workers.dev/
- Changelog RSS feed: https://verifymcp.io/servers/dev-workers-mike-tusa-robotsgate-robotsgate/robotsgate.xml
- Changelog JSON feed: https://verifymcp.io/servers/dev-workers-mike-tusa-robotsgate-robotsgate/robotsgate.json
- HTML version of this page: https://verifymcp.io/servers/dev-workers-mike-tusa-robotsgate-robotsgate/robotsgate