# ML Referee (remote · api.mlreferee.com)

Evaluate ML features beyond a baseline and test bounded feature repairs. Invite-only beta.

- Trust score: 62/100 (medium)
- Change this week: +3
- Registry status: active
- Liveness: live
- Owner verified: no
- Last scored: 2026-11-04

## Components

- remote · `api.mlreferee.com`: 62/100 (this document), [markdown](https://verifymcp.io/servers/scarrico-mlreferee/api.md), [page](https://verifymcp.io/servers/scarrico-mlreferee/api)

## Channel facts

- Endpoint: `https://api.mlreferee.com/mcp`
- Transports: `streamable-http`
- Auth: `none`
- Version: `1.1.0`

## Trust breakdown

How this component scores in each security and reliability category. Every signal is checked automatically against the live server, and we only credit what we can confirm. Scores are 0–100 per category. Scoring method: https://verifymcp.io/docs/scoring (what has changed: https://verifymcp.io/docs/scoring/changelog)

Scored 2026-11-04.

- **Endpoint Security**: 57/100
  - The endpoint's TLS certificate is valid, in date, and uses a strong key.
  - Authorisation check failed: no authorisation is required to call this server, and it exposes a tool marked destructive (delete_audit).
  - HTTPS is enforced; there's no plaintext access path.
  - HSTS check failed: the Strict-Transport-Security header is absent.
  - DNSSEC check failed: this domain isn't protected by DNSSEC.
- **Transport & Reachability**: 100/100
  - Verified streamable-http transport via a live MCP handshake.
- **Schema Quality & AI Usability**: 56/100
  - AI-judged instruction clarity (fair).
  - Tool/resource definitions use about 1051 tokens (~40/item across 26 items; 26 tools + 0 resources), lean.
  - Usage-examples check failed: none of the tools include examples.
- **Stability & Change Management**: 27/100
  - Stability observed for 8 of 30 days with no destabilising changes; credit accrues until the full window elapses.
- **Tool Coverage**: 67/100
  - 100% of tools have a non-trivial description (not blank, and not just the tool's name).
  - 0% of tool parameters carry a description.
- **Tool Safety**: 100/100
  - No prompt-injection markers were found in the server instructions, tool names or descriptions we captured.
  - All 2 tool(s) whose name or description implies an irreversible operation declare an MCP destructiveHint annotation.
  - An AI judge read all 26 captured unit(s) of tool text and found none that tries to manipulate the model reading it.
- **Capabilities**: 100/100
  - Implements a supported MCP spec version (2025-11-25); the latest is 2026-07-28.

## Install

### How do I install the ML Referee MCP server?

ML Referee is a hosted endpoint at https://api.mlreferee.com/mcp, so there is nothing to install locally. Ready-made configuration for Claude, Cursor, VS Code, Codex and 5 more is on this page, copied from each client's own documentation.

### Claude

```bash
claude mcp add --transport http scarrico-mlreferee 'https://api.mlreferee.com/mcp'
```

### Cursor

```json
{
  "mcpServers": {
    "scarrico-mlreferee": {
      "url": "https://api.mlreferee.com/mcp"
    }
  }
}
```

### VS Code

```json
{
  "servers": {
    "scarrico-mlreferee": {
      "type": "http",
      "url": "https://api.mlreferee.com/mcp"
    }
  }
}
```

### Codex

```toml
[mcp_servers.scarrico-mlreferee]
url = "https://api.mlreferee.com/mcp"
```

### opencode

```json
{
  "$schema": "https://opencode.ai/config.json",
  "mcp": {
    "scarrico-mlreferee": {
      "type": "remote",
      "url": "https://api.mlreferee.com/mcp",
      "enabled": true
    }
  }
}
```

### OpenClaw

```bash
openclaw mcp add scarrico-mlreferee --url 'https://api.mlreferee.com/mcp' --transport streamable-http
```

### Hermes

```yaml
mcp_servers:
  scarrico-mlreferee:
    url: "https://api.mlreferee.com/mcp"
```

### Netclaw

```json
{
  "McpServers": {
    "scarrico-mlreferee": {
      "Transport": "http",
      "Url": "https://api.mlreferee.com/mcp"
    }
  }
}
```

### Vellum

```bash
assistant mcp add scarrico-mlreferee -t streamable-http -u 'https://api.mlreferee.com/mcp'
```

### Other

```json
{
  "mcpServers": {
    "scarrico-mlreferee": {
      "type": "http",
      "url": "https://api.mlreferee.com/mcp"
    }
  }
}
```

The mcpServers block is a cross-client convention. Remote transports vary, so check your client's docs.

## Changelog

Every change recorded for this component, newest first. Days that predate change tracking, or that we cannot explain, say so: "we were watching and nothing happened" and "we were not watching" are different claims.

### 2026-11-04 (score 62, +1)

No change was recorded against any check on this day. Stability & Change Management went from 23 to 27. That category is still filling its 30-day observation window: 7 days of observed history at the previous scan, 8 at this one. The score rises as the window fills, whether or not the server changes.

### 2026-11-01 (score 61, +1)

No change was recorded against any check on this day. Stability & Change Management went from 13 to 17. That category is still filling its 30-day observation window: 4 days of observed history at the previous scan, 5 at this one. The score rises as the window fills, whether or not the server changes.

### 2026-10-30 (score 60, +1)

No change was recorded against any check on this day. Stability & Change Management went from 7 to 10. That category is still filling its 30-day observation window: 2 days of observed history at the previous scan, 3 at this one. The score rises as the window fills, whether or not the server changes.

### 2026-10-28 (score 59, +1)

- [functional improvement] Stability: unverified → 0.03

### 2026-10-27 (score 58)

First indexed and scored.

## MCP tools (26)

### `describe_referee` (~20 tokens)

Discover capabilities, privacy, and the approved public catalog.

### `list_verticals` (~20 tokens)

List the three supported evaluation profiles and their discovery tags.

### `get_referee_readiness` (~27 tokens)

Report connection, credential, audit-credit, and submission readiness with the next action.

### `get_referee_usage` (~35 tokens)

Read aggregate, privacy-preserving Referee activity and outcome evidence for the selected audience.

Input parameters:

- `window_days` (integer)

### `get_input_schema` (~27 tokens)

Get neutral column roles and validation defaults for a vertical.

Input parameters:

- `vertical` (string, required)

### `get_repair_schema` (~23 tokens)

Get the feature-only contract for diagnosing degradation and testing bounded repairs.

### `upload_private_dataset` (~48 tokens)

Encrypt a de-identified CSV/JSON payload for a short tenant-private retention window.

Input parameters:

- `data` (string, required)
- `format` (string)
- `retention_seconds` (integer)

### `upload_campaign_shard` (~57 tokens)

Encrypt one of up to 64 private feature shards for a large campaign; retain for at least 4,200 seconds.

Input parameters:

- `data` (string, required)
- `format` (string)
- `retention_seconds` (integer)

### `upload_repair_matrix` (~54 tokens)

Encrypt one complete de-identified repair matrix of up to 100 MB and 1,000 diagnostic features.

Input parameters:

- `data` (string, required)
- `format` (string)
- `retention_seconds` (integer)

### `preflight_dataset` (~46 tokens)

Validate a tenant-private upload and its baseline evidence/row identity without spending a credit.

Input parameters:

- `contract` (required)
- `upload_id` (string, required)
- `vertical` (string, required)

### `preflight_repair` (~36 tokens)

Validate a tenant-private feature-repair matrix without spending a credit.

Input parameters:

- `contract` (required)
- `upload_id` (string, required)

### `quote_audit` (~57 tokens)

Create a contract-bound quote for one audit credit; baseline mode and evidence cannot be substituted.

Input parameters:

- `contract` (required)
- `idempotency_key` (string, required)
- `upload_id` (string, required)
- `vertical` (string, required)

### `quote_repair` (~51 tokens)

Preflight and quote label-based feature repair; large jobs return a measured ETA and credit requirement.

Input parameters:

- `contract` (required)
- `idempotency_key` (string, required)
- `upload_id` (string, required)

### `quote_campaign` (~62 tokens)

Preflight all private feature shards, time a pilot, and quote credits and ETA for up to 1,000 candidates in one corrected search family.

Input parameters:

- `idempotency_key` (string, required)
- `shards` (array, required)
- `vertical` (string, required)

### `get_credits` (~19 tokens)

Read beta or paid audit-credit availability and usage.

### `redeem_beta_invite` (~38 tokens)

Redeem one operator-issued invite for free beta audit credits before January 1, 2027.

Input parameters:

- `invite_code` (string, required)

### `request_beta_access` (~45 tokens)

Request operator-reviewed beta access without purchasing or granting credits.

Input parameters:

- `idempotency_key` (string, required)
- `lane` (string, required)
- `use_case` (string, required)

### `get_beta_request_status` (~28 tokens)

Read the authenticated tenant's beta-access request status.

Input parameters:

- `request_id` (string, required)

### `submit_audit` (~39 tokens)

Accept a quote and atomically reserve its disclosed audit credits.

Input parameters:

- `idempotency_key` (string, required)
- `quote_id` (string, required)

### `record_audit_followup` (~47 tokens)

Record whether a completed audit changed a decision or was confirmed prospectively.

Input parameters:

- `audit_id` (string, required)
- `kind` (string, required)
- `value` (string, required)

### `get_audit_status` (~30 tokens)

Read the state of an audit owned by the authenticated tenant.

Input parameters:

- `audit_id` (string, required)

### `get_audit_result` (~29 tokens)

Read a completed tenant-private result and signed output manifest.

Input parameters:

- `audit_id` (string, required)

### `get_audit_receipt` (~46 tokens)

Read a tenant-private report fingerprint retained after report expiry or deletion. Older reports may have no record. This does not retrieve report contents.

Input parameters:

- `audit_id` (string, required)

### `verify_audit_report` (~110 tokens)

Compare a locally computed SHA-256 with the retained tenant-private receipt. Use the exact downloaded result JSON bytes (not the API envelope), or UTF-8 result.report text with representation report_text. Never copy the expected hash as the candidate: compute it from the supplied artifact. A digest match does not validate a summary, factual correctness, or absence of harm. No report is uploaded.

Input parameters:

- `audit_id` (string, required)
- `representation` (string)
- `sha256` (string, required)

### `delete_audit` (~30 tokens)

Delete retained inputs and results for an audit owned by this tenant.

Input parameters:

- `audit_id` (string, required)

### `delete_upload` (~27 tokens)

Delete a tenant-private upload that is no longer needed.

Input parameters:

- `upload_id` (string, required)

## Diagnostics

Captured diagnostic sections: TLS, DNSSEC, Authorisation, Transports. The full working is on the page: https://verifymcp.io/servers/scarrico-mlreferee/api#diagnostics

## Score history

- 2026-11-04: 62
- 2026-11-03: 61
- 2026-11-02: 61
- 2026-11-01: 61
- 2026-10-31: 60
- 2026-10-30: 60
- 2026-10-29: 59
- 2026-10-28: 59
- 2026-10-27: 58

## Common questions

### What is the ML Referee MCP server?

ML Referee is an MCP server listed in the public MCP registry as io.github.scarrico/mlreferee. Evaluate ML features beyond a baseline and test bounded feature repairs. Invite-only beta. This page covers its hosted endpoint (https://api.mlreferee.com/mcp).

### Is the ML Referee MCP server safe to use?

ML Referee scores 62 out of 100 on VerifyMCP. That is a record of what we were able to check automatically, not an endorsement. The category breakdown on this page shows every signal behind the number, including the ones we could not confirm.

### What tools does the ML Referee MCP server expose?

ML Referee exposes 26 tools: describe_referee, list_verticals, get_referee_readiness, get_referee_usage, get_input_schema, and 21 more. Their descriptions and schemas cost roughly 1,051 tokens of context every time the server is loaded.

### Does the ML Referee MCP server require authentication?

No. We connected to ML Referee without credentials and it answered, so anything it exposes is reachable by anyone who knows the address.

### Is the ML Referee MCP server still maintained?

ML Referee is still listed as active in the MCP registry. We last reached this channel on 4 November 2026. Those dates come from our own scans of the registry and the channel itself, not from anything the publisher announced.

## Links

- Remote endpoint: https://api.mlreferee.com/mcp
- Website: https://mlreferee.com/
- Changelog RSS feed: https://verifymcp.io/servers/scarrico-mlreferee/api.xml
- Changelog JSON feed: https://verifymcp.io/servers/scarrico-mlreferee/api.json
- HTML version of this page: https://verifymcp.io/servers/scarrico-mlreferee/api