# retriEVAL (oci · ghcr.io/hcarrillo001/retrieval-mcp:1.0.0)

LLM evals as MCP tools: score outputs for faithfulness, relevancy, and hallucination.

- Trust score: 46/100 (low)
- Change this week: +3
- Registry status: active
- Liveness: live
- Owner verified: no
- Last scored: 2026-09-20

## Components

- oci · `ghcr.io/hcarrillo001/retrieval-mcp:1.0.0`: 46/100 (this document), [markdown](https://verifymcp.io/servers/hcarrillo001-retrieval-mcp/ghcr-io-hcarrillo001-retrieval-mcp-1-0-0.md), [page](https://verifymcp.io/servers/hcarrillo001-retrieval-mcp/ghcr-io-hcarrillo001-retrieval-mcp-1-0-0)

## Channel facts

- Registry: `oci`
- Package: `ghcr.io/hcarrillo001/retrieval-mcp:1.0.0`
- Transport: `stdio`

## Trust breakdown

How this component scores in each security and reliability category. Every signal is checked automatically from public evidence about the published package, including repeated runs of it in an isolated sandbox, and we only credit what we can confirm. Scores are 0–100 per category. Scoring method: https://verifymcp.io/docs/scoring (what has changed: https://verifymcp.io/docs/scoring/changelog)

Scored 2026-09-20.

- **Supply Chain Security**: 0/100
  - Malware scan not yet available for this package.
  - Known CVEs could not be checked: this artifact ships no SBOM, so there is no dependency list to read. Publishing one would let us assess it.
  - Install-script risk not yet assessed.
  - Dependency health could not be checked: this artifact ships no SBOM, so there is no dependency list to read. Publishing one would let us assess it.
- **Provenance & Transparency**: 32/100
  - Source repository is publicly reachable at the declared URL.
  - Provenance check failed: no build-provenance attestation is published.
  - License check failed: no license is declared.
  - Actively maintained (last published 27 days ago).
  - Disclosure check failed: no security disclosure policy was found in the source repository.
- **Schema Quality & AI Usability**: 75/100
  - AI-judged instruction clarity (good).
  - Tool/resource definitions use about 1320 tokens (~94/item across 14 items; 14 tools + 0 resources), lean.
  - Usage-examples check failed: none of the tools include examples.
- **Stability & Change Management**: 90/100
  - Stability observed for 27 of 30 days with no destabilising changes; credit accrues until the full window elapses.
- **Tool Coverage**: 67/100
  - 100% of tools have a non-trivial description (not blank, and not just the tool's name).
  - 0% of tool parameters carry a description.
- **Tool Safety**: 75/100
  - No prompt-injection markers were found in the server instructions, tool names or descriptions we captured.
  - 0 of 1 tool(s) whose name or description implies an irreversible operation declare an MCP destructiveHint annotation; "run_eval" implies "eval" and declares no destructiveHint at all, which the MCP spec reads as destructive by default.
  - An AI judge read all 14 captured unit(s) of tool text and found none that tries to manipulate the model reading it.
- **Capabilities**: 100/100
  - Implements a supported MCP spec version (2025-11-25); the latest is 2026-07-28.

**Unverified: 1 category.** A category scored 0 because we could not verify it: a data source with nothing on this package, evidence we could not reach, or a check we could not run. We only credit what we can confirm.

## Install

### How do I install the retriEVAL MCP server?

retriEVAL runs locally as a container image, launched with docker run --rm -i ghcr.io/hcarrillo001/retrieval-mcp:1.0.0. Ready-made configuration for Claude, Cursor, VS Code, Codex and 3 more is on this page, copied from each client's own documentation.

### Claude

```bash
claude mcp add hcarrillo001-retrieval-mcp -- docker run --rm -i ghcr.io/hcarrillo001/retrieval-mcp:1.0.0
```

### Cursor

```json
{
  "mcpServers": {
    "hcarrillo001-retrieval-mcp": {
      "command": "docker",
      "args": [
        "run",
        "--rm",
        "-i",
        "ghcr.io/hcarrillo001/retrieval-mcp:1.0.0"
      ]
    }
  }
}
```

### VS Code

```json
{
  "servers": {
    "hcarrillo001-retrieval-mcp": {
      "command": "docker",
      "args": [
        "run",
        "--rm",
        "-i",
        "ghcr.io/hcarrillo001/retrieval-mcp:1.0.0"
      ]
    }
  }
}
```

### Codex

```bash
codex mcp add hcarrillo001-retrieval-mcp -- docker run --rm -i ghcr.io/hcarrillo001/retrieval-mcp:1.0.0
```

### opencode

```json
{
  "$schema": "https://opencode.ai/config.json",
  "mcp": {
    "hcarrillo001-retrieval-mcp": {
      "type": "local",
      "command": [
        "docker",
        "run",
        "--rm",
        "-i",
        "ghcr.io/hcarrillo001/retrieval-mcp:1.0.0"
      ],
      "enabled": true
    }
  }
}
```

### Hermes

```yaml
mcp_servers:
  hcarrillo001-retrieval-mcp:
    command: "docker"
    args: ["run", "--rm", "-i", "ghcr.io/hcarrillo001/retrieval-mcp:1.0.0"]
```

### Netclaw

```json
{
  "McpServers": {
    "hcarrillo001-retrieval-mcp": {
      "Transport": "stdio",
      "Command": "docker",
      "Arguments": [
        "run",
        "--rm",
        "-i",
        "ghcr.io/hcarrillo001/retrieval-mcp:1.0.0"
      ]
    }
  }
}
```

### Other

```json
{
  "mcpServers": {
    "hcarrillo001-retrieval-mcp": {
      "command": "docker",
      "args": [
        "run",
        "--rm",
        "-i",
        "ghcr.io/hcarrillo001/retrieval-mcp:1.0.0"
      ]
    }
  }
}
```

## Changelog

Every change recorded for this component, newest first. Days that predate change tracking, or that we cannot explain, say so: "we were watching and nothing happened" and "we were not watching" are different claims.

### 2026-09-19 (score 46, +1)

No change was recorded against any check on this day. Stability & Change Management went from 83 to 87. That category is still filling its 30-day observation window: 25 days of observed history at the previous scan, 26 at this one. The score rises as the window fills, whether or not the server changes.

### 2026-09-16 (score 45, +1)

No change was recorded against any check on this day. Stability & Change Management went from 73 to 77. That category is still filling its 30-day observation window: 22 days of observed history at the previous scan, 23 at this one. The score rises as the window fills, whether or not the server changes.

### 2026-09-14 (score 44, +1)

No change was recorded against any check on this day. Stability & Change Management went from 67 to 70. That category is still filling its 30-day observation window: 20 days of observed history at the previous scan, 21 at this one. The score rises as the window fills, whether or not the server changes.

### 2026-09-12 (score 43, +1)

No change was recorded against any check on this day. Stability & Change Management went from 60 to 63. That category is still filling its 30-day observation window: 18 days of observed history at the previous scan, 19 at this one. The score rises as the window fills, whether or not the server changes.

### 2026-09-10 (score 42, +1)

No change was recorded against any check on this day. Stability & Change Management went from 53 to 57. That category is still filling its 30-day observation window: 16 days of observed history at the previous scan, 17 at this one. The score rises as the window fills, whether or not the server changes.

### 2026-09-08 (score 41, +1)

No change was recorded against any check on this day. Stability & Change Management went from 47 to 50. That category is still filling its 30-day observation window: 14 days of observed history at the previous scan, 15 at this one. The score rises as the window fills, whether or not the server changes.

### 2026-09-06 (score 40, +1)

No change was recorded against any check on this day. Stability & Change Management went from 40 to 43. That category is still filling its 30-day observation window: 12 days of observed history at the previous scan, 13 at this one. The score rises as the window fills, whether or not the server changes.

### 2026-09-04 (score 39, +1)

No change was recorded against any check on this day. Stability & Change Management went from 33 to 37. That category is still filling its 30-day observation window: 10 days of observed history at the previous scan, 11 at this one. The score rises as the window fills, whether or not the server changes.

## MCP tools (14)

### `list_metrics` (~20 tokens)

List built-in metrics and any custom metrics authored this session.

### `load_golden_set` (~84 tokens)

Load a golden set from a file path, a JSON array string, or JSONL text.
    Field names are normalized (question->input, answer->actual_output,
    ground_truth->expected_output, contexts->context, passages->retrieval_context, ...).

Input parameters:

- `fmt` (string)
- `name` (string, required)
- `source` (string, required)

### `list_golden_sets` (~18 tokens)

Show loaded golden sets and their sizes.

### `author_metric` (~70 tokens)

Author a custom metric from a plain-language description (+ optional golden
    examples). The judge converts your criteria into explicit evaluation steps,
    which run as a G-Eval-style scorer in run_eval / evaluate_case.

Input parameters:

- `criteria` (string, required)
- `examples`
- `name` (string, required)

### `run_eval` (~382 tokens)

Run metrics over a set of cases. Pass `cases` with the data inline (JSON,
    JSONL, CSV or TSV text, or a file path) to score in a single call — nothing is
    stored server-side and no prior load is needed. Or pass `golden_set` to reuse a
    set loaded earlier with load_golden_set. Inline `cases` is preferred: named sets
    live only in this server process, are shared with anyone else using it, and are
    lost when it restarts.
    All cases are scored, but only the
    `limit` lowest-scoring cases are returned by default (3) to keep replies short;
    `total_cases`/`shown` tell you how many more exist — call show_run_cases to page.
    Pass `generator_model` (which LLM produced the outputs) and `judge_model` (which
    LLM scored them) so the dashboard can compare across models; judge_model defaults
    to the configured judge. The run is saved to history (file or Supabase).
    IMPORTANT FOR CALLERS: the reply includes `view_url`, a link to the full
    visual report. ALWAYS include that link in your response to the user, even
    when you summarise everything else. `summary_md` is a ready-to-render
    markdown block (score table + link) that can be shown verbatim.
    Raises if the spend cap is hit mid-run; partial spend is still metered.

Input parameters:

- `cases` (string)
- `generator_model` (string)
- `golden_set` (string)
- `judge_model` (string)
- `label` (string)
- `limit` (integer)
- `metrics` (array, required)
- `outputs`
- `threshold` (number)

### `show_run_cases` (~73 tokens)

Page through the full per-case results of a saved run (the cases run_eval
    didn't show). Optionally sort by a single metric's score (worst first).

Input parameters:

- `limit` (integer)
- `metric` (string)
- `offset` (integer)
- `run_id` (string)

### `get_budget` (~30 tokens)

Current judge spend, the cap (RETRIEVAL_BUDGET_USD), and remaining headroom.

### `reset_budget` (~23 tokens)

Reset the cumulative spend ledger to $0 (the cap itself is unchanged).

### `evaluate_case` (~110 tokens)

Score a single output inline, without loading a golden set. The result is
    saved as a one-case run (golden set "inline") so it has a dashboard link;
    pass save=False to score without recording it.

Input parameters:

- `actual_output` (string, required)
- `context`
- `expected_output` (string)
- `input` (string, required)
- `metrics` (array, required)
- `retrieval_context`
- `save` (boolean)
- `threshold` (number)

### `list_runs` (~40 tokens)

List saved runs (most recent last) with their per-metric mean scores.

Input parameters:

- `golden_set` (string)
- `last_n` (integer)

### `plot_metric_trend` (~119 tokens)

One metric's mean score across recent runs, with a threshold reference.
    fmt="markdown" (default) returns a rich markdown report (tables, inline bars,
    case strip, distribution); fmt="text" returns plain ASCII;
    fmt="image" returns a PNG line chart (only useful where images display).

Input parameters:

- `fmt` (string)
- `golden_set` (string)
- `last_n` (integer)
- `metric` (string, required)
- `show_range` (boolean)
- `threshold` (number)

### `plot_run` (~102 tokens)

Every metric's mean score for a single run (default: latest).
    fmt="markdown" (default) returns a rich markdown report (tables, inline bars,
    case strip, distribution); fmt="text" returns plain ASCII;
    fmt="image" returns a PNG bar chart (only useful where images display).

Input parameters:

- `fmt` (string)
- `golden_set` (string)
- `run_id` (string)
- `threshold` (number)

### `compare_runs` (~85 tokens)

Compare several runs across all shared metrics.
    fmt="markdown" (default) returns a markdown matrix; fmt="text" returns plain ASCII;
    fmt="image" returns a PNG grouped-bar chart.

Input parameters:

- `fmt` (string)
- `golden_set` (string)
- `last_n` (integer)
- `run_ids`
- `threshold` (number)

### `ground_against_url` (~164 tokens)

Fetch a web page and check OUTPUT for *consistency* with it: faithfulness
    (every claim supported by the page) and, if a question is given, answer
    relevancy. Use it to ask "does my answer agree with what this page says?".

    IMPORTANT — this is NOT a correctness check. There are no labels here: the
    page is an unverified source, not ground truth. A high score means the output
    matches the page, not that either is right. The returned `caveat` says so on
    every call; surface it to the user.

Input parameters:

- `output` (string, required)
- `question` (string)
- `threshold` (number)
- `url` (string, required)

## Diagnostics

Captured diagnostic sections: Provenance. The full working is on the page: https://verifymcp.io/servers/hcarrillo001-retrieval-mcp/ghcr-io-hcarrillo001-retrieval-mcp-1-0-0#diagnostics

## Score history

- 2026-09-20: 46
- 2026-09-19: 46
- 2026-09-18: 45
- 2026-09-17: 45
- 2026-09-16: 45
- 2026-09-15: 44
- 2026-09-14: 44
- 2026-09-13: 43
- 2026-09-12: 43
- 2026-09-11: 42
- 2026-09-10: 42
- 2026-09-09: 41
- 2026-09-08: 41
- 2026-09-07: 40
- 2026-09-06: 40
- 2026-09-05: 39
- 2026-09-04: 39
- 2026-09-03: 38
- 2026-09-02: 38
- 2026-09-01: 38
- 2026-08-31: 34
- 2026-08-30: 34
- 2026-08-29: 34
- 2026-08-28: 34
- 2026-08-27: 34
- 2026-08-26: 34
- 2026-08-25: 33
- 2026-08-24: 33

## Common questions

### What is the retriEVAL MCP server?

retriEVAL is an MCP server listed in the public MCP registry as io.github.hcarrillo001/retrieval-mcp. LLM evals as MCP tools: score outputs for faithfulness, relevancy, and hallucination. This page covers its container image (ghcr.io/hcarrillo001/retrieval-mcp:1.0.0).

### Is the retriEVAL MCP server safe to use?

retriEVAL scores 46 out of 100 on VerifyMCP. That is a record of what we were able to check automatically, not an endorsement. The category breakdown on this page shows every signal behind the number, including the ones we could not confirm.

### What tools does the retriEVAL MCP server expose?

retriEVAL exposes 14 tools: list_metrics, load_golden_set, list_golden_sets, author_metric, run_eval, and 9 more. Their descriptions and schemas cost roughly 1,320 tokens of context every time the server is loaded.

### Is the retriEVAL MCP server still maintained?

retriEVAL is still listed as active in the MCP registry. We last reached this channel on 20 September 2026. Those dates come from our own scans of the registry and the channel itself, not from anything the publisher announced.

## Links

- Repository: https://github.com/hcarrillo001/retrieval-mcp
- Website: https://retrieval-mcp.com/
- Changelog RSS feed: https://verifymcp.io/servers/hcarrillo001-retrieval-mcp/ghcr-io-hcarrillo001-retrieval-mcp-1-0-0.xml
- Changelog JSON feed: https://verifymcp.io/servers/hcarrillo001-retrieval-mcp/ghcr-io-hcarrillo001-retrieval-mcp-1-0-0.json
- HTML version of this page: https://verifymcp.io/servers/hcarrillo001-retrieval-mcp/ghcr-io-hcarrillo001-retrieval-mcp-1-0-0