# io.github.MetriLLM/metrillm (npm · metrillm-mcp)

Benchmark local LLM models — speed, quality & hardware fitness verdict from any MCP client

- Trust score: 79/100 (medium)
- Change this week: +33
- Registry status: active
- Liveness: live
- Owner verified: no
- Last scored: 2026-08-03

## Components

- npm · `metrillm-mcp`: 79/100 (this document), [markdown](https://verifymcp.io/servers/metrillm-metrillm/metrillm-mcp.md), [page](https://verifymcp.io/servers/metrillm-metrillm/metrillm-mcp)

## Channel facts

- Registry: `npm`
- Package: `metrillm-mcp`
- Version: `0.2.6`
- Transport: `stdio`

## Trust breakdown

How this component scores in each security and reliability category. Every signal is checked automatically from public evidence about the published package, including repeated runs of it in an isolated sandbox, and we only credit what we can confirm. Scores are 0–100 per category. Scoring method: https://verifymcp.io/docs/scoring (what has changed: https://verifymcp.io/docs/scoring/changelog)

Scored 2026-08-03.

- **Supply Chain Security**: 87/100
  - No malware found by supply-chain analysis.
  - Only part of the dependency tree could be resolved (136 of 140), so this covers what we could see, not the whole tree.
  - No install/post-install scripts declared.
  - Only part of the dependency tree could be resolved (136 of 140), so this covers what we could see, not the whole tree.
- **Provenance & Transparency**: 97/100
  - Source repository is publicly reachable at the declared URL.
  - Cryptographically verified build provenance (signed, bound to MetriLLM/metrillm).
  - Clear OSI-approved license (Apache-2.0).
  - Actively maintained (last published 139 days ago).
  - Disclosure check failed: no security disclosure policy was found in the source repository.
- **Schema Quality & AI Usability**: 71/100
  - AI-judged instruction clarity (good).
  - Tool/resource definitions use about 373 tokens (~93/item across 4 items; 4 tools + 0 resources), lean.
  - Usage-examples check failed: none of the tools include examples.
- **Stability & Change Management**: 27/100
  - Stability observed for 8 of 30 days with no destabilising changes; credit accrues until the full window elapses.
- **Tool Coverage**: 100/100
  - 100% of tools have a non-trivial description (not blank, and not just the tool's name).
  - 100% of tool parameters carry a description.
- **Capabilities**: 100/100
  - Implements a supported MCP spec version (2025-11-25); the latest is 2026-07-28.

## Install

### Claude

```bash
claude mcp add metrillm-metrillm -- npx -y metrillm-mcp
```

### Codex

```bash
codex mcp add metrillm-metrillm -- npx -y metrillm-mcp
```

### opencode

```json
{
  "$schema": "https://opencode.ai/config.json",
  "mcp": {
    "metrillm-metrillm": {
      "type": "local",
      "command": [
        "npx",
        "-y",
        "metrillm-mcp"
      ],
      "enabled": true
    }
  }
}
```

### OpenClaw

```bash
openclaw mcp add metrillm-metrillm --command npx --arg -y --arg metrillm-mcp
```

### Hermes

```yaml
mcp_servers:
  metrillm-metrillm:
    command: "npx"
    args: ["-y", "metrillm-mcp"]
```

### Other

```json
{
  "mcpServers": {
    "metrillm-metrillm": {
      "command": "npx",
      "args": [
        "-y",
        "metrillm-mcp"
      ]
    }
  }
}
```

## Changelog

Every change recorded for this component, newest first. Days that predate change tracking, or that we cannot explain, say so: "we were watching and nothing happened" and "we were not watching" are different claims.

### 2026-08-03 (score 79, +1)

No change was recorded against any check on this day. Stability & Change Management went from 23 to 27. That category is still filling its 30-day observation window: 7 days of observed history at the previous scan, 8 at this one. The score rises as the window fills, whether or not the server changes.

### 2026-08-02 (score 78, +57)

- [security improvement] Install scripts: unverified → pass
- [security improvement] Provenance: unverified → pass
- [security improvement] Known CVEs: unverified → partial
- [security improvement] Malware scan: unverified → pass
- [security] The attested source repository moved: MetriLLM/metrillm
- [functional regression] Tool coverage: 100 → unverified
- [functional improvement] Maintenance: unverified → pass
- [functional improvement] MCP protocol: unverified → pass
- [functional improvement] Stability: unverified → 0.23
- [functional improvement] Schema quality: unverified → good
- [functional improvement] License: unverified → pass
- [functional improvement] Dependency health: unverified → partial
- [functional] First check of Schema quality: unverified
- [functional] Licence: Apache-2.0

### 2026-07-31 (score 21, −25)

- [functional] We updated how we score, so this day's move reflects our rubric, not a change to the server

### 2026-07-27 (score 46)

First indexed and scored.

## MCP tools (4)

### `list_models` (~60 tokens)

List all LLM models available locally on the inference runtime (e.g. Ollama). Returns model name, size, parameter count, quantization, and family.

Input parameters:

- `runtime` (string): Inference runtime to use (ollama | lm-studio).

### `run_benchmark` (~150 tokens)

Run an MetriLLM benchmark on a local model. Measures performance (tokens/s, TTFT, memory) and optionally quality (reasoning, math, coding, instruction following, structured output, multilingual). Returns a detailed fitness verdict. Warning: benchmarks take 30s to 5+ minutes depending on model size.

Input parameters:

- `model` (string, required): Model name to benchmark (e.g. 'llama3.2:3b', 'qwen2.5:7b')
- `perfOnly` (boolean): If true, skip quality benchmarks and only measure speed/memory/TTFT
- `runtime` (string): Inference runtime to use (ollama | lm-studio).

### `get_results` (~70 tokens)

Retrieve previously saved benchmark results from ~/.metrillm/results/. Optionally filter by model name. Returns an array of full benchmark result objects.

Input parameters:

- `model` (string): Filter results by model name (substring match)
- `runtime` (string): Inference runtime to use (ollama | lm-studio).

### `share_result` (~93 tokens)

Upload a benchmark result to the public MetriLLM leaderboard. Uses official upload defaults; METRILLM_* environment variables can override for self-hosted deployments. The resultFile must be an absolute path to a JSON file in ~/.metrillm/results/.

Input parameters:

- `resultFile` (string, required): Absolute path to a benchmark result JSON file (from ~/.metrillm/results/). Must be inside ~/.metrillm/results/.

## Diagnostics

Captured diagnostic sections: Provenance, Dependencies. The full working is on the page: https://verifymcp.io/servers/metrillm-metrillm/metrillm-mcp#diagnostics

## Score history

- 2026-08-03: 79
- 2026-08-02: 78
- 2026-08-01: 21
- 2026-07-31: 21
- 2026-07-30: 46
- 2026-07-28: 46
- 2026-07-27: 46

## Links

- npm package: https://www.npmjs.com/package/metrillm-mcp
- Socket report: https://socket.dev/npm/package/metrillm-mcp
- Repository: https://github.com/MetriLLM/metrillm
- Changelog RSS feed: https://verifymcp.io/servers/metrillm-metrillm/metrillm-mcp/changelog.xml
- Changelog JSON feed: https://verifymcp.io/servers/metrillm-metrillm/metrillm-mcp/changelog.json
- HTML version of this page: https://verifymcp.io/servers/metrillm-metrillm/metrillm-mcp