io.github.MetriLLM/metrillm
NPM · METRILLM-MCP
Benchmark local LLM models — speed, quality & hardware fitness verdict from any MCP client
Available components
Last scanned (18 hours ago)
Scores higher than 93% of npm servers
Category AI and Machine Learning
How this component scores in each security and reliability category. Every signal is checked automatically from public evidence about the published package, including repeated runs of it in an isolated sandbox, and we only credit what we can confirm. How we score → Why this is hard to score →
Supply Chain Security81
- No malware found by supply-chain analysis.Pass
- CVE check failed: a known high-severity CVE affects zod 3.25.76, a direct dependency. A fixed version is available. View diagnostics → Fail
- No install/post-install scripts declared.Pass
- 39 of 134 dependencies flagged as unhealthy. View diagnostics → Partial
Provenance & Transparency99
- Source repository is publicly reachable at the declared URL. View diagnostics → Pass
- Cryptographically verified build provenance (signed, bound to MetriLLM/metrillm). View diagnostics → Pass
- Clear OSI-approved license (Apache-2.0).Pass
- Actively maintained (last published 232 days ago).Pass
- Publishes a security disclosure policy (SECURITY.md).Pass
Schema Quality & AI Usability81
- AI-judged instruction clarity (excellent).Pass
- Tool/resource definitions use about 373 tokens (~93/item across 4 items; 4 tools + 0 resources), lean.Pass
- Usage-examples check failed: none of the tools include examples. See how to fix → Fail
Stability & Change Management97
- Stability observed for 29 of 30 days with no destabilising changes; credit accrues until the full window elapses.Partial
Tool Coverage100
- 100% of tools have a non-trivial description (not blank, and not just the tool's name).Pass
- 100% of tool parameters carry a description.Pass
Tool Safety100
- No prompt-injection markers were found in the server instructions, tool names or descriptions we captured.Pass
- We read all 4 captured tool definition(s), and no name or description among them implies an irreversible operation.Pass
- An AI judge read all 4 captured unit(s) of tool text and found none that tries to manipulate the model reading it.Pass
Capabilities100
- Implements a supported MCP spec version (2025-11-25); the latest is 2026-07-28.Pass
Every change we have recorded for this component, newest first. Security-relevant changes are always shown. ▲ marks a change for the better, ▼ a change for the worse; unmarked changes are neutral.
- 3 Nov 26 −1
- Stability: pass → 0.93 functional
- 31 Oct 26 +1
- Security disclosure: unverified → pass ▲ functional
- 29 Oct 26 −1
- Security disclosure: pass → unverified ▼ functional
- 28 Oct 26 +1
- Stability: 0.97 → pass security
- 27 Oct 26 −1
- Stability: pass → 0.97 functional
- 24 Oct 26 −5
- CVE-2023-54404 affects this package: high ▼ security
- Known CVEs: pass → fail ▼ security
- 21 Oct 26 +1
- Stability: 0.97 → pass security
- Security disclosure: unverified → pass ▲ functional
- 20 Oct 26 −1
- Security disclosure: pass → unverified ▼ functional
- Stability: pass → 0.97 functional
Diagnostic detail from the automated scan of this channel: what the scanner observed at each step, so you can see exactly where a check passed or failed. It is informational only and never changes the trust score.
Captured 4 Nov 2026 · Analysed npm/metrillm-mcp@0.2.6
Provenance Verified
A signed build attestation was found and verified, binding this exact artifact to the source repository it claims to come from.
| Result | Verified |
|---|---|
| Ecosystem | npm |
| Reason | Verified |
| Discovered via | Registry attestation endpoint |
| Source repo | MetriLLM/metrillm |
| Certificate issuer | https://token.actions.githubusercontent.com |
| Certificate SAN | https://github.com/MetriLLM/metrillm/.github/workflows/release.yml@refs/tags/v0.2.6 |
| Rekor log index | 1109821839 |
| Predicate type | SLSA build provenance https://slsa.dev/provenance/v1 |
| Subject digest | sha512:22d38759c6bb5219279d56f1f5a8814881fad840d166304464329fe77ded592d22fa1532125c2c53ed4fa4898364573c3a2878a683010445e1cddfa8b |
Background: How many MCP packages publish verified provenance →
Vulnerabilities 1 finding
| ID | CVE | Severity | Vector | Fix available |
|---|---|---|---|---|
| GHSA-mrhx-c7xc-m3w5 | CVE-2023-54404 | high | CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:H | yes |
Background: What a vulnerability scan can and cannot prove →
Dependencies 134 packages
| Packages resolved | 134 |
|---|---|
| Stale | 39 |
| Tree resolution | Complete |
Background: SBOMs and build attestations, explained →
The tools this component advertises to a client, with an estimated token cost for each. Expand a tool to see its parameters and schema. The per-tool counts are indicative and are not scored directly; the schema's total context footprint is one signal in Schema Quality & AI Usability. A tool's description is untrusted text the model reads on every call, which is what makes this list a security surface and not just an inventory: how tool poisoning works →
get_results ~70
Retrieve previously saved benchmark results from ~/.metrillm/results/. Optionally filter by model name. Returns an array of full benchmark result objects.
| Name | Type | Req | Description |
|---|---|---|---|
| model | string | – | Filter results by model name (substring match) |
| runtime | string | – | Inference runtime to use (ollama | lm-studio). |
No output schema declared.
No examples provided.
list_models ~60
List all LLM models available locally on the inference runtime (e.g. Ollama). Returns model name, size, parameter count, quantization, and family.
| Name | Type | Req | Description |
|---|---|---|---|
| runtime | string | – | Inference runtime to use (ollama | lm-studio). |
No output schema declared.
No examples provided.
run_benchmark ~150
Run an MetriLLM benchmark on a local model. Measures performance (tokens/s, TTFT, memory) and optionally quality (reasoning, math, coding, instruction following, structured output, multilingual). Returns a detailed fitness verdict. Warning: benchmarks take 30s to 5+ minutes depending on model size.
| Name | Type | Req | Description |
|---|---|---|---|
| model | string | yes | Model name to benchmark (e.g. 'llama3.2:3b', 'qwen2.5:7b') |
| perfOnly | boolean | – | If true, skip quality benchmarks and only measure speed/memory/TTFT |
| runtime | string | – | Inference runtime to use (ollama | lm-studio). |
No output schema declared.
No examples provided.
share_result ~93
Upload a benchmark result to the public MetriLLM leaderboard. Uses official upload defaults; METRILLM_* environment variables can override for self-hosted deployments. The resultFile must be an absolute path to a JSON file in ~/.metrillm/results/.
| Name | Type | Req | Description |
|---|---|---|---|
| resultFile | string | yes | Absolute path to a benchmark result JSON file (from ~/.metrillm/results/). Must be inside ~/.metrillm/results/. |
No output schema declared.
No examples provided.
How do I install the io.github.MetriLLM/metrillm MCP server?
io.github.MetriLLM/metrillm runs locally as an npm package, launched with npx -y metrillm-mcp. Ready-made configuration for Claude, Cursor, VS Code, Codex and 5 more is on this page, copied from each client's own documentation.
npm · metrillm-mcp
claude mcp add metrillm-metrillm -- npx -y metrillm-mcp
What is the io.github.MetriLLM/metrillm MCP server?
io.github.MetriLLM/metrillm is an MCP server listed in the public MCP registry as io.github.MetriLLM/metrillm. Benchmark local LLM models, speed, quality & hardware fitness verdict from any MCP client. This page covers its npm package (metrillm-mcp).
Is the io.github.MetriLLM/metrillm MCP server safe to use?
io.github.MetriLLM/metrillm scores 90 out of 100 on VerifyMCP. We recorded 1 known advisory against it as of 4 November 2026. It declares no install or post-install scripts. Its build provenance is signed and verified. That is a record of what we were able to check automatically, not an endorsement. The category breakdown on this page shows every signal behind the number, including the ones we could not confirm.
What tools does the io.github.MetriLLM/metrillm MCP server expose?
io.github.MetriLLM/metrillm exposes 4 tools: list_models, run_benchmark, get_results, share_result. Their descriptions and schemas cost roughly 373 tokens of context every time the server is loaded.
Is the io.github.MetriLLM/metrillm MCP server still maintained?
io.github.MetriLLM/metrillm is still listed as active in the MCP registry. We last reached this channel on 4 November 2026. Those dates come from our own scans of the registry and the channel itself, not from anything the publisher announced.
What licence is the io.github.MetriLLM/metrillm MCP server under?
io.github.MetriLLM/metrillm declares the Apache-2.0 licence, which is OSI-approved. That covers the source only, and says nothing about the cost of any service it calls.