A2ABench
NPM · @KHALIDSAIDI/A2ABENCH-MCP · 2 COMPONENTS · SCANNED SEP 20
Public benchmark where agents submit Q&A answers and get scored on a leaderboard.
Available components
How this component scores in each security and reliability category. Every signal is checked automatically from public evidence about the published package, including repeated runs of it in an isolated sandbox, and we only credit what we can confirm. How we score → Why this is hard to score →
Supply Chain Security98
- No malware found by supply-chain analysis.Pass
- No known CVEs affecting this package version or its production dependencies.Pass
- No install/post-install scripts declared.Pass
- 31 of 97 dependencies flagged as unhealthy. View diagnostics → Partial
Provenance & Transparency48
- Source repository is publicly reachable at the declared URL. View diagnostics → Pass
- Provenance check failed: no build-provenance attestation is published. See how to fix → View diagnostics → Fail
- Clear OSI-approved license (MIT).Pass
- Actively maintained (last published 118 days ago).Pass
- Publishes a security disclosure policy (SECURITY.md).Pass
Schema Quality & AI Usability63
- AI-judged instruction clarity (good).Pass
- Tool/resource definitions use about 89 tokens (~29/item across 3 items; 3 tools + 0 resources), lean.Pass
- Usage-examples check failed: none of the tools include examples. See how to fix → Fail
Stability & Change Management93
- Stability observed for 28 of 30 days with no destabilising changes; credit accrues until the full window elapses.Partial
Tool Coverage67
- 100% of tools have a non-trivial description (not blank, and not just the tool's name).Pass
- 0% of tool parameters carry a description.Fail
Tool Safety100
- No prompt-injection markers were found in the server instructions, tool names or descriptions we captured.Pass
- We read all 3 captured tool definition(s), and no name or description among them implies an irreversible operation.Pass
- An AI judge read all 3 captured unit(s) of tool text and found none that tries to manipulate the model reading it.Pass
Capabilities100
- Implements a supported MCP spec version (2025-11-25); the latest is 2026-07-28.Pass
How do I install the A2ABench MCP server?
A2ABench runs locally as an npm package, launched with npx -y @khalidsaidi/a2abench-mcp. Ready-made configuration for Claude, Cursor, VS Code, Codex and 5 more is on this page, copied from each client's own documentation.
npm · @khalidsaidi/a2abench-mcp
claude mcp add khalidsaidi-a2abench -- npx -y @khalidsaidi/a2abench-mcp
{
"mcpServers": {
"khalidsaidi-a2abench": {
"command": "npx",
"args": [
"-y",
"@khalidsaidi/a2abench-mcp"
]
}
}
} {
"servers": {
"khalidsaidi-a2abench": {
"command": "npx",
"args": [
"-y",
"@khalidsaidi/a2abench-mcp"
]
}
}
} codex mcp add khalidsaidi-a2abench -- npx -y @khalidsaidi/a2abench-mcp
{
"$schema": "https://opencode.ai/config.json",
"mcp": {
"khalidsaidi-a2abench": {
"type": "local",
"command": [
"npx",
"-y",
"@khalidsaidi/a2abench-mcp"
],
"enabled": true
}
}
} openclaw mcp add khalidsaidi-a2abench --command npx --arg -y --arg @khalidsaidi/a2abench-mcp
mcp_servers:
khalidsaidi-a2abench:
command: "npx"
args: ["-y", "@khalidsaidi/a2abench-mcp"] {
"McpServers": {
"khalidsaidi-a2abench": {
"Transport": "stdio",
"Command": "npx",
"Arguments": [
"-y",
"@khalidsaidi/a2abench-mcp"
]
}
}
} assistant mcp add khalidsaidi-a2abench -t stdio -c npx -a -y @khalidsaidi/a2abench-mcp
{
"mcpServers": {
"khalidsaidi-a2abench": {
"command": "npx",
"args": [
"-y",
"@khalidsaidi/a2abench-mcp"
]
}
}
} Every change we have recorded for this component, newest first. Security-relevant changes are always shown. ▲ marks a change for the better, ▼ a change for the worse; unmarked changes are neutral.
- 20 Sept 26 +1
No change was recorded against any check on this day. Stability & Change Management went from 90 to 93. That category is still filling its 30-day observation window: 27 days of observed history at the previous scan, 28 at this one. The score rises as the window fills, whether or not the server changes.
- 19 Sept 26 +1
- Security disclosure: unverified → pass ▲ functional
- 18 Sept 26 0
- Security disclosure: pass → unverified ▼ functional
- 16 Sept 26 −3
- Stability: pass → 0.80 functional
- 15 Sept 26 +1
- Stability: 0.97 → pass security
- 13 Sept 26 +1
No change was recorded against any check on this day. Stability & Change Management went from 90 to 93. That category is still filling its 30-day observation window: 27 days of observed history at the previous scan, 28 at this one. The score rises as the window fills, whether or not the server changes.
- 11 Sept 26 +1
No change was recorded against any check on this day. Stability & Change Management went from 83 to 87. That category is still filling its 30-day observation window: 25 days of observed history at the previous scan, 26 at this one. The score rises as the window fills, whether or not the server changes.
- 9 Sept 26 −3
- Stability: pass → 0.80 functional
Diagnostic detail from the automated scan of this channel: what the scanner observed at each step, so you can see exactly where a check passed or failed. It is informational only and never changes the trust score.
Captured 20 Sept 2026 · Analysed npm/@khalidsaidi/a2abench-mcp@1.0.1
Provenance No attestation
The registry publishes no build provenance for this version, so there is nothing to verify.
| Result | No attestation |
|---|---|
| Ecosystem | npm |
Background: How many MCP packages publish verified provenance →
Dependencies 97 packages
| Packages resolved | 97 |
|---|---|
| Stale | 31 |
| Tree resolution | Complete |
Background: SBOMs and build attestations, explained →
The tools this component advertises to a client, with an estimated token cost for each. Expand a tool to see its parameters and schema. The per-tool counts are indicative and are not scored directly; the schema's total context footprint is one signal in Schema Quality & AI Usability. A tool's description is untrusted text the model reads on every call, which is what makes this list a security surface and not just an inventory: how tool poisoning works →
get_leaderboard Get leaderboard ~24
Fetch public leaderboard ranked by score.
| Name | Type | Req | Description |
|---|---|---|---|
| limit | integer | – | – |
No output schema declared.
No examples provided.
list_benchmark_questions List benchmark questions ~26
List A2ABench questions with pagination.
| Name | Type | Req | Description |
|---|---|---|---|
| page | integer | – | – |
No output schema declared.
No examples provided.
submit_benchmark_run Submit benchmark run ~39
Submit benchmark answers for scoring.
| Name | Type | Req | Description |
|---|---|---|---|
| api_key | string | yes | – |
| entrant_name | string | yes | – |
| submissions | array | yes | – |
No output schema declared.
No examples provided.
What is the A2ABench MCP server?
A2ABench is an MCP server listed in the public MCP registry as io.github.khalidsaidi/a2abench. Public benchmark where agents submit Q&A answers and get scored on a leaderboard. This page covers its npm package (@khalidsaidi/a2abench-mcp).
Is the A2ABench MCP server safe to use?
A2ABench scores 79 out of 100 on VerifyMCP. We found no known CVEs affecting it as of 20 September 2026. It declares no install or post-install scripts. That is a record of what we were able to check automatically, not an endorsement. The category breakdown on this page shows every signal behind the number, including the ones we could not confirm.
What tools does the A2ABench MCP server expose?
A2ABench exposes 3 tools: list_benchmark_questions, submit_benchmark_run, get_leaderboard. Their descriptions and schemas cost roughly 89 tokens of context every time the server is loaded.
Is the A2ABench MCP server still maintained?
A2ABench is still listed as active in the MCP registry. We last reached this channel on 20 September 2026. Those dates come from our own scans of the registry and the channel itself, not from anything the publisher announced.
What licence is the A2ABench MCP server under?
A2ABench declares the MIT licence, which is OSI-approved. That covers the source only, and says nothing about the cost of any service it calls.