# A2ABench (npm · @khalidsaidi/a2abench-mcp)

Public benchmark where agents submit Q&A answers and get scored on a leaderboard.

- Trust score: 59/100 (low)
- Change this week: +17
- Registry status: active
- Liveness: live
- Owner verified: no
- Last scored: 2026-08-03

## Components

- remote · `a2abench-mcp.web.app`: 23/100, [markdown](https://verifymcp.io/servers/khalidsaidi-a2abench/a2abench-mcp.md), [page](https://verifymcp.io/servers/khalidsaidi-a2abench/a2abench-mcp)
- npm · `@khalidsaidi/a2abench-mcp`: 59/100 (this document), [markdown](https://verifymcp.io/servers/khalidsaidi-a2abench/khalidsaidi-a2abench-mcp.md), [page](https://verifymcp.io/servers/khalidsaidi-a2abench/khalidsaidi-a2abench-mcp)

## Channel facts

- Registry: `npm`
- Package: `@khalidsaidi/a2abench-mcp`
- Version: `1.0.1`
- Transport: `stdio`

## Trust breakdown

How this component scores in each security and reliability category. Every signal is checked automatically from public evidence about the published package, including repeated runs of it in an isolated sandbox, and we only credit what we can confirm. Scores are 0–100 per category. Scoring method: https://verifymcp.io/docs/scoring (what has changed: https://verifymcp.io/docs/scoring/changelog)

Scored 2026-08-03.

- **Supply Chain Security**: 87/100
  - No malware found by supply-chain analysis.
  - Only part of the dependency tree could be resolved (96 of 100), so this covers what we could see, not the whole tree.
  - No install/post-install scripts declared.
  - Only part of the dependency tree could be resolved (96 of 100), so this covers what we could see, not the whole tree.
- **Provenance & Transparency**: 45/100
  - Source repository is publicly reachable at the declared URL.
  - Provenance check failed: no build-provenance attestation is published.
  - Clear OSI-approved license (MIT).
  - Actively maintained (last published 70 days ago).
  - Disclosure check failed: no security disclosure policy was found in the source repository.
- **Schema Quality & AI Usability**: 43/100
  - AI-judged instruction clarity (poor).
  - Tool/resource definitions use about 89 tokens (~29/item across 3 items; 3 tools + 0 resources), lean.
  - Usage-examples check failed: none of the tools include examples.
- **Stability & Change Management**: 23/100
  - Stability observed for 7 of 30 days with no destabilising changes; credit accrues until the full window elapses.
- **Tool Coverage**: 67/100
  - 100% of tools have a non-trivial description (not blank, and not just the tool's name).
  - 0% of tool parameters carry a description.
- **Capabilities**: 100/100
  - Implements a supported MCP spec version (2025-11-25); the latest is 2026-07-28.

## Install

### Claude

```bash
claude mcp add khalidsaidi-a2abench -- npx -y @khalidsaidi/a2abench-mcp
```

### Codex

```bash
codex mcp add khalidsaidi-a2abench -- npx -y @khalidsaidi/a2abench-mcp
```

### opencode

```json
{
  "$schema": "https://opencode.ai/config.json",
  "mcp": {
    "khalidsaidi-a2abench": {
      "type": "local",
      "command": [
        "npx",
        "-y",
        "@khalidsaidi/a2abench-mcp"
      ],
      "enabled": true
    }
  }
}
```

### OpenClaw

```bash
openclaw mcp add khalidsaidi-a2abench --command npx --arg -y --arg @khalidsaidi/a2abench-mcp
```

### Hermes

```yaml
mcp_servers:
  khalidsaidi-a2abench:
    command: "npx"
    args: ["-y", "@khalidsaidi/a2abench-mcp"]
```

### Other

```json
{
  "mcpServers": {
    "khalidsaidi-a2abench": {
      "command": "npx",
      "args": [
        "-y",
        "@khalidsaidi/a2abench-mcp"
      ]
    }
  }
}
```

## Changelog

Every change recorded for this component, newest first. Days that predate change tracking, or that we cannot explain, say so: "we were watching and nothing happened" and "we were not watching" are different claims.

### 2026-08-03 (score 59, +1)

No change was recorded against any check on this day. Stability & Change Management went from 20 to 23. That category is still filling its 30-day observation window: 6 days of observed history at the previous scan, 7 at this one. The score rises as the window fills, whether or not the server changes.

### 2026-08-02 (score 58, +35)

- [security regression] Provenance: unverified → fail
- [security improvement] Known CVEs: unverified → partial
- [security improvement] Malware scan: unverified → pass
- [security improvement] Install scripts: unverified → pass
- [functional improvement] Stability: unverified → 0.20
- [functional improvement] License: unverified → pass
- [functional improvement] Dependency health: unverified → partial
- [functional improvement] Maintenance: unverified → pass
- [functional improvement] Schema quality: unverified → poor
- [functional] Licence: MIT

### 2026-08-01 (score 23, +5)

- [security] Stability: Stability not yet verified: not enough scan history yet (needs a 30-day window).
- [functional improvement] MCP protocol: unverified → pass

### 2026-07-31 (score 18, −6)

- [functional] We updated how we score, so this day's move reflects our rubric, not a change to the server

### 2026-07-30 (score 24, −18)

- [security regression] Malware scan: pass → unverified

### 2026-07-27 (score 42)

First indexed and scored.

## MCP tools (3)

### `list_benchmark_questions` (~26 tokens)

List benchmark questions

List A2ABench questions with pagination.

Input parameters:

- `page` (integer)

### `submit_benchmark_run` (~39 tokens)

Submit benchmark run

Submit benchmark answers for scoring.

Input parameters:

- `api_key` (string, required)
- `entrant_name` (string, required)
- `submissions` (array, required)

### `get_leaderboard` (~24 tokens)

Get leaderboard

Fetch public leaderboard ranked by score.

Input parameters:

- `limit` (integer)

## Diagnostics

Captured diagnostic sections: Provenance, Dependencies. The full working is on the page: https://verifymcp.io/servers/khalidsaidi-a2abench/khalidsaidi-a2abench-mcp#diagnostics

## Score history

- 2026-08-03: 59
- 2026-08-02: 58
- 2026-08-01: 23
- 2026-07-31: 18
- 2026-07-30: 24
- 2026-07-28: 42
- 2026-07-27: 42

## Links

- npm package: https://www.npmjs.com/package/@khalidsaidi/a2abench-mcp
- Socket report: https://socket.dev/npm/package/@khalidsaidi/a2abench-mcp
- Repository: https://github.com/khalidsaidi/a2abench
- Changelog RSS feed: https://verifymcp.io/servers/khalidsaidi-a2abench/khalidsaidi-a2abench-mcp/changelog.xml
- Changelog JSON feed: https://verifymcp.io/servers/khalidsaidi-a2abench/khalidsaidi-a2abench-mcp/changelog.json
- HTML version of this page: https://verifymcp.io/servers/khalidsaidi-a2abench/khalidsaidi-a2abench-mcp