The Aggregate — LLM benchmark aggregate
REMOTE · THEAGGREGATE.AI · SCANNED SEP 24
Fused LLM rankings: one IRT/Elo scale across ~5,000 public benchmark leaderboards, updated daily.
Available components
How this component scores in each security and reliability category. Every signal is checked automatically against the live server, and we only credit what we can confirm. How we score → Why this is hard to score →
Endpoint Security60
- The endpoint's TLS certificate is valid, in date, and uses a strong key. View diagnostics → Pass
- Authorisation not fully verified: no authorisation is required to call this server, and 8 tool(s) never declared a destructiveHint. The MCP spec treats an absent hint as destructive by default, so we cannot call this surface safe. See how to fix → View diagnostics → Unverified
- HTTPS is enforced; there's no plaintext access path. View diagnostics → Pass
- HSTS check failed: the Strict-Transport-Security header is absent. See how to fix → View diagnostics → Fail
- DNSSEC is configured correctly; the domain's records validate against the full chain to the root. View diagnostics → Pass
Transport & Reachability100
- Verified streamable-http transport via a live MCP handshake. View diagnostics → Pass
Schema Quality & AI Usability81
- AI-judged instruction clarity (excellent).Pass
- Tool/resource definitions use about 732 tokens (~91/item across 8 items; 8 tools + 0 resources), lean.Pass
- Usage-examples check failed: none of the tools include examples. See how to fix → Fail
Stability & Change Management100
- No destabilizing schema changes in the last 30 days.Pass
Tool Coverage100
- 100% of tools have a non-trivial description (not blank, and not just the tool's name).Pass
- 100% of tool parameters carry a description.Pass
Tool Safety100
- No prompt-injection markers were found in the server instructions, tool names or descriptions we captured.Pass
- We read all 8 captured tool definition(s), and no name or description among them implies an irreversible operation.Pass
- An AI judge read all 9 captured unit(s) of tool text and found none that tries to manipulate the model reading it.Pass
Capabilities60
- Spec-recency check failed: implements MCP spec 2025-06-18; the latest is 2026-07-28. See how to fix → Fail
How do I install the The Aggregate — LLM benchmark aggregate MCP server?
The Aggregate — LLM benchmark aggregate is a hosted endpoint at https://theaggregate.ai/mcp, so there is nothing to install locally. Ready-made configuration for Claude, Cursor, VS Code, Codex and 5 more is on this page, copied from each client's own documentation.
remote · theaggregate.ai
claude mcp add --transport http ai-theaggregate-the-aggregate 'https://theaggregate.ai/mcp'
{
"mcpServers": {
"ai-theaggregate-the-aggregate": {
"url": "https://theaggregate.ai/mcp"
}
}
} {
"servers": {
"ai-theaggregate-the-aggregate": {
"type": "http",
"url": "https://theaggregate.ai/mcp"
}
}
} [mcp_servers.ai-theaggregate-the-aggregate] url = "https://theaggregate.ai/mcp"
{
"$schema": "https://opencode.ai/config.json",
"mcp": {
"ai-theaggregate-the-aggregate": {
"type": "remote",
"url": "https://theaggregate.ai/mcp",
"enabled": true
}
}
} openclaw mcp add ai-theaggregate-the-aggregate --url 'https://theaggregate.ai/mcp' --transport streamable-http
mcp_servers:
ai-theaggregate-the-aggregate:
url: "https://theaggregate.ai/mcp" {
"McpServers": {
"ai-theaggregate-the-aggregate": {
"Transport": "http",
"Url": "https://theaggregate.ai/mcp"
}
}
} assistant mcp add ai-theaggregate-the-aggregate -t streamable-http -u 'https://theaggregate.ai/mcp'
{
"mcpServers": {
"ai-theaggregate-the-aggregate": {
"type": "http",
"url": "https://theaggregate.ai/mcp"
}
}
} The mcpServers block is a cross-client convention. Remote transports vary, so check your client's docs.
Every change we have recorded for this component, newest first. Security-relevant changes are always shown. ▲ marks a change for the better, ▼ a change for the worse; unmarked changes are neutral.
- 12 Sept 26 0
- The server rewrote its instructions, which are the text every model session reads security
- Tool “get_leaderboard” rewrote its description, which is the text the model reads security
- 8 Sept 26 0
- The server rewrote its instructions, which are the text every model session reads security
- Tool “get_leaderboard” rewrote its description, which is the text the model reads security
- 26 Aug 26 79
- We updated how we score, so this day's move reflects our rubric, not a change to the server See what changed → functional
- 25 Aug 26 0
- Stability: 0.97 → pass security
- 16 Aug 26 0
- Tool “get_leaderboard” rewrote its description, which is the text the model reads security
- Tool “search_models” rewrote its description, which is the text the model reads security
- Schema quality: 76 → 90 ▼ functional
- “get_leaderboard” added an optional parameter “include_variants” cosmetic
- “search_models” added an optional parameter “include_variants” cosmetic
- 11 Aug 26 0
- We updated how we score, so this day's move reflects our rubric, not a change to the server See what changed → functional
- 31 Jul 26 0
- We updated how we score, so this day's move reflects our rubric, not a change to the server See what changed → functional
- 30 Jul 26 0
- We updated how we score, so this day's move reflects our rubric, not a change to the server See what changed → functional
Diagnostic detail from the automated scan of this channel: what the scanner observed at each step, so you can see exactly where a check passed or failed. It is informational only and never changes the trust score.
Captured 24 Sept 2026 · Probed https://theaggregate.ai/mcp
TLS valid
Negotiated TLS 1.3 with TLS_AES_128_GCM_SHA256 .
| Subject | Issuer | Valid from | Valid until | Key | Signature | Serial |
|---|---|---|---|---|---|---|
| CN=theaggregate.ai | CN=WE1,O=Google Trust Services,C=US | 19 Sept 2026 | 18 Dec 2026 | ECDSA 256 | ECDSA-SHA256 | ee872805f1dd786513d5e07b0997af72 |
| SANs: theaggregate.ai, *.theaggregate.ai | ||||||
| CN=WE1,O=Google Trust Services,C=US (CA) | CN=GTS Root R4,O=Google Trust Services LLC,C=US | 13 Dec 2023 | 20 Feb 2029 | ECDSA 256 | ECDSA-SHA384 | 7ff31977972c224a76155d13b6d685e3 |
| CN=GTS Root R4,O=Google Trust Services LLC,C=US (CA) | CN=GlobalSign Root CA,OU=Root CA,O=GlobalSign nv-sa,C=BE | 15 Nov 2023 | 28 Jan 2028 | ECDSA 384 | SHA256-RSA | 7fe530bf331343bedd821610493d8a1b |
Background: What to check on a remote MCP endpoint →
DNSSEC secure
Validation of theaggregate.ai. — Secure
| Zone | DS | Keys | Algorithms | Outcome |
|---|---|---|---|---|
| . | trust_anchor | 20326, 38696 | 8, 8 | Verified |
| ai. | present | 3799 | 8 | Verified |
| theaggregate.ai. | present | 2371 | 13 | Verified |
| theaggregate.ai. | Verified address RRset verified with the apex keys |
Authentication No authorisation required
The endpoint answered without asking for a token. Anyone who knows the URL can reach it.
| Result | No authorisation required |
|---|---|
| HTTP status | 200 |
Background: How OAuth 2.1 works in the 2026 MCP spec →
Transports 2 probes
| Transport | URL | Outcome | Status | Location |
|---|---|---|---|---|
| streamable-http | https://theaggregate.ai/mcp | Verified | 200 | |
| http (plaintext) | http://theaggregate.ai/mcp | HTTPS enforced | 308 | https://theaggregate.ai/mcp |
The tools this component advertises to a client, with an estimated token cost for each. Expand a tool to see its parameters and schema. The per-tool counts are indicative and are not scored directly; the schema's total context footprint is one signal in Schema Quality & AI Usability. A tool's description is untrusted text the model reads on every call, which is what makes this list a security surface and not just an inventory: how tool poisoning works →
about_the_aggregate About The Aggregate ~35
What this data is: how the IRT fusion works, current coverage counts, update cadence, and how to cite it.
Input schema present but exposes no named parameters.
No output schema declared.
No examples provided.
compare_models Compare models ~54
Head-to-head between 2-4 models: aggregate ranks, Elo gap with a significance note based on the standard errors, and notable benchmarks they share.
| Name | Type | Req | Description |
|---|---|---|---|
| models | array | yes | Two to four model names or slugs. |
No output schema declared.
No examples provided.
get_benchmark Benchmark detail ~88
One benchmark in depth: what it measures, the original source leaderboard URL, IRT stats (difficulty, noise, model coverage), skill weights, and the current top models on it.
| Name | Type | Req | Description |
|---|---|---|---|
| benchmark | string | yes | Benchmark name or slug, e.g. "Aider polyglot". |
| top | number | – | How many top models to list (1-50, default 10). |
No output schema declared.
No examples provided.
get_leaderboard Aggregate leaderboard ~146
Top of the cross-benchmark aggregate ranking: every model placed on one Elo scale by an IRT model fit over public benchmark leaderboards (call about_the_aggregate for the current coverage counts). One row per model by default, fused across reasoning-effort settings. Supports paging via limit/offset.
| Name | Type | Req | Description |
|---|---|---|---|
| include_variants | boolean | – | Rank each reasoning-effort variant separately (e.g. "Claude Opus 4.6 (High)") instead of one fused row per model. Default false. |
| limit | number | – | Rows to return (1-100, default 25). |
| offset | number | – | Rows to skip from the top (default 0). |
No output schema declared.
No examples provided.
get_model Model profile ~76
One model in depth: aggregate rank, Elo with standard error, provider, what it is, cost per task where known, and its most notable benchmark results (with percentiles).
| Name | Type | Req | Description |
|---|---|---|---|
| model | string | yes | Model name or slug, e.g. "Claude Opus 4.5" or "gpt-5-5". |
No output schema declared.
No examples provided.
get_prediction_duel Prediction duel standings ~58
Guesswork — the public prediction duel: every day frontier LLMs and The Aggregate's own IRT model predict newly scraped benchmark scores before seeing them, and the errors are scored. Returns the current monthly standings, wins and losses included.
Input schema present but exposes no named parameters.
No output schema declared.
No examples provided.
search_benchmarks Search benchmarks ~77
Find benchmarks in the aggregate by (partial) name. Returns model coverage, difficulty on the Elo scale, and the benchmark page URL.
| Name | Type | Req | Description |
|---|---|---|---|
| limit | number | – | Max results (1-25, default 10). |
| query | string | yes | Benchmark name fragment, e.g. "swe-bench" or "arena". |
No output schema declared.
No examples provided.
search_models Search models ~124
Find ranked models by (partial) name or provider. Returns rank, Elo and the model page URL. One row per model by default, fused across reasoning-effort settings.
| Name | Type | Req | Description |
|---|---|---|---|
| include_variants | boolean | – | Return each reasoning-effort variant separately (e.g. "Claude Opus 4.6 (High)") instead of one fused row per model. Default false. |
| limit | number | – | Max results (1-25, default 10). |
| query | string | yes | Model or provider name fragment, e.g. "opus" or "deepseek". |
No output schema declared.
No examples provided.
What is the The Aggregate — LLM benchmark aggregate MCP server?
The Aggregate — LLM benchmark aggregate is an MCP server listed in the public MCP registry as ai.theaggregate/the-aggregate. Fused LLM rankings: one IRT/Elo scale across ~5,000 public benchmark leaderboards, updated daily. This page covers its hosted endpoint (https://theaggregate.ai/mcp).
Is the The Aggregate — LLM benchmark aggregate MCP server safe to use?
The Aggregate — LLM benchmark aggregate scores 79 out of 100 on VerifyMCP. That is a record of what we were able to check automatically, not an endorsement. The category breakdown on this page shows every signal behind the number, including the ones we could not confirm.
What tools does the The Aggregate — LLM benchmark aggregate MCP server expose?
The Aggregate — LLM benchmark aggregate exposes 8 tools: get_leaderboard, search_models, get_model, compare_models, search_benchmarks, and 3 more. Their descriptions and schemas cost roughly 658 tokens of context every time the server is loaded.
Does the The Aggregate — LLM benchmark aggregate MCP server require authentication?
No. We connected to The Aggregate — LLM benchmark aggregate without credentials and it answered, so anything it exposes is reachable by anyone who knows the address.
Is the The Aggregate — LLM benchmark aggregate MCP server still maintained?
The Aggregate — LLM benchmark aggregate is still listed as active in the MCP registry. We last reached this channel on 24 September 2026. Those dates come from our own scans of the registry and the channel itself, not from anything the publisher announced.