The Aggregate — LLM benchmark aggregate
REMOTE · THEAGGREGATE.AI · SCANNED AUG 3
Fused LLM rankings: one IRT/Elo scale across ~5,000 public benchmark leaderboards, updated daily.
Available components
How this component scores in each security and reliability category. Every signal is checked automatically against the live server, and we only credit what we can confirm. How we score →
Endpoint Security60
- The endpoint's TLS certificate is valid, in date, and uses a strong key. View diagnostics → Pass
- Authorisation not fully verified: no authorisation is required to call this server, and 8 tool(s) never declared a destructiveHint. The MCP spec treats an absent hint as destructive by default, so we cannot call this surface safe. See how to fix → View diagnostics → Unverified
- HTTPS is enforced; there's no plaintext access path. View diagnostics → Pass
- HSTS check failed: the Strict-Transport-Security header is absent. See how to fix → View diagnostics → Fail
- DNSSEC is configured correctly; the domain's records validate against the full chain to the root. View diagnostics → Pass
Transport & Reachability100
- Verified streamable-http transport via a live MCP handshake. View diagnostics → Pass
Schema Quality & AI Usability77
- AI-judged instruction clarity (excellent).Pass
- Tool/resource definitions use about 615 tokens (~76/item across 8 items; 8 tools + 0 resources), lean.Pass
- Usage-examples check failed: none of the tools include examples. See how to fix → Fail
Stability & Change Management27
- Stability observed for 8 of 30 days with no destabilising changes; credit accrues until the full window elapses.Partial
Tool Coverage100
- 100% of tools have a non-trivial description (not blank, and not just the tool's name).Pass
- 100% of tool parameters carry a description.Pass
Capabilities60
- Spec-recency check failed: implements MCP spec 2025-06-18; the latest is 2026-07-28. See how to fix → Fail
Add this component to your MCP client. Where a client-specific snippet is available, pick your client below and copy it straight into your config; otherwise use the connection detail shown.
remote · theaggregate.ai
claude mcp add --transport http ai-theaggregate-the-aggregate https://theaggregate.ai/mcp
[mcp_servers.ai-theaggregate-the-aggregate] url = "https://theaggregate.ai/mcp"
{
"$schema": "https://opencode.ai/config.json",
"mcp": {
"ai-theaggregate-the-aggregate": {
"type": "remote",
"url": "https://theaggregate.ai/mcp",
"enabled": true
}
}
} openclaw mcp add ai-theaggregate-the-aggregate --url https://theaggregate.ai/mcp --transport streamable-http
mcp_servers:
ai-theaggregate-the-aggregate:
url: "https://theaggregate.ai/mcp" {
"mcpServers": {
"ai-theaggregate-the-aggregate": {
"type": "http",
"url": "https://theaggregate.ai/mcp"
}
}
} The mcpServers block is a cross-client convention. Remote transports vary, so check your client's docs.
Every change we have recorded for this component, newest first. Security-relevant changes are always shown. ▲ marks a change for the better, ▼ a change for the worse; unmarked changes are neutral.
- 2 Aug 26 +1
No change was recorded against any check on this day. Stability & Change Management went from 20 to 23. That category is still filling its 30-day observation window: 6 days of observed history at the previous scan, 7 at this one. The score rises as the window fills, whether or not the server changes.
- 31 Jul 26 0
- We updated how we score, so this day's move reflects our rubric, not a change to the server See what changed → functional
- 30 Jul 26 −1
- We updated how we score, so this day's move reflects our rubric, not a change to the server See what changed → functional
- 28 Jul 26 +1
No change was recorded against any check on this day. Stability & Change Management went from 3 to 7. That category is still filling its 30-day observation window: 1 days of observed history at the previous scan, 2 at this one. The score rises as the window fills, whether or not the server changes.
- 27 Jul 26 +1
- We updated how we score, so this day's move reflects our rubric, not a change to the server See what changed → functional
- 26 Jul 26 64
First indexed and scored.
Diagnostic detail from the automated scan of this channel: what the scanner observed at each step, so you can see exactly where a check passed or failed. It is informational only and never changes the trust score.
Captured 3 Aug 2026 · Probed https://theaggregate.ai/mcp
TLS valid
Negotiated TLS 1.3 with TLS_AES_128_GCM_SHA256 .
| Subject | Issuer | Valid from | Valid until | Key | Signature | Serial |
|---|---|---|---|---|---|---|
| CN=theaggregate.ai | CN=WE1,O=Google Trust Services,C=US | 22 Jul 2026 | 20 Oct 2026 | ECDSA 256 | ECDSA-SHA256 | 44c31a1b2ff9a57d0e71b64b4b948142 |
| SANs: theaggregate.ai, *.theaggregate.ai | ||||||
| CN=WE1,O=Google Trust Services,C=US (CA) | CN=GTS Root R4,O=Google Trust Services LLC,C=US | 13 Dec 2023 | 20 Feb 2029 | ECDSA 256 | ECDSA-SHA384 | 7ff31977972c224a76155d13b6d685e3 |
| CN=GTS Root R4,O=Google Trust Services LLC,C=US (CA) | CN=GlobalSign Root CA,OU=Root CA,O=GlobalSign nv-sa,C=BE | 15 Nov 2023 | 28 Jan 2028 | ECDSA 384 | SHA256-RSA | 7fe530bf331343bedd821610493d8a1b |
DNSSEC secure
Validation of theaggregate.ai. — Secure
| Zone | DS | Keys | Algorithms | Outcome |
|---|---|---|---|---|
| . | trust_anchor | 20326, 38696 | 8, 8 | Verified |
| ai. | present | 3799 | 8 | Verified |
| theaggregate.ai. | present | 2371 | 13 | Verified |
| theaggregate.ai. | Verified address RRset verified with the apex keys |
Authentication No authorisation required
The endpoint answered without asking for a token. Anyone who knows the URL can reach it.
| Result | No authorisation required |
|---|---|
| HTTP status | 200 |
Transports 2 probes
| Transport | URL | Outcome | Status | Location |
|---|---|---|---|---|
| streamable-http | https://theaggregate.ai/mcp | Verified | 200 | |
| http (plaintext) | http://theaggregate.ai/mcp | HTTPS enforced | 308 | https://theaggregate.ai/mcp |
The tools this component advertises to a client, with an estimated token cost for each. Expand a tool to see its parameters and schema. The per-tool counts are indicative and are not scored directly; the schema's total context footprint is one signal in Schema Quality & AI Usability.
about_the_aggregate About The Aggregate ~35
What this data is: how the IRT fusion works, current coverage counts, update cadence, and how to cite it.
Input schema present but exposes no named parameters.
No output schema declared.
No examples provided.
compare_models Compare models ~54
Head-to-head between 2-4 models: aggregate ranks, Elo gap with a significance note based on the standard errors, and notable benchmarks they share.
| Name | Type | Req | Description |
|---|---|---|---|
| models | array | yes | Two to four model names or slugs. |
No output schema declared.
No examples provided.
get_benchmark Benchmark detail ~88
One benchmark in depth: what it measures, the original source leaderboard URL, IRT stats (difficulty, noise, model coverage), skill weights, and the current top models on it.
| Name | Type | Req | Description |
|---|---|---|---|
| benchmark | string | yes | Benchmark name or slug, e.g. "Aider polyglot". |
| top | number | — | How many top models to list (1-50, default 10). |
No output schema declared.
No examples provided.
get_leaderboard Aggregate leaderboard ~84
Top of the cross-benchmark aggregate ranking: every model placed on one Elo scale by an IRT model fit over ~5,000 public benchmark leaderboards. Supports paging via limit/offset.
| Name | Type | Req | Description |
|---|---|---|---|
| limit | number | — | Rows to return (1-100, default 25). |
| offset | number | — | Rows to skip from the top (default 0). |
No output schema declared.
No examples provided.
get_model Model profile ~76
One model in depth: aggregate rank, Elo with standard error, provider, what it is, cost per task where known, and its most notable benchmark results (with percentiles).
| Name | Type | Req | Description |
|---|---|---|---|
| model | string | yes | Model name or slug, e.g. "Claude Opus 4.5" or "gpt-5-5". |
No output schema declared.
No examples provided.
get_prediction_duel Prediction duel standings ~58
Guesswork — the public prediction duel: every day frontier LLMs and The Aggregate's own IRT model predict newly scraped benchmark scores before seeing them, and the errors are scored. Returns the current monthly standings, wins and losses included.
Input schema present but exposes no named parameters.
No output schema declared.
No examples provided.
search_benchmarks Search benchmarks ~77
Find benchmarks in the aggregate by (partial) name. Returns model coverage, difficulty on the Elo scale, and the benchmark page URL.
| Name | Type | Req | Description |
|---|---|---|---|
| limit | number | — | Max results (1-25, default 10). |
| query | string | yes | Benchmark name fragment, e.g. "swe-bench" or "arena". |
No output schema declared.
No examples provided.
search_models Search models ~69
Find ranked models by (partial) name or provider. Returns rank, Elo and the model page URL.
| Name | Type | Req | Description |
|---|---|---|---|
| limit | number | — | Max results (1-25, default 10). |
| query | string | yes | Model or provider name fragment, e.g. "opus" or "deepseek". |
No output schema declared.
No examples provided.