Scoring changelog: what has changed in how we score

Scores move for two reasons. Usually a server changed. Occasionally we changed, because we added a check, corrected one, or reweighted the categories. This page records the second kind, in date order, so any movement you cannot explain from your own release history can be explained from ours.

We version each rubric internally so that a stored score always names the rules it was computed under. Those identifiers are an implementation detail and are not published; this page is the public record instead.

29 August 2026: a new category for what your tool text tells the model

The shared core gained a fifth category, tool safety, worth 6% of every score on both tracks. It reads the text your server hands a host model, meaning the instructions returned on connect, the name, title and description of every tool, and the description of every parameter, and it asks what that text tells the model to do.

That is a different question from the categories either side of it. Tool coverage asks whether the text is there. Schema quality asks whether it reads well. A server can be documented completely and read beautifully while its descriptions quietly instruct the model to ignore its operator, and until this change nothing in the score said so.

Three signals. Half the category’s weight sits on a scan for injection markers: a fixed list of phrases and characters that address the model reading them rather than the person choosing a tool. A quarter sits on destructive annotations, comparing what a tool’s name and leading description plainly imply against the MCP annotations it declares. The last quarter is the judged reading that was already happening, described below.

Every other shared category paid for it. Schema quality and AI usability fell from 20% to 18%, stability and change management from 15% to 14%, tool coverage from 10% to 8%, and capabilities from 5% to 4%. Nothing was taken from either track’s own half: endpoint security is still 40% and transport 10% on a remote endpoint, supply chain still 30% and provenance still 20% on a package. The shared core still comes to half of every score, and both rubrics still add up to 100.

The weight is small deliberately. Two of the three signals are fixed text scans that prefer precision to recall. They are built to stay quiet unless they are fairly sure, and to be worth listening to when they are not. A category built that way should say something specific and cost a few points; it should not decide a server’s grade on a pattern match. Tool safety never caps a score.

Attempted manipulation is now reported here as well, and it costs a little more. Since 9 August a server that writes text aimed at the scorer rather than at an agent has scored zero for schema quality’s judged signal. That zero was indistinguishable from a server that had simply written no descriptions, which are opposite findings sharing one number. Tool safety does not take that penalty over, it adds a second one: a flagged component still scores zero inside schema quality and now also scores zero for this category’s judged reading. In practice that is roughly a point and a half on top of what a flagged server already lost. The judgement itself is unchanged, the same reading by the same model, so no new server becomes flagged by this release. What changes is that the finding is now stated plainly instead of hiding inside a low schema-quality number.

As before, describing an attack is not performing one. A security or red-team server that documents injection techniques is scanned on the same fixed list as everything else, and a phrase quoted inside your own text does not count against you.

What will move. Most components will shift by a point or two from the re-weighting alone, in either direction depending on where they were already strong. The real movement is on the annotation gap: the MCP spec treats an absent destructiveHint as true, and most servers today declare nothing at all, so a server with tools that delete, publish, transfer or execute will lose part of this category until it declares them. That is a fixable gap, and the fix is an annotation rather than a rewrite. The tool safety guide covers every finding.

The checks landed in our code on 23 August 2026 and reach published scores on 29 August 2026.

9 August 2026: instruction quality is judged tool by tool

Schema quality includes one judgement of how clear and actionable a server’s descriptions are. Until 9 August 2026 that was a single verdict on the whole server, read in one go. From that date each tool is judged on its own, along with the server instructions and the resource list, and the results are averaged.

Every tool now counts for the same amount. Under the old method a server with forty good tools and five undescribed ones was read as one body of text, and what the five cost it depended on how they struck the reader. Now each tool carries an equal share of the result, so a poorly described tool costs a predictable amount rather than whatever an overall impression happened to give it. On a large server that share is still small, because the average is taken over every tool; what has changed is that it is consistent, and it no longer depends on how the rest of the server reads. Servers with a few poorly described tools among many will generally score slightly lower; small, uniformly documented servers will barely move.

Tools are not weighted by how much they have written. A long description is not worth more than a short one, and a terse but complete description is not penalised for being short.

Attempted manipulation now scores zero, and you will see it for the first time. Some servers embed text aimed at the scorer rather than at an agent, such as an instruction to ignore the rubric or to assign a particular score. That has scored zero in our code since 8 August 2026, but existing judgements were not recomputed at the time, so it reaches published scores on 9 August 2026. Describing an attack is not performing one: a security or red-team server that documents injection techniques is judged normally, on the same rubric as everything else. Where a server does try it, the whole component scores zero for this signal, not just the offending tool.

The model that reads the descriptions has changed. This signal is the one place in the score where a language model makes the judgement, so the model is part of the ruler, and replacing it moves the number by itself. We changed it in the same release as the two changes above, deliberately, so that this signal moved once rather than three times in a week. The rubric it is asked to apply is word for word the one described here, and we re-judged every server against it rather than mixing old and new verdicts. Expect small movements in either direction on servers whose descriptions sat near a rubric boundary.

Nothing else moved. No category was reweighted, and no check gained or lost a signal. Only the instruction-quality number is measured differently, so its share of the total is unchanged.

One practical note, since it is the reason this landed when it did: judging a server as a single block meant that when a hosting platform edited one sentence in a boilerplate tool it ships with every server it hosts, we re-judged every one of those servers in full. Judging tool by tool means shared text is judged once.

31 July 2026: one shared core for both tracks

The largest change we have made. Both rubrics were rebuilt around a shared core: the four categories that measure the MCP contract itself (schema quality, stability, tool coverage and capabilities) are now scored by the same checks, at the same weights, whether the component is a package or a remote endpoint. That core is worth half of every score. The other half is what only that kind of component can be judged on.

Every weight changed. On remote endpoints, endpoint security rose from 25% to 40%, schema quality fell from 25% to 20%, stability from 20% to 15%, tool coverage from 15% to 10%, and capabilities from 15% to 5%.

Transport modernity also moved out of capabilities into its own category, transport and reachability, worth 10%. It measures the endpoint rather than the MCP contract, and it was the one signal a package could never earn, because a package’s transport is always stdio.

No remote check gained, lost or changed a signal, so every remote score moved purely by re-weighting.

Packages changed more. They gained the two shared categories they did not previously have, capabilities and stability, while supply chain fell from 36% to 30% and provenance from 24% to 20%. Capabilities cost nothing to add, because it is read from the sandbox capture we already take. Stability did not: it needs two verified captures to compare, and until this change nothing reliably kept the older one. So almost every package now reports insufficient history, scores 0 for that category and keeps its 15% weight. A package that scored 100 before this change lands near 85.

That is deliberate. We price the gap rather than hiding it, and it resolves by itself for each package as its second capture lands.

We also fixed a bug the same day: the package stability check could compare a capture against itself, which meant a package captured once could publish a stability figure it had not earned.

28 July 2026: support for MCP 2026-07-28

Three changes moved published numbers, and all three were the rubric catching up with a new spec revision rather than any server changing.

2025-06-18 left the supported window. We give full protocol-recency marks to the current revision and the one before it. With 2026-07-28 as current, that window became 2026-07-28 and 2025-11-25, so servers on 2025-06-18 dropped from full marks to partial credit.

HTTP+SSE was marked down further, because the revision reclassified it as deprecated under the spec’s lifecycle policy, making it eligible for removal.

A new signal covers client registration. An authorization server offering only the deprecated RFC 7591 dynamic client registration is capped, against full marks for Client ID Metadata Documents.

There is more detail in what MCP 2026-07-28 changes.

26 July 2026: we stopped discounting what we cannot read

Until this change, a category we could not verify dropped out of the calculation entirely, and the remaining categories were re-normalised over what was left. The effect was backwards: a component we could learn nothing about was scored only on the parts we could see, so a package with no readable schema could publish a very high number having proved almost nothing.

Now every category always counts. If we cannot verify one, it scores 0, keeps its weight, and says exactly why it could not be checked.

Packages without a verified sandbox capture fell by roughly 40 points. This was intended, and it is transitional: sandbox extraction runs for every ecosystem, and each component recovers what it can prove as its capture lands.

On remote endpoints only one case moved. A server we could fully read that listed no tools at all used to be scored as though tool coverage did not apply to it, which was a discount for having nothing to show. It is now scored like any other category we cannot credit.

24 July 2026: package schema and tool coverage start counting

We had been capturing what a package’s server actually serves, by installing and running it in an isolated sandbox, and showing the results without scoring them. From this date, schema quality and tool coverage count toward a package’s score at the same weights the remote track uses.

Only packages with a verified capture moved.

23 July 2026: stability ramps over the first 30 days

Stability used to be close to a binary gate on 30 days of observation. A server we had watched for a day was treated much like one we had watched for a month.

It now ramps: a clean history earns credit in proportion to how long we have actually observed it, and churn is scaled by the same window. This means a young server that changes nothing can no longer be outscored by a young server that removed tools, which was possible before.

Young servers moved across the board, by design.

16 July 2026: “tool quality” became “tool coverage”

A naming fix, not a scoring change. The category has always measured something deterministic: how much of the tool surface carries descriptions, parameter documentation and output schemas. It has never been a judgement of whether your tools are any good, and the old name implied more than the checks assess.

No check, weight or threshold changed.

14 July 2026: authorisation scored once, and scoring beyond npm

Authorisation stopped being counted twice. Two separate signals were reading the same underlying fact and penalising it in both places. They became one graded signal, which also gives credit to a server that protects tool calls while leaving discovery open.

We started testing the call path, not just the handshake. Authorisation used to be measured on discovery alone, so a server that answered discovery anonymously but rejected actual tool calls was scored as though it were wide open. We now probe a deliberately non-existent tool inside the live session, which has no side effect and proves whether calls are gated.

Package scoring expanded beyond npm to PyPI, NuGet, OCI and MCPB, with real dependency resolution and ecosystem-native provenance verification.

13 July 2026: the score is the result

The status field used to carry a verdict of its own (verified, partial or inconclusive) alongside the number, which meant two things claimed to summarise one component. Status now records only whether a critical failure fired. The 0–100 score is the result.

Several scoring corrections landed with it:

• The context-footprint curve was wrong, and could score a perfectly lean server close to 0.
• A missing destructive-operation hint is now treated as unknown rather than safe, because the spec defaults it to destructive. The old behaviour rewarded leaving it off.
• Serving MCP over plaintext HTTP became a critical failure.
• Three facts that were being counted in two categories at once were removed from one of them.
• Tool-name style consistency was dropped. It measured house style, not a trust property.

11 July 2026: tool coverage becomes its own category

Tool surface documentation was pulled out into a category of its own, with deterministic signals for description coverage, parameter documentation and output schemas, and the remote weights were rebalanced around it.

28 June 2026: trust scoring goes live

The first public rubric. Signals aggregate into categories, categories into a single 0–100 score, and a small number of critical findings cap the total regardless of everything else. Within days it gained stable reason codes, so every number could be explained down to the individual check, and live verification rather than metadata alone.