Scoring changelog: what has changed in how we score

Scores move for two reasons. Usually a server changed. Occasionally we changed, because we added a check, corrected one, or reweighted the categories. This page records the second kind, in date order, so any movement you cannot explain from your own release history can be explained from ours.

We version each rubric internally so that a stored score always names the rules it was computed under. Those identifiers are an implementation detail and are not published; this page is the public record instead.

31 July 2026: one shared core for both tracks

The largest change we have made. Both rubrics were rebuilt around a shared core: the four categories that measure the MCP contract itself (schema quality, stability, tool coverage and capabilities) are now scored by the same checks, at the same weights, whether the component is a package or a remote endpoint. That core is worth half of every score. The other half is what only that kind of component can be judged on.

Every weight changed. On remote endpoints, endpoint security rose from 25% to 40%, schema quality fell from 25% to 20%, stability from 20% to 15%, tool coverage from 15% to 10%, and capabilities from 15% to 5%.

Transport modernity also moved out of capabilities into its own category, transport and reachability, worth 10%. It measures the endpoint rather than the MCP contract, and it was the one signal a package could never earn, because a package’s transport is always stdio.

No remote check gained, lost or changed a signal, so every remote score moved purely by re-weighting.

Packages changed more. They gained the two shared categories they did not previously have, capabilities and stability, while supply chain fell from 36% to 30% and provenance from 24% to 20%. Capabilities cost nothing to add, because it is read from the sandbox capture we already take. Stability did not: it needs two verified captures to compare, and until this change nothing reliably kept the older one. So almost every package now reports insufficient history, scores 0 for that category and keeps its 15% weight. A package that scored 100 before this change lands near 85.

That is deliberate. We price the gap rather than hiding it, and it resolves by itself for each package as its second capture lands.

We also fixed a bug the same day: the package stability check could compare a capture against itself, which meant a package captured once could publish a stability figure it had not earned.

28 July 2026: support for MCP 2026-07-28

Three changes moved published numbers, and all three were the rubric catching up with a new spec revision rather than any server changing.

2025-06-18 left the supported window. We give full protocol-recency marks to the current revision and the one before it. With 2026-07-28 as current, that window became 2026-07-28 and 2025-11-25, so servers on 2025-06-18 dropped from full marks to partial credit.

HTTP+SSE was marked down further, because the revision reclassified it as deprecated under the spec’s lifecycle policy, making it eligible for removal.

A new signal covers client registration. An authorization server offering only the deprecated RFC 7591 dynamic client registration is capped, against full marks for Client ID Metadata Documents.

There is more detail in what MCP 2026-07-28 changes.

26 July 2026: we stopped discounting what we cannot read

Until this change, a category we could not verify dropped out of the calculation entirely, and the remaining categories were re-normalised over what was left. The effect was backwards: a component we could learn nothing about was scored only on the parts we could see, so a package with no readable schema could publish a very high number having proved almost nothing.

Now every category always counts. If we cannot verify one, it scores 0, keeps its weight, and says exactly why it could not be checked.

Packages without a verified sandbox capture fell by roughly 40 points. This was intended, and it is transitional: sandbox extraction runs for every ecosystem, and each component recovers what it can prove as its capture lands.

On remote endpoints only one case moved. A server we could fully read that listed no tools at all used to be scored as though tool coverage did not apply to it, which was a discount for having nothing to show. It is now scored like any other category we cannot credit.

24 July 2026: package schema and tool coverage start counting

We had been capturing what a package’s server actually serves, by installing and running it in an isolated sandbox, and showing the results without scoring them. From this date, schema quality and tool coverage count toward a package’s score at the same weights the remote track uses.

Only packages with a verified capture moved.

23 July 2026: stability ramps over the first 30 days

Stability used to be close to a binary gate on 30 days of observation. A server we had watched for a day was treated much like one we had watched for a month.

It now ramps: a clean history earns credit in proportion to how long we have actually observed it, and churn is scaled by the same window. This means a young server that changes nothing can no longer be outscored by a young server that removed tools, which was possible before.

Young servers moved across the board, by design.

16 July 2026: “tool quality” became “tool coverage”

A naming fix, not a scoring change. The category has always measured something deterministic: how much of the tool surface carries descriptions, parameter documentation and output schemas. It has never been a judgement of whether your tools are any good, and the old name implied more than the checks assess.

No check, weight or threshold changed.

14 July 2026: authorisation scored once, and scoring beyond npm

Authorisation stopped being counted twice. Two separate signals were reading the same underlying fact and penalising it in both places. They became one graded signal, which also gives credit to a server that protects tool calls while leaving discovery open.

We started testing the call path, not just the handshake. Authorisation used to be measured on discovery alone, so a server that answered discovery anonymously but rejected actual tool calls was scored as though it were wide open. We now probe a deliberately non-existent tool inside the live session, which has no side effect and proves whether calls are gated.

Package scoring expanded beyond npm to PyPI, NuGet, OCI and MCPB, with real dependency resolution and ecosystem-native provenance verification.

13 July 2026: the score is the result

The status field used to carry a verdict of its own (verified, partial or inconclusive) alongside the number, which meant two things claimed to summarise one component. Status now records only whether a critical failure fired. The 0–100 score is the result.

Several scoring corrections landed with it:

• The context-footprint curve was wrong, and could score a perfectly lean server close to 0.
• A missing destructive-operation hint is now treated as unknown rather than safe, because the spec defaults it to destructive. The old behaviour rewarded leaving it off.
• Serving MCP over plaintext HTTP became a critical failure.
• Three facts that were being counted in two categories at once were removed from one of them.
• Tool-name style consistency was dropped. It measured house style, not a trust property.

11 July 2026: tool coverage becomes its own category

Tool surface documentation was pulled out into a category of its own, with deterministic signals for description coverage, parameter documentation and output schemas, and the remote weights were rebalanced around it.

28 June 2026: trust scoring goes live

The first public rubric. Signals aggregate into categories, categories into a single 0–100 score, and a small number of critical findings cap the total regardless of everything else. Within days it gained stable reason codes, so every number could be explained down to the individual check, and live verification rather than metadata alone.