Package rubric: npm, PyPI, NuGet, OCI, MCPB

A packaged server is code you download and run, so half the score is verifiable supply-chain and provenance evidence: 30% supply chain, 20% provenance and transparency.

The other half is the shared core, the five categories that measure the MCP contract itself and are scored identically on packages and remote endpoints: 18% schema quality and AI usability, 14% stability and change management, 8% tool coverage, 6% tool safety, 4% capabilities. A package does not publish its schema anywhere we can read, so we install and run it in an isolated sandbox and score the schema it serves there.

All seven categories are always scored and always shown. When we do not yet have a capture, the shared five score 0 and say why, exactly like any other check we cannot confirm. Sandbox capture is enabled for npm, PyPI, NuGet, OCI and MCPB, and we are working through the indexed fleet, so a package indexed recently may sit at that 0 for a short while before its capture lands. A clean, well-provenanced package whose captured schema is well documented can reach 100.

Ceilings vary by ecosystem, today

We resolve the full dependency tree for npm, PyPI, NuGet and MCPB, and for OCI only when the publisher attaches an SBOM (discovered through the OCI Referrers API, falling back to the legacy sha256-<digest>.sbom tag). Most images attach none, so their dependency-derived signals stay inconclusive rather than being scored as clean. Malware coverage reaches npm, PyPI and NuGet but not OCI or MCPB, because our vendor has no package-URL coverage there. So the highest score an ecosystem can currently reach still varies sharply. This is an honest limitation of the analysis we have built so far, not a judgement about those ecosystems; the fix is to extend the same analysis to the ecosystems it misses.

The limitation is entirely in the package-only half of the rubric (supply chain and provenance). The shared core is scored from a sandbox run, which works the same way for every ecosystem, so it is not where the ceilings come from.

These are estimates read off the signal weights and each ecosystem’s known permanent gaps, not measurements from a scoring sweep. That sweep, scoring a fixed set of real components per ecosystem, has not been run, which is why this table gives bands rather than the two-decimal figures the arithmetic would happily produce. Treat the shape as reliable and the precision as absent.

An ecosystem we can analyse less thoroughly still reaches the middle of the range on the strength of a well-documented schema, because the shared core is half the score and is measured from a sandbox run that works the same way everywhere. That is the point of scoring the two halves separately.

A component we have not yet captured in the sandbox keeps the package-only half, so roughly 50% of the figures above: all five shared categories score 0 and keep their weight. That is a transitional state, not a property of the ecosystem.

Supply chain (30%)

Whether the code is safe to run today.

Culpability, not exposure

The CVE check resolves your production dependency tree (dev and peer dependencies excluded) and queries public advisory databases in batch across the whole tree, not just the root. Findings are attributed as direct (a dependency you declare) versus transitive (something deeper), and each names the vulnerable package, its version, and the path from your package to it; direct findings weigh more than transitive ones.

A vulnerability counts against your score only when a fixed version exists: when your declared range still resolves to a vulnerable version and an upgrade is on the shelf. An advisory with no published fix is shown as information, not scored as a failure: nobody can install their way out of it, so charging for it would measure luck, not diligence. If the tree only partly resolves, the CVE and dependency-health signals are scored on what resolved and marked partial; a half-walked tree never rounds up to a clean bill of health.

Sources: our malware-scanning provider for the malware verdict, public advisory databases for CVEs across the resolved tree, and our own resolution of the full dependency tree for dependency health (and for the tree the CVE check walks). A source with no data for your package leaves its signal inconclusive: we can’t confirm it, so it scores 0 and counts against your score (see below), not merely “not failed”. Likewise, when no malware-scanning provider is configured on our side the malware signal is marked inconclusive rather than silently scoring on.

Install scripts are classified by their command string, not merely their presence: no hook or a recognised native-build command (node-gyp, tsc, prebuild-install, and similar, with plain arguments) earns full marks; an unambiguous dropper (a curl | sh, node -e, base64 decode, chmod +x, an inline URL) scores 0; anything else opaque scores 0.5 (a hook runs and we haven’t established what it does; we don’t open the tarball). Dependency health scores the resolved production set (deprecated, stale, or no source repo) and names the specific unhealthy dependencies so you can act on them.

Provenance & transparency (20%)

Whether you can prove where the code came from and who stands behind it.

A reachable source repository underpins several of these (we read it for the security policy, and it anchors the provenance attestation) so linking a verifiable repository is often the highest-leverage single fix.

We deliberately don’t score “verified publisher.” npm performs no identity verification on accounts or organisations (both are free to create) so a namespace can’t prove who stands behind a package, and crediting it would reward a signal anyone could fake. The accountable-identity question is answered instead by build provenance: a cryptographically signed attestation, following the SLSA provenance model, that binds the artifact to the source repository and CI workflow that produced it. Both GitHub Actions and GitLab CI attestations are accepted. An attestation that is cryptographically sound but names a different repository than the registry metadata declares is reported as a stale-metadata mismatch, not an accusation of forgery: it usually means a moved repo or a stale repository field, so it earns no credit but accuses nobody.

The shared core (50%)

Whether an agent that installs your package can actually understand and rely on what it exposes. We run the published package in an isolated sandbox with no credentials and strict resource limits, speak MCP to it over stdio, and capture the schema it serves: its tools, parameters, prompts, resources and server instructions. That capture is then scored by the same signals as the remote rubric, with the same reason codes:

• Schema quality & AI usability (18%): prompt and resource description completeness, an AI-judged instruction-clarity signal (fixed rubric, cached so an unchanged schema is never re-judged), the context-window token footprint, and usage examples.
• Stability & change management (14%): whether your tool surface stays stable across the versions we have captured. See below.
• Tool coverage (8%): tool description coverage, parameter documentation, and declared structured output schemas.
• Tool safety (6%): what that same tool text tells a host model to do. A scan for injection markers, whether tools that perform irreversible operations declare their MCP annotations, and an AI-judged reading for text written to manipulate the model. See tool safety.
• Capabilities (4%): how current the MCP spec revision you negotiate is, plus a small bonus if you ship UI template metadata.

How stability works on a package

A published version is immutable, so stability here is not about your package changing under anyone: it cannot. It is about what changes between versions, which is what someone meets when they upgrade. We compare the earliest capture we hold from the last 30 days against the most recent one, and a removed tool or an incompatibly changed parameter counts against you. Added tools do not.

Two consequences worth knowing:

• If both captures record the same version and they disagree, we report that we could not tell what happened, rather than counting it against you. Either our sandbox is unreliable for your package (the usual cause is a tool list that depends on a secret or a native dependency it resolved differently between runs), or the coordinate you publish under is one that can be re-pointed, like an OCI tag. We cannot tell those apart from the outside, so we say so instead of guessing either way. Two captures of one version that agree are simply stable.
• Until we hold two captures we say so and score 0, rather than calling an unmeasured surface stable. We began retaining the history this needs in July 2026, so most packages will show that for a while.

Unlike the remote rubric, nothing here penalises a dropped authentication requirement or a changed transport. Neither is something a package has.

When we cannot read your schema

All four are always scored and always shown. When we do not have a verified capture of your current version, whether because we have never run your package, our capture is of a version you have since superseded, the run yielded nothing, our storage trim cut the capture short, or your package declared that it provides tools and our run could not list them, each category records a single unverified finding naming which of those it was, and scores 0 while keeping its full weight. That is the same rule as everywhere else here: we do not credit what we cannot confirm, and we do not hide a category we could not read. A failed sandbox run is still our limitation rather than your defect, and the finding says so, but leaving the category out would overstate what we have actually verified about your package.

Capabilities is the one exception to the truncation rule: if a capture was too large to store in full we still read the protocol version out of it, because trimming only removes descriptive text.

When a capture succeeds and the package genuinely exposes no tools, prompts or resources, the findings say exactly that. Those are confirmed readings rather than gaps, so they read as findings, not as “unverified”.

The captured tool list, with per-tool token estimates, is shown on your server page’s MCP tools card.

What unverifiable signals do here

Some signals we simply couldn’t verify from public data: a source with no data for the package, an unpublished version, or a repository that isn’t publicly accessible. Because we only credit what we can confirm, they score 0 and count against your score: an unverifiable signal is never given the benefit of the doubt. Making the underlying evidence public (a reachable repo, a published version) is what lets the signal be verified and count in your favour.

Two of these gaps are ours rather than yours: an ecosystem we cannot yet analyse fully, and a sandbox run that fails on our side. They score 0 for the same reason (we only credit what we confirm) but there is nothing for you to fix, and the finding on your page says so.

Full machine spec: docs/scoring-package.md in the repository.