Package rubric: npm, PyPI, NuGet, OCI, MCPB
A packaged server is code you download and run, so half the score is verifiable supply-chain and provenance evidence: 30% supply chain, 20% provenance and transparency.
The other half is the shared core, the four categories that measure the MCP contract itself and are scored identically on packages and remote endpoints: 20% schema quality and AI usability, 15% stability and change management, 10% tool coverage, 5% capabilities. A package does not publish its schema anywhere we can read, so we install and run it in an isolated sandbox and score the schema it serves there.
All six categories are always scored and always shown. When we do not yet have a capture, the shared four score 0 and say why, exactly like any other check we cannot confirm. Sandbox capture is enabled for npm, PyPI, NuGet, OCI and MCPB, and we are working through the indexed fleet, so a package indexed recently may sit at that 0 for a short while before its capture lands. A clean, well-provenanced package whose captured schema is well documented can reach 100.
Ceilings vary by ecosystem, today
We resolve the full dependency tree for npm today, and malware and CVE coverage are npm-first, so the highest score an ecosystem can currently reach varies sharply. This is an honest limitation of the analysis we have built so far, not a judgement about those ecosystems; the fix is to extend the same analysis to the other ecosystems.
The limitation is entirely in the package-only half of the rubric (supply chain and provenance). The shared core is scored from a sandbox run, which works the same way for every ecosystem, so it is not where the ceilings come from.
| Ecosystem | Malware scanning configured | Package half | Ceiling | Ceiling while we await a capture |
|---|---|---|---|---|
| npm | yes | 100 | 100 | 50 |
| npm | no | ~70 | ~85 | ~35 |
| PyPI, NuGet | yes | ~42 | ~71 | ~21 |
| OCI, MCPB | yes | ~12 | ~56 | ~6 |
PyPI and NuGet cap around 42 on the package half because their CVE, install-script and dependency-health signals go inconclusive (we cannot yet resolve their trees), and each inconclusive signal scores 0 while keeping its weight. Without malware scanning configured on our side, npm caps around 70 because the heaviest signal, malware, goes inconclusive. The container (OCI) and bundle (MCPB) types are scored only on the handful of signals that apply (repo link, licence, maintenance), so they sit lowest until we build container and bundle inspection.
The Ceiling column is that package half at 50% of the score, plus a full shared core at the other 50%. So an ecosystem we can barely analyse statically can still reach the middle of the range on the strength of a well-documented schema, which is the point of scoring the two halves separately.
The right-hand column is the ceiling for a package we have not yet captured in the sandbox: all four shared categories score 0 and keep their weight, which is 50% of the total. It is a transitional number, not a property of the ecosystem.
Supply chain (30%)
Whether the code is safe to run today.
| Signal | Passing | Needs a fix |
|---|---|---|
| Malware | No malware found by supply-chain analysis | A flagged malware risk, or a confirmed critical verdict → malware |
| Known CVEs | No known CVEs affecting the package or its production dependencies | A known CVE in a direct or transitive dependency, or an unpatched critical one → vulnerabilities |
| Install scripts | No install scripts, or only a recognised native-build step | An unrecognised install-time script, or one that fetches or executes arbitrary code → install scripts |
| Dependency health | The resolved dependencies assess healthy, or there are none | named unhealthy dependencies → dependency health |
Culpability, not exposure
The CVE check resolves your production dependency tree (dev and peer dependencies excluded) and queries public advisory databases in batch across the whole tree, not just the root. Findings are attributed as direct (a dependency you declare) versus transitive (something deeper), and each names the vulnerable package, its version, and the path from your package to it; direct findings weigh more than transitive ones.
A vulnerability counts against your score only when a fixed version exists: when your declared range still resolves to a vulnerable version and an upgrade is on the shelf. An advisory with no published fix is shown as information, not scored as a failure: nobody can install their way out of it, so charging for it would measure luck, not diligence. If the tree only partly resolves, the CVE and dependency-health signals are scored on what resolved and marked partial; a half-walked tree never rounds up to a clean bill of health.
Sources: our malware-scanning provider for the malware verdict, public advisory databases for CVEs across the resolved tree, and our own resolution of the full dependency tree for dependency health (and for the tree the CVE check walks). A source with no data for your package leaves its signal inconclusive: we can’t confirm it, so it scores 0 and counts against your score (see below), not merely “not failed”. Likewise, when no malware-scanning provider is configured on our side the malware signal is marked inconclusive rather than silently scoring on.
Install scripts are classified by their command string, not merely their
presence: no hook or a recognised native-build command (node-gyp, tsc,
prebuild-install, and similar, with plain arguments) earns full marks; an
unambiguous dropper (a curl | sh, node -e, base64 decode, chmod +x, an inline
URL) scores 0; anything else opaque scores 0.5 (a hook runs and we haven’t
established what it does; we don’t open the tarball). Dependency health scores the
resolved production set (deprecated, stale, or no source repo) and names the
specific unhealthy dependencies so you can act on them.
Provenance & transparency (20%)
Whether you can prove where the code came from and who stands behind it.
| Signal | Passing | Needs a fix |
|---|---|---|
| Source repository | Publicly reachable at the declared URL | No repository declared, a redirecting or unreachable URL, a malformed address, or an unrecognised source-code host → repository |
| Build provenance | A cryptographically verified, signed attestation bound to the repository | No attestation published, one that failed verification, or one naming a different repository to the one declared → build provenance |
| Licence | A clear OSI-approved licence | No licence declared, or one that isn’t OSI-approved → license |
| Maintenance | Actively maintained (published recently) | Infrequently maintained → maintenance |
| Security policy | A security disclosure policy (SECURITY.md) is published | No policy found in the source repository → security policy |
A reachable source repository underpins several of these (we read it for the security policy, and it anchors the provenance attestation) so linking a verifiable repository is often the highest-leverage single fix.
We deliberately don’t score “verified publisher.” npm performs no identity
verification on accounts or organisations (both are free to create) so a
namespace can’t prove who stands behind a package, and crediting it would reward a
signal anyone could fake. The accountable-identity question is answered instead by
build provenance: a cryptographically signed attestation, following the
SLSA provenance model, that binds the artifact to the source
repository and CI workflow that produced it. Both GitHub Actions and GitLab CI
attestations are accepted. An attestation that is cryptographically sound but names
a different repository than the registry metadata declares is reported as a
stale-metadata mismatch, not an accusation of forgery: it usually means a moved
repo or a stale repository field, so it earns no credit but accuses nobody.
The shared core (50%)
Whether an agent that installs your package can actually understand and rely on what it exposes. We run the published package in an isolated sandbox with no credentials and strict resource limits, speak MCP to it over stdio, and capture the schema it serves: its tools, parameters, prompts, resources and server instructions. That capture is then scored by the same signals as the remote rubric, with the same reason codes:
- Schema quality & AI usability (20%): prompt and resource description completeness, an AI-judged instruction-clarity signal (fixed rubric, cached so an unchanged schema is never re-judged), the context-window token footprint, and usage examples.
- Stability & change management (15%): whether your tool surface stays stable across the versions we have captured. See below.
- Tool coverage (10%): tool description coverage, parameter documentation, and declared structured output schemas.
- Capabilities (5%): how current the MCP spec revision you negotiate is, plus a small bonus if you ship UI template metadata.
How stability works on a package
A published version is immutable, so stability here is not about your package changing under anyone: it cannot. It is about what changes between versions, which is what someone meets when they upgrade. We compare the earliest capture we hold from the last 30 days against the most recent one, and a removed tool or an incompatibly changed parameter counts against you. Added tools do not.
Two consequences worth knowing:
- If both captures record the same version and they disagree, we report that we could not tell what happened, rather than counting it against you. Either our sandbox is unreliable for your package (the usual cause is a tool list that depends on a secret or a native dependency it resolved differently between runs), or the coordinate you publish under is one that can be re-pointed, like an OCI tag. We cannot tell those apart from the outside, so we say so instead of guessing either way. Two captures of one version that agree are simply stable.
- Until we hold two captures we say so and score 0, rather than calling an unmeasured surface stable. We began retaining the history this needs in July 2026, so most packages will show that for a while.
Unlike the remote rubric, nothing here penalises a dropped authentication requirement or a changed transport. Neither is something a package has.
When we cannot read your schema
All four are always scored and always shown. When we do not have a verified capture of your current version, whether because we have never run your package, our capture is of a version you have since superseded, the run yielded nothing, our storage trim cut the capture short, or your package declared that it provides tools and our run could not list them, each category records a single unverified finding naming which of those it was, and scores 0 while keeping its full weight. That is the same rule as everywhere else here: we do not credit what we cannot confirm, and we do not hide a category we could not read. A failed sandbox run is still our limitation rather than your defect, and the finding says so, but leaving the category out would overstate what we have actually verified about your package.
Capabilities is the one exception to the truncation rule: if a capture was too large to store in full we still read the protocol version out of it, because trimming only removes descriptive text.
When a capture succeeds and the package genuinely exposes no tools, prompts or resources, the findings say exactly that. Those are confirmed readings rather than gaps, so they read as findings, not as “unverified”.
The captured tool list, with per-tool token estimates, is shown on your server page’s MCP tools card.
What unverifiable signals do here
Some signals we simply couldn’t verify from public data: a source with no data for the package, an unpublished version, or a repository that isn’t publicly accessible. Because we only credit what we can confirm, they score 0 and count against your score: an unverifiable signal is never given the benefit of the doubt. Making the underlying evidence public (a reachable repo, a published version) is what lets the signal be verified and count in your favour.
Two of these gaps are ours rather than yours: an ecosystem we cannot yet analyse fully, and a sandbox run that fails on our side. They score 0 for the same reason (we only credit what we confirm) but there is nothing for you to fix, and the finding on your page says so.
Full machine spec: docs/scoring-package.md in the repository.