Known issues in VerifyMCP scoring
The package rubric explains the general rule: a check we could not complete scores 0 and keeps its weight, because crediting something we did not verify would make the whole number meaningless.
This page is narrower. It lists the specific, current cases where that rule bites for a reason you cannot fix from your side, so you can tell “VerifyMCP cannot inspect this” apart from “VerifyMCP found a problem”.
Bundles that ship a native binary cannot be run
An MCPB bundle whose manifest declares server.type: "binary" is not executed. We run
untrusted code in a locked-down Linux sandbox with no network, a read-only root
filesystem and a forced non-root user. An arbitrary compiled binary gives us no way to
make meaningful guarantees about what actually runs inside that box, so it is out of
scope rather than run unsafely.
The five categories that come from a sandbox run (schema quality, stability, tool coverage, tool safety and capabilities) therefore score 0 and keep their weight for these bundles.
There is a second-order effect worth knowing about. A project that compiles to a native binary usually publishes one bundle per platform, and the registry lists each of those as its own component. So a single project can appear several times over, with every entry reporting the same limitation, even though only one of those artefacts could ever have run on our Linux sandbox in the first place.
We are not going to paper over this by scoring one variant and hiding the rest, and we are not going to award a pass we did not earn. If bundle inspection reaches a point where we can verify a binary bundle honestly, this is the entry that will change.
The sandbox only runs stdio servers
A component that declares any transport other than stdio reports as unsupported on
every ecosystem. The harness works by spawning your server as a child process and
speaking MCP over its standard input and output, so a component that expects to be
reached some other way has nothing for it to talk to.
This is a limit of the current harness architecture rather than a judgement about the transport.
PyPI dependencies must publish a wheel
Your own release does not need a wheel. If it ships only an sdist, we build it, since
that runs your own code, which the sandbox is about to run anyway. Your dependencies,
and the build requirements your sdist declares, are different: we install them with
--only-binary=:all:, so every package in your dependency tree, and every package your
build needs, must publish a wheel. Building one of their sdists would mean executing an
arbitrary setup.py or PEP 517 build backend from a third party whose code we are not
otherwise running.
Two different situations are reported differently, and the difference matters if you are trying to work out what to change:
- Your own release will not build or install (no distribution for that version, its build fails, or a build requirement it declares publishes no wheel). That reports as unsupported, meaning we could not examine it, not that it is broken.
- Your own release installs fine, but something in your dependency tree will not, because a transitive package publishes no wheel (or none for Python 3.13), or an unbounded version pin resolved to a release that breaks. That reports as a failed run that our sandbox could not complete, not as unsupported, and not as your package failing to install.
The first usually means a broken build. The second is usually a pin worth tightening, or a dependency with no wheel.
A failed run does not tell you whose fault it was
When a sandbox run yields nothing, the finding says the run yielded nothing. It does not currently distinguish, in what you can see, between your package failing to start and our own infrastructure having been unable to complete the run at all.
We record that distinction internally on every such finding, so we can tell the two apart when we look at fleet health, and so an outage on our side cannot quietly masquerade as a wave of package defects. It is not yet surfaced to you.
Ecosystem coverage is uneven
Dependency-tree resolution, malware scanning and CVE matching are furthest along on npm, so the highest score other ecosystems can currently reach is lower. This is sized honestly, per ecosystem, in the ceilings table on the package rubric. It is a limit of the analysis we have built so far, and the fix is to extend it rather than to grade the other ecosystems on a curve.