Known issues in VerifyMCP scoring
The package rubric explains the general rule: a check we could not complete scores 0 and keeps its weight, because crediting something we did not verify would make the whole number meaningless.
This page is narrower. It lists the specific, current cases where that rule bites for a reason you cannot fix from your side, so you can tell “VerifyMCP cannot inspect this” apart from “VerifyMCP found a problem”.
Bundles that ship a native binary cannot be run
An MCPB bundle whose manifest declares server.type: "binary" is not executed. We run
untrusted code in a locked-down Linux sandbox with no network, a read-only root
filesystem and a forced non-root user. An arbitrary compiled binary gives us no way to
make meaningful guarantees about what actually runs inside that box, so it is out of
scope rather than run unsafely.
The four categories that come from a sandbox run (schema quality, stability, tool coverage and capabilities) therefore score 0 and keep their weight for these bundles.
There is a second-order effect worth knowing about. A project that compiles to a native binary usually publishes one bundle per platform, and the registry lists each of those as its own component. So a single project can appear several times over, with every entry reporting the same limitation, even though only one of those artefacts could ever have run on our Linux sandbox in the first place.
We are not going to paper over this by scoring one variant and hiding the rest, and we are not going to award a pass we did not earn. If bundle inspection reaches a point where we can verify a binary bundle honestly, this is the entry that will change.
The sandbox only runs stdio servers
A component that declares any transport other than stdio reports as unsupported on
every ecosystem. The harness works by spawning your server as a child process and
speaking MCP over its standard input and output, so a component that expects to be
reached some other way has nothing for it to talk to.
This is a limit of the current harness architecture rather than a judgement about the transport.
PyPI packages must publish a wheel
We install PyPI packages with --only-binary=:all:, so a release that ships only an
sdist is reported as unsupported. Installing an sdist means executing an arbitrary
setup.py or PEP 517 build backend at install time, which is exactly the untrusted code
execution the two-phase sandbox exists to contain.
Two different situations are reported differently, and the difference matters if you are trying to work out what to change:
- Your own release publishes no compatible wheel. That reports as unsupported.
- Your own wheel is fine, but something in your dependency tree will not install, because a transitive package publishes no wheel, or an unbounded version pin resolved to a release that breaks. That reports as a failed run, not as unsupported.
Publishing a wheel resolves the first. The second is usually a pin worth tightening.
A failed run does not tell you whose fault it was
When a sandbox run yields nothing, the finding says the run yielded nothing. It does not currently distinguish, in what you can see, between your package failing to start and our own infrastructure having been unable to complete the run at all.
We record that distinction internally on every such finding, so we can tell the two apart when we look at fleet health, and so an outage on our side cannot quietly masquerade as a wave of package defects. It is not yet surfaced to you.
Ecosystem coverage is uneven
Dependency-tree resolution, malware scanning and CVE matching are furthest along on npm, so the highest score other ecosystems can currently reach is lower. This is sized honestly, per ecosystem, in the ceilings table on the package rubric. It is a limit of the analysis we have built so far, and the fix is to extend it rather than to grade the other ecosystems on a curve.