io.github.bch1212/modelwatch
NPM · MODELWATCH-MCP
Continuous behavioral drift monitoring for LLM apps — catches silent provider model updates.
Available components
Last scanned (5 hours ago)
Scores higher than 96% of npm servers
Category Testing and QA
How this component scores in each security and reliability category. Every signal is checked automatically from public evidence about the published package, including repeated runs of it in an isolated sandbox, and we only credit what we can confirm. How we score → Why this is hard to score →
Supply Chain Security98
- No malware found by supply-chain analysis.Pass
- No known CVEs affecting this package version or its production dependencies.Pass
- No install/post-install scripts declared.Pass
- 30 of 92 dependencies flagged as unhealthy. View diagnostics → Partial
Provenance & Transparency97
- Source repository is publicly reachable at the declared URL. View diagnostics → Pass
- Cryptographically verified build provenance (signed, bound to bch1212/modelwatch). View diagnostics → Pass
- Clear OSI-approved license (MIT).Pass
- Actively maintained (last published 28 days ago).Pass
- Disclosure check failed: no security disclosure policy was found in the source repository. See how to fix → Fail
Schema Quality & AI Usability74
- AI-judged instruction clarity (good).Pass
- Tool/resource definitions use about 712 tokens (~79/item across 9 items; 9 tools + 0 resources), lean.Pass
- Usage-examples check failed: none of the tools include examples. See how to fix → Fail
Stability & Change Management100
- No destabilizing schema changes in the last 30 days.Pass
Tool Coverage91
- 100% of tools have a non-trivial description (not blank, and not just the tool's name).Pass
- 73% of tool parameters carry a description.Partial
Tool Safety100
- No prompt-injection markers were found in the server instructions, tool names or descriptions we captured.Pass
- We read all 9 captured tool definition(s), and no name or description among them implies an irreversible operation.Pass
- An AI judge read all 9 captured unit(s) of tool text and found none that tries to manipulate the model reading it.Pass
Capabilities100
- Implements a supported MCP spec version (2025-11-25); the latest is 2026-07-28.Pass
Every change we have recorded for this component, newest first. Security-relevant changes are always shown. ▲ marks a change for the better, ▼ a change for the worse; unmarked changes are neutral.
- 27 Oct 26 0
- Stability: 0.97 → pass security
- 26 Oct 26 0
- Stability: pass → 0.97 functional
- 21 Oct 26 0
- Security disclosure: unverified → fail ▼ functional
- 20 Oct 26 0
- Security disclosure: fail → unverified ▼ functional
- 17 Oct 26 +1
- Stability: 0.87 → pass security
- 16 Oct 26 +1
No change was recorded against any check on this day. Stability & Change Management went from 83 to 87. That category is still filling its 30-day observation window: 25 days of observed history at the previous scan, 26 at this one. The score rises as the window fills, whether or not the server changes.
- 15 Oct 26 −2
- Stability: pass → 0.83 functional
- 14 Oct 26 +44
- Injection markers: unverified → pass ▲ security
- Stability: unverified → pass ▲ security
- Tool coverage: unverified → 100 ▲ functional
- MCP protocol: unverified → pass ▲ functional
Diagnostic detail from the automated scan of this channel: what the scanner observed at each step, so you can see exactly where a check passed or failed. It is informational only and never changes the trust score.
Captured 5 Nov 2026 · Analysed npm/modelwatch-mcp@0.1.3
Provenance Verified
A signed build attestation was found and verified, binding this exact artifact to the source repository it claims to come from.
| Result | Verified |
|---|---|
| Ecosystem | npm |
| Reason | Verified |
| Discovered via | Registry attestation endpoint |
| Source repo | bch1212/modelwatch |
| Certificate issuer | https://token.actions.githubusercontent.com |
| Certificate SAN | https://github.com/bch1212/modelwatch/.github/workflows/publish-mcp.yml@refs/tags/mcp-v0.1.3 |
| Rekor log index | 3133481234 |
| Predicate type | SLSA build provenance https://slsa.dev/provenance/v1 |
| Subject digest | sha512:471ac30d076acb4406bbf53b940f8661598fa85136812bb3a562572da1095b65cf676ecb976d8438170638f430073d167da8ae94ce76ec9059f71c456 |
Background: How many MCP packages publish verified provenance →
Dependencies 92 packages
| Packages resolved | 92 |
|---|---|
| Stale | 30 |
| Tree resolution | Complete |
Background: SBOMs and build attestations, explained →
The tools this component advertises to a client, with an estimated token cost for each. Expand a tool to see its parameters and schema. The per-tool counts are indicative and are not scored directly; the schema's total context footprint is one signal in Schema Quality & AI Usability. A tool's description is untrusted text the model reads on every call, which is what makes this list a security surface and not just an inventory: how tool poisoning works →
create_endpoint ~102
Register an LLM endpoint to monitor. The workspace must already have a stored API key for the provider (use the dashboard to add one). Returns the new endpoint's id.
| Name | Type | Req | Description |
|---|---|---|---|
| model | string | yes | Model identifier, e.g. 'gpt-4o-mini' or 'claude-sonnet-4-6'. |
| name | string | yes | Human-readable label, e.g. 'GPT-4o mini prod'. |
| provider | string | yes | – |
No output schema declared.
No examples provided.
create_spec ~152
Create a behavioral spec. The first run after creation sets the baseline output; subsequent scheduled runs are scored against that baseline across 5 axes (semantic, format, refusal, length, contains). Provide a semantic similarity threshold or use the API default.
| Name | Type | Req | Description |
|---|---|---|---|
| endpoint_id | string | yes | Endpoint to monitor (from list_endpoints). |
| frequency | string | – | How often to run the spec (on_trigger for manual runs only). |
| name | string | yes | Spec label, e.g. 'Refusal canary' or 'JSON schema check'. |
| prompt | string | yes | The exact prompt to send to the model. |
| semantic_threshold | number | – | Optional. Semantic similarity threshold (API default 0.85). |
No output schema declared.
No examples provided.
get_drift_events ~85
Fetch recent drift events across the workspace, newest first. Each event has spec_id, run_id, severity, drift_score, summary, and created_at. Use this for weekly review or to drive an automation.
| Name | Type | Req | Description |
|---|---|---|---|
| limit | integer | – | Max events to return (default 20, max 100). |
| spec_id | string | – | Optional. Filter to one spec. |
No output schema declared.
No examples provided.
get_health ~50
Per-spec health across the workspace: spec_id, spec_name, endpoint_name, status (green/yellow/red), last_drift_score and last_run_at. This is not a plan or billing KPI endpoint.
Input schema present but exposes no named parameters.
No output schema declared.
No examples provided.
get_spec_history ~62
Get the run history for a single spec. Returns each run's status, drift score, semantic similarity, format match, and timestamp — useful for trending charts and reasoning about when behavior shifted.
| Name | Type | Req | Description |
|---|---|---|---|
| limit | integer | – | – |
| spec_id | string | yes | – |
No output schema declared.
No examples provided.
list_endpoints ~37
List the LLM endpoints currently monitored in this workspace. Returns id, name, provider, model, is_active, created_at for each.
Input schema present but exposes no named parameters.
No output schema declared.
No examples provided.
list_specs ~71
List behavioral specs in the workspace. A spec is a stored prompt + expectation that ModelWatch replays on a schedule and diffs against a baseline. Returns id, name, input_text, schedule, has_baseline, and the parent endpoint_id.
| Name | Type | Req | Description |
|---|---|---|---|
| endpoint_id | string | – | Optional. Filter by endpoint. |
No output schema declared.
No examples provided.
reset_baseline ~69
Clear a spec's baseline. The next run will record a new baseline instead of being diffed against the old one. Use this after you've intentionally changed your prompt template, model version, or the behavior you expect — otherwise every future run will look like drift.
| Name | Type | Req | Description |
|---|---|---|---|
| spec_id | string | yes | – |
No output schema declared.
No examples provided.
run_spec ~84
Run a spec on demand and return the drift score immediately. Use this to (1) set the baseline manually right after create_spec, or (2) sanity-check a spec without waiting for the next scheduled run. Returns the run status, output, drift score, semantic similarity, format match, refusal flag, and timestamp.
| Name | Type | Req | Description |
|---|---|---|---|
| spec_id | string | yes | From list_specs. |
No output schema declared.
No examples provided.
How do I install the io.github.bch1212/modelwatch MCP server?
io.github.bch1212/modelwatch runs locally as an npm package, launched with npx -y modelwatch-mcp. Ready-made configuration for Claude, Cursor, VS Code, Codex and 5 more is on this page, copied from each client's own documentation.
npm · modelwatch-mcp
claude mcp add bch1212-modelwatch -- npx -y modelwatch-mcp
What is the io.github.bch1212/modelwatch MCP server?
io.github.bch1212/modelwatch is an MCP server listed in the public MCP registry as io.github.bch1212/modelwatch. Continuous behavioral drift monitoring for LLM apps, catches silent provider model updates. This page covers its npm package (modelwatch-mcp).
Is the io.github.bch1212/modelwatch MCP server safe to use?
io.github.bch1212/modelwatch scores 93 out of 100 on VerifyMCP. We found no known CVEs affecting it as of 5 November 2026. It declares no install or post-install scripts. Its build provenance is signed and verified. That is a record of what we were able to check automatically, not an endorsement. The category breakdown on this page shows every signal behind the number, including the ones we could not confirm.
What tools does the io.github.bch1212/modelwatch MCP server expose?
io.github.bch1212/modelwatch exposes 9 tools: list_endpoints, create_endpoint, list_specs, create_spec, run_spec, and 4 more. Their descriptions and schemas cost roughly 712 tokens of context every time the server is loaded.
Is the io.github.bch1212/modelwatch MCP server still maintained?
io.github.bch1212/modelwatch is still listed as active in the MCP registry. We last reached this channel on 5 November 2026. Those dates come from our own scans of the registry and the channel itself, not from anything the publisher announced.
What licence is the io.github.bch1212/modelwatch MCP server under?
io.github.bch1212/modelwatch declares the MIT licence, which is OSI-approved. That covers the source only, and says nothing about the cost of any service it calls.