Skip to content
verify mcp Beta VerifyMCP is currently in beta. If you notice any issues, email [email protected] and we’ll put it right.

io.github.bch1212/modelwatch

NPM · MODELWATCH-MCP · SCANNED AUG 3

Continuous behavioral drift monitoring for LLM apps — catches silent provider model updates.

Available components

npm · modelwatch-mcp 79
+34 this week 79 Trust /100
Trust breakdown (6 categories)

How this component scores in each security and reliability category. Every signal is checked automatically from public evidence about the published package, including repeated runs of it in an isolated sandbox, and we only credit what we can confirm. How we score →

Supply Chain Security86
  • No malware found by supply-chain analysis.Pass
  • Only part of the dependency tree could be resolved (94 of 98), so this covers what we could see, not the whole tree.Partial
  • No install/post-install scripts declared.Pass
  • Only part of the dependency tree could be resolved (94 of 98), so this covers what we could see, not the whole tree. View diagnostics → Partial
Provenance & Transparency97
  • Source repository is publicly reachable at the declared URL. View diagnostics → Pass
  • Cryptographically verified build provenance (signed, bound to bch1212/modelwatch). View diagnostics → Pass
  • Clear OSI-approved license (MIT).Pass
  • Actively maintained (last published 85 days ago).Pass
  • Disclosure check failed: no security disclosure policy was found in the source repository. See how to fix → Fail
Schema Quality & AI Usability77
  • AI-judged instruction clarity (excellent).Pass
  • Tool/resource definitions use about 745 tokens (~82/item across 9 items; 9 tools + 0 resources), lean.Pass
  • Usage-examples check failed: none of the tools include examples. See how to fix → Fail
Stability & Change Management27
  • Stability observed for 8 of 30 days with no destabilising changes; credit accrues until the full window elapses.Partial
Tool Coverage92
  • 100% of tools have a non-trivial description (not blank, and not just the tool's name).Pass
  • 75% of tool parameters carry a description.Partial
Capabilities100
  • Implements a supported MCP spec version (2025-11-25); the latest is 2026-07-28.Pass
Install

Add this component to your MCP client. Where a client-specific snippet is available, pick your client below and copy it straight into your config; otherwise use the connection detail shown.

npm · modelwatch-mcp

# add to Claude Code
claude mcp add bch1212-modelwatch -- npx -y modelwatch-mcp
# add to Codex CLI
codex mcp add bch1212-modelwatch -- npx -y modelwatch-mcp
// opencode.json
{
  "$schema": "https://opencode.ai/config.json",
  "mcp": {
    "bch1212-modelwatch": {
      "type": "local",
      "command": [
        "npx",
        "-y",
        "modelwatch-mcp"
      ],
      "enabled": true
    }
  }
}
# add to OpenClaw
openclaw mcp add bch1212-modelwatch --command npx --arg -y --arg modelwatch-mcp
# ~/.hermes/config.yaml
mcp_servers:
  bch1212-modelwatch:
    command: "npx"
    args: ["-y", "modelwatch-mcp"]
// mcp.json
{
  "mcpServers": {
    "bch1212-modelwatch": {
      "command": "npx",
      "args": [
        "-y",
        "modelwatch-mcp"
      ]
    }
  }
}
Changelog

Every change we have recorded for this component, newest first. Security-relevant changes are always shown. ▲ marks a change for the better, ▼ a change for the worse; unmarked changes are neutral.

  • 3 Aug 26 +1

    No change was recorded against any check on this day. Stability & Change Management went from 23 to 27. That category is still filling its 30-day observation window: 7 days of observed history at the previous scan, 8 at this one. The score rises as the window fills, whether or not the server changes.

  • 2 Aug 26 +43
    • Known CVEs: unverified → partial ▲ security
    • Install scripts: unverified → pass ▲ security
    • Provenance: unverified → pass ▲ security
    • Malware scan: unverified → pass ▲ security
    • The attested source repository moved: bch1212/modelwatch security
    • Maintenance: unverified → pass ▲ functional
    • Dependency health: unverified → partial ▲ functional
    • License: unverified → pass ▲ functional
    • Stability: unverified → 0.23 ▲ functional
    • Licence: MIT functional
  • 1 Aug 26 +30
    • Stability: Stability not yet verified: not enough scan history yet (needs a 30-day window). security
    • Security disclosure: unverified → fail ▼ functional
    • MCP protocol: unverified → pass ▲ functional
    • Tool coverage: unverified → 100 ▲ functional
  • 31 Jul 26 −56
    • We updated how we score, so this day's move reflects our rubric, not a change to the server See what changed → functional
  • 30 Jul 26 −30
    • Malware scan: pass → unverified ▼ security
    • Known CVEs: partial → unverified ▼ security
    • Dependency health: partial → unverified ▼ functional
  • 29 Jul 26 +65
    • Known CVEs: unverified → partial ▲ security
    • Install scripts: unverified → pass ▲ security
    • Provenance: unverified → pass ▲ security
    • The attested source repository moved: bch1212/modelwatch security
    • License: unverified → pass ▲ functional
    • Tool coverage: unverified → 100 ▲ functional
    • Schema quality: unverified → excellent ▲ functional
    • Maintenance: unverified → pass ▲ functional
    • Licence: MIT functional
  • 28 Jul 26 −19
    • Tool coverage: 100 → unverified ▼ functional
    • Dependency health: unverified → partial ▲ functional
    • First check of Schema quality: unverified functional
  • 27 Jul 26 45

    First indexed and scored.

Diagnostics

Diagnostic detail from the automated scan of this channel: what the scanner observed at each step, so you can see exactly where a check passed or failed. It is informational only and never changes the trust score.

Captured 3 Aug 2026 · Analysed npm/[email protected]

Provenance verified

Ecosystem: npm · Outcome: verified

Reason: verified

Source repo:
bch1212/modelwatch
Certificate issuer:
https://token.actions.githubusercontent.com
Certificate SAN:
https://github.com/bch1212/modelwatch/.github/workflows/publish-mcp.yml@refs/tags/mcp-v0.1.1
Rekor log index:
1486480195
Predicate type:
https://slsa.dev/provenance/v1
Subject digest:
sha512:3a8e1a7d88b963d5bdb928086dd5f09846d1877be0bb4ea96ed425e4deeb4249274cf2aa2ba91523b9cbf6cbfdf291dca88530b97e1fb89ed430a9fd9
Discovery method:
attestation_endpoint
Dependencies 94 packages

94 packages in the resolved dependency tree · 94 deprecated · 29 stale.

The dependency tree was only partially resolved, so these counts may be incomplete.

MCP tools — 9 exposed · ~745 tokens

The tools this component advertises to a client, with an estimated token cost for each. Expand a tool to see its parameters and schema. The per-tool counts are indicative and are not scored directly; the schema's total context footprint is one signal in Schema Quality & AI Usability.

Tool Tokens
create_endpoint ~130

Register an LLM endpoint to monitor. The workspace must already have a stored API key for the provider (use the dashboard to add one). Returns the new endpoint's id.

NameTypeReqDescription
base_urlstring—Optional. Override base URL for OpenAI-compatible endpoints (vLLM, LiteLLM, Together).
modelstringyesModel identifier, e.g. 'gpt-4o-mini' or 'claude-sonnet-4-6'.
namestringyesHuman-readable label, e.g. 'GPT-4o mini prod'.
providerstringyes—

No output schema declared.

No examples provided.

create_spec ~167

Create a behavioral spec. The first run after creation sets the baseline output; subsequent scheduled runs are scored against that baseline across 5 axes (semantic, format, refusal, length, contains). An alert is sent when the drift score crosses the threshold.

NameTypeReqDescription
endpoint_idstringyesEndpoint to monitor (from list_endpoints).
frequencystring—How often to run the spec.
namestringyesSpec label, e.g. 'Refusal canary' or 'JSON schema check'.
promptstringyesThe exact prompt to send to the model.
thresholdstring—Severity at which to fire an alert. Buckets: low ≥0.05, medium ≥0.15, high ≥0.35, critical ≥0.6.

No output schema declared.

No examples provided.

get_drift_events ~92

Fetch recent drift events across the workspace, newest first. Each event has spec_id, spec_name, severity, drift_score, axes breakdown, baseline_output, current_output, and detected_at. Use this for weekly review or to drive an automation.

NameTypeReqDescription
limitinteger—Max events to return (default 20, max 100).
spec_idstring—Optional. Filter to one spec.

No output schema declared.

No examples provided.

get_health ~35

Workspace KPIs: plan, spec count, runs this month, plan limits, active drift events. Useful as a daily status check.

Input schema present but exposes no named parameters.

No output schema declared.

No examples provided.

get_spec_history ~59

Get the run history for a single spec. Returns each run's drift score, severity, axes breakdown, and timestamp — useful for trending charts and reasoning about when behavior shifted.

NameTypeReqDescription
limitinteger——
spec_idstringyes—

No output schema declared.

No examples provided.

list_endpoints ~37

List the LLM endpoints currently monitored in this workspace. Returns id, name, provider, model, base_url, created_at for each.

Input schema present but exposes no named parameters.

No output schema declared.

No examples provided.

list_specs ~72

List behavioral specs in the workspace. A spec is a stored prompt + expectation that ModelWatch replays on a schedule and diffs against a baseline. Returns id, name, prompt, frequency, threshold, last_severity, and the parent endpoint_id.

NameTypeReqDescription
endpoint_idstring—Optional. Filter by endpoint.

No output schema declared.

No examples provided.

reset_baseline ~69

Clear a spec's baseline. The next run will record a new baseline instead of being diffed against the old one. Use this after you've intentionally changed your prompt template, model version, or the behavior you expect — otherwise every future run will look like drift.

NameTypeReqDescription
spec_idstringyes—

No output schema declared.

No examples provided.

run_spec ~84

Run a spec on demand and return the drift score immediately. Use this to (1) set the baseline manually right after create_spec, or (2) sanity-check a spec without waiting for the next scheduled run. Returns the drift score, severity bucket, per-axis scores, and the drift_event_id if one was created.

NameTypeReqDescription
spec_idstringyesFrom list_specs.

No output schema declared.

No examples provided.