# llmshift (pypi · llmshift)

Find retiring LLM calls in code and test the replacement model on real traffic before switching.

- Trust score: 67/100 (medium)
- Change this week: +3
- Registry status: active
- Liveness: live
- Owner verified: no
- Last scored: 2026-11-05

## Components

- pypi · `llmshift`: 67/100 (this document), [markdown](https://verifymcp.io/servers/dechuks-hub-llmshift/llmshift.md), [page](https://verifymcp.io/servers/dechuks-hub-llmshift/llmshift)

## Channel facts

- Registry: `pypi`
- Package: `llmshift`
- Version: `0.1.7`
- Transport: `stdio`

## Trust breakdown

How this component scores in each security and reliability category. Every signal is checked automatically from public evidence about the published package, including repeated runs of it in an isolated sandbox, and we only credit what we can confirm. Scores are 0–100 per category. Scoring method: https://verifymcp.io/docs/scoring (what has changed: https://verifymcp.io/docs/scoring/changelog)

Scored 2026-11-05.

- **Supply Chain Security**: 50/100
  - Malware scan not yet available for this package.
  - No known CVEs affecting this package version or its production dependencies.
  - Runs hatchling.build at install time, a recognised build step with no custom scripting around it.
  - 0 of 1 dependencies flagged as unhealthy.
- **Provenance & Transparency**: 100/100
  - Source repository is publicly reachable at the declared URL.
  - Cryptographically verified build provenance (signed, bound to DeChuks-hub/llmshift).
  - Clear OSI-approved license (MIT).
  - Actively maintained (last published 6 days ago).
  - Publishes a security disclosure policy (SECURITY.md).
- **Schema Quality & AI Usability**: 57/100
  - AI-judged instruction clarity (good).
  - Context-footprint check failed: tool/resource definitions use about 2815 tokens (~281/item across 10 items; 10 tools + 0 resources), over budget; trim descriptions and params.
  - Usage-examples check failed: none of the tools include examples.
- **Stability & Change Management**: 40/100
  - Stability observed for 12 of 30 days with no destabilising changes; credit accrues until the full window elapses.
- **Tool Coverage**: 98/100
  - 100% of tools have a non-trivial description (not blank, and not just the tool's name).
  - 94% of tool parameters carry a description.
- **Tool Safety**: 100/100
  - No prompt-injection markers were found in the server instructions, tool names or descriptions we captured.
  - We read all 10 captured tool definition(s), and no name or description among them implies an irreversible operation.
  - An AI judge read all 11 captured unit(s) of tool text and found none that tries to manipulate the model reading it.
- **Capabilities**: 60/100
  - Spec-recency check failed: implements MCP spec 2025-06-18; the latest is 2026-07-28.

## Install

### How do I install the llmshift MCP server?

llmshift runs locally as a PyPI package, launched with uvx llmshift. Ready-made configuration for Claude, Cursor, VS Code, Codex and 5 more is on this page, copied from each client's own documentation.

### Claude

```bash
claude mcp add dechuks-hub-llmshift -- uvx llmshift
```

### Cursor

```json
{
  "mcpServers": {
    "dechuks-hub-llmshift": {
      "command": "uvx",
      "args": [
        "llmshift"
      ]
    }
  }
}
```

### VS Code

```json
{
  "servers": {
    "dechuks-hub-llmshift": {
      "command": "uvx",
      "args": [
        "llmshift"
      ]
    }
  }
}
```

### Codex

```bash
codex mcp add dechuks-hub-llmshift -- uvx llmshift
```

### opencode

```json
{
  "$schema": "https://opencode.ai/config.json",
  "mcp": {
    "dechuks-hub-llmshift": {
      "type": "local",
      "command": [
        "uvx",
        "llmshift"
      ],
      "enabled": true
    }
  }
}
```

### OpenClaw

```bash
openclaw mcp add dechuks-hub-llmshift --command uvx --arg llmshift
```

### Hermes

```yaml
mcp_servers:
  dechuks-hub-llmshift:
    command: "uvx"
    args: ["llmshift"]
```

### Netclaw

```json
{
  "McpServers": {
    "dechuks-hub-llmshift": {
      "Transport": "stdio",
      "Command": "uvx",
      "Arguments": [
        "llmshift"
      ]
    }
  }
}
```

### Vellum

```bash
assistant mcp add dechuks-hub-llmshift -t stdio -c uvx -a llmshift
```

### Other

```json
{
  "mcpServers": {
    "dechuks-hub-llmshift": {
      "command": "uvx",
      "args": [
        "llmshift"
      ]
    }
  }
}
```

## Changelog

Every change recorded for this component, newest first. Days that predate change tracking, or that we cannot explain, say so: "we were watching and nothing happened" and "we were not watching" are different claims.

### 2026-11-05 (score 67, −15)

- [security regression] Malware scan: pass → unverified

### 2026-11-04 (score 82, +16)

- [security improvement] Malware scan: unverified → pass

### 2026-11-03 (score 66, −15)

- [security regression] Malware scan: pass → unverified

### 2026-11-02 (score 81, +1)

No change was recorded against any check on this day. Stability & Change Management went from 27 to 30. That category is still filling its 30-day observation window: 8 days of observed history at the previous scan, 9 at this one. The score rises as the window fills, whether or not the server changes.

### 2026-10-31 (score 80, +1)

No change was recorded against any check on this day. Stability & Change Management went from 20 to 23. That category is still filling its 30-day observation window: 6 days of observed history at the previous scan, 7 at this one. The score rises as the window fills, whether or not the server changes.

### 2026-10-30 (score 79, +15)

- [security improvement] Malware scan: unverified → pass

### 2026-10-29 (score 64, +1)

- [security improvement] Malware scan: unverified → pass
- [functional] Package version: 0.1.6 → 0.1.7

### 2026-10-28 (score 63, −13)

- [security regression] Malware scan: pass → unverified
- [functional improvement] Stability: unverified → 0.13
- [functional improvement] Security disclosure: fail → pass
- [functional] Package version: 0.1.4 → 0.1.6
- [functional] Package version: 0.1.4 → 0.1.5

## MCP tools (10)

### `llmshift_scan` (~232 tokens)

Find model calls and retirement dates

Find every AI model ID the code in this project calls (Python, JS/TS, Go, Java, configs, .env, notebooks) and report each one's lifecycle: retired, retiring (date and days left), deprecated or active, the recommended replacement, and every file:line where it is used. Use it first when asked to migrate off a model, to find all the places to change, and again afterwards to confirm that no retired model remains.

Input parameters:

- `cwd` (string): Project directory to work in, absolute or relative to where the server started (default: that directory).
- `exclude` (array): Glob patterns to skip, e.g. tests/fixtures or **/*.min.js.
- `fail_on` (string): Which states make would_fail_ci true (default retired).
- `include_docs` (boolean): Also scan Markdown and text files.
- `paths` (array): Files, directories or globs to scan (default: scan.paths in llmshift.yaml, else the project).
- `warn_days` (integer): Models retiring within this many days are urgent (default 180).

### `llmshift_models` (~239 tokens)

Look up model retirement dates

Look up retirement dates and replacements in llmshift's model lifecycle registry (OpenAI, Anthropic, Google, Mistral, Cohere, xAI, DeepSeek). Pass `model` for one exact ID (Bedrock, Vertex, Azure and OpenRouter forms are understood) or `query` to search, optionally filtered by provider, status or days until retirement.

Input parameters:

- `cwd` (string): Project directory to work in, absolute or relative to where the server started (default: that directory).
- `limit` (integer): Maximum models returned (default 50).
- `model` (string): One model ID to look up exactly, e.g. gpt-4o-2024-05-13.
- `provider` (string): openai, anthropic, google, google-vertex, mistral, cohere, xai or deepseek.
- `query` (string): Substring of model IDs to search for, e.g. gpt-4o or claude-3.
- `retiring_within_days` (integer): Only models retiring within this many days.
- `status` (string)

### `llmshift_snapshot` (~305 tokens)

Turn usage logs into a test set

Turn LLM usage logs (JSONL, JSON or CSV, optionally .gz: OpenAI chat/responses/batch, Anthropic, Gemini, Helicone, Langfuse/LangSmith exports, llmshift.record logs) into a regression test set: real requests, the current model's answers, and checks inferred from them. Run it once before switching models; llmshift_compare and llmshift_fix use it.

Input parameters:

- `baseline_model` (string): Generate answers with this model for logs that have none (makes model calls).
- `cwd` (string): Project directory to work in, absolute or relative to where the server started (default: that directory).
- `max_cases` (integer)
- `mode` (string): create (default) refuses to replace an existing file; append adds new cases only.
- `models` (array): Only calls to these models (globs, e.g. gpt-4o*).
- `output` (string): Test file to write (default: tests in llmshift.yaml).
- `per_group` (integer): Maximum cases per prompt template.
- `redact_pii` (boolean): Also replace e-mails, phone and card numbers with placeholders.
- `since` (string): Only calls on/after this date (YYYY-MM-DD or ISO-8601).
- `sources` (array, required): Log files, directories or globs.
- `until` (string): Only calls on/before this date.

### `llmshift_compare` (~426 tokens)

Compare a model on the test set

Run the project's test set (made by llmshift_snapshot) on a model and report exactly what changed versus the current model's recorded answers: broken formats (invalid JSON, code fences), wrong or missing facts and numbers, tone and length changes, tool-call differences and refusals. Returns the pass rate, whether it meets the threshold, the changed cases with their failing checks, and HTML/JSON report files. Use it after changing a model ID or a prompt to prove nothing broke. It calls the model once per test case (cached on re-runs).

Input parameters:

- `baseline_model` (string): Also run the old model live; failures it shares are treated as noise.
- `case_ids` (array): Only these case IDs.
- `cwd` (string): Project directory to work in, absolute or relative to where the server started (default: that directory).
- `drop_params` (array): Leave these settings out of every test request (e.g. temperature, once the code no longer sends a setting the model rejects).
- `groups` (array): Only these prompt templates (substring match).
- `judge` (string): Model for LLM-judged checks (default: judge in llmshift.yaml).
- `limit` (integer): Only the first N cases.
- `max_cases_shown` (integer): Changed cases included in the result (default 10).
- `min_pass_rate` (number): Threshold for passed_threshold (default: min_pass_rate in llmshift.yaml).
- `no_cache` (boolean): Call the model even for requests answered before.
- `repeat` (integer): Run each case N times to find flaky cases and narrow the pass-rate range (default 1).
- `tags` (array): Only cases with these tags.
- `tests` (string): Test file or directory (default: tests in llmshift.yaml, usually .llmshift/tests.jsonl).
- `to` (string): The model to test, e.g. openai:gpt-5.6-sol (default: candidate in llmshift.yaml).

### `llmshift_fix` (~329 tokens)

Suggest prompt fixes for a new model

Search for system-prompt changes that make a new model reproduce the current model's behaviour on the test set, validated by re-running the tests on the new model. Returns, per prompt template, the instructions to add or the full suggested prompt, where that prompt lives in the code (file:line), and the pass rate before and after. It never edits code: apply the prompts you agree with, then call llmshift_compare again.

Input parameters:

- `cwd` (string): Project directory to work in, absolute or relative to where the server started (default: that directory).
- `groups` (array): Only these prompt templates (substring match).
- `judge` (string): Model for LLM-judged checks.
- `limit` (integer): Only the first N cases.
- `no_cache` (boolean)
- `optimizer` (string): Optional model that rewrites prompts, e.g. openai:gpt-5.6-sol (rule-based fixes work without it).
- `rounds` (integer): Improvement rounds per prompt template (default 3).
- `source` (array): Where to look for the prompts in the code (default: the project).
- `tests` (string): Test file or directory (default: tests in llmshift.yaml, usually .llmshift/tests.jsonl).
- `to` (string): The new model, e.g. openai:gpt-5.6-sol (default: candidate in llmshift.yaml).
- `write_tests` (string): Also write a copy of the test set with the suggested prompts to this new file.

### `llmshift_pick` (~292 tokens)

Find the cheapest model that still passes

Find the cheapest model that still passes the project's test set: runs the tests on several candidate models at once and reports each one's pass rate (with a 95% range), cost per 1,000 requests and latency, then recommends the cheapest that meets the quality bar, with the saving against today's model (per month when `volume` is given). Calls every candidate once per test case (cached on re-runs).

Input parameters:

- `baseline` (string): The model used today, for its cost (default: baseline in llmshift.yaml, else the recorded model).
- `cwd` (string): Project directory to work in, absolute or relative to where the server started (default: that directory).
- `groups` (array): Only these prompt templates (substring match).
- `limit` (integer): Only the first N cases.
- `min_pass_rate` (number): Quality bar (default: min_pass_rate in llmshift.yaml).
- `models` (array, required): Candidate models, e.g. openai:gpt-6-luna, anthropic:claude-haiku-5-5.
- `no_cache` (boolean): Call the models even for requests answered before.
- `tests` (string): Test file or directory (default: tests in llmshift.yaml, usually .llmshift/tests.jsonl).
- `volume` (integer): Requests per month, to state the monthly saving.

### `llmshift_safety` (~231 tokens)

Is the new model riskier?

Check whether a new model is riskier than the current one: refusals on the project's real requests, prompt-injection probes (hidden instructions in messages, quoted documents or tool results) and leaks of confidential data from the system prompt, all built from the project's own test cases and run on both models. Reports old vs new rates, the change, a paired significance test and examples. Calls both models.

Input parameters:

- `baseline_model` (string): The current model (default: baseline in llmshift.yaml).
- `categories` (array): Which checks to run (default all).
- `cwd` (string): Project directory to work in, absolute or relative to where the server started (default: that directory).
- `no_cache` (boolean): Call the models even for requests answered before.
- `probes` (integer): Probes per category (default 20).
- `tests` (string): Test file or directory (default: tests in llmshift.yaml, usually .llmshift/tests.jsonl).
- `to` (string): The new model (default: candidate in llmshift.yaml).

### `llmshift_check` (~221 tokens)

Test edited prompts

Test edited prompts: finds the system prompts that changed since a git base (default origin/main) and runs only their test cases, with the edited text, on the model in production. Tells you whether a prompt edit changes how the model answers. Use it after editing a prompt in the code. Calls the model once per affected test case.

Input parameters:

- `all` (boolean): Run every prompt template, not only the edited ones.
- `base` (string): Git ref to compare against (default: origin/main).
- `cwd` (string): Project directory to work in, absolute or relative to where the server started (default: that directory).
- `files` (array): Only look at these changed files.
- `min_pass_rate` (number)
- `model` (string): Model to test with (default: baseline in llmshift.yaml, else candidate).
- `no_cache` (boolean)
- `tests` (string): Test file or directory (default: tests in llmshift.yaml, usually .llmshift/tests.jsonl).

### `llmshift_upgrade` (~262 tokens)

Plan a model upgrade

Plan (and optionally apply) the code changes that move this project off a retiring model: swaps the model ID exactly where llmshift_scan finds it, renames max_tokens to max_completion_tokens where OpenAI's newer models need it, and lists settings to check by hand. Returns a unified diff. Nothing is written unless apply is true. Use `auto` for every retiring model that has a known replacement.

Input parameters:

- `apply` (boolean): Write the changes to the files (default false: only the diff). A file git cannot restore is first copied to NAME.bak, listed in `backups`.
- `auto` (boolean): Every retired or retiring model found that has a replacement.
- `cwd` (string): Project directory to work in, absolute or relative to where the server started (default: that directory).
- `force` (boolean): Allow swaps across providers (the calling code is never rewritten).
- `from` (string): The model to move off, e.g. gpt-4o-2024-05-13.
- `paths` (array): Where to look (default: scan.paths in llmshift.yaml, else the project).
- `to` (string): The replacement (default: the registry's recommended replacement).

### `llmshift_doctor` (~100 tokens)

Check the llmshift setup

Check the llmshift setup: version, the config file in use, which provider credential environment variables are set (names only, never values), whether the test set exists, and whether the configured baseline, candidate and judge models resolve. Use it when another llmshift tool fails with a configuration or credentials error.

Input parameters:

- `cwd` (string): Project directory to work in, absolute or relative to where the server started (default: that directory).

## Diagnostics

Captured diagnostic sections: Provenance, Install scripts, Dependencies. The full working is on the page: https://verifymcp.io/servers/dechuks-hub-llmshift/llmshift#diagnostics

## Score history

- 2026-11-05: 67
- 2026-11-04: 82
- 2026-11-03: 66
- 2026-11-02: 81
- 2026-11-01: 80
- 2026-10-31: 80
- 2026-10-30: 79
- 2026-10-29: 64
- 2026-10-28: 63
- 2026-10-27: 76
- 2026-10-26: 76
- 2026-10-25: 76
- 2026-10-24: 61

## Common questions

### What is the llmshift MCP server?

llmshift is an MCP server listed in the public MCP registry as io.github.DeChuks-hub/llmshift. Find retiring LLM calls in code and test the replacement model on real traffic before switching. This page covers its PyPI package (llmshift).

### Is the llmshift MCP server safe to use?

llmshift scores 67 out of 100 on VerifyMCP. We found no known CVEs affecting it as of 5 November 2026. Its build provenance is signed and verified. That is a record of what we were able to check automatically, not an endorsement. The category breakdown on this page shows every signal behind the number, including the ones we could not confirm.

### What tools does the llmshift MCP server expose?

llmshift exposes 10 tools: llmshift_scan, llmshift_models, llmshift_snapshot, llmshift_compare, llmshift_fix, and 5 more. Their descriptions and schemas cost roughly 2,637 tokens of context every time the server is loaded.

### Is the llmshift MCP server still maintained?

llmshift is still listed as active in the MCP registry. We last reached this channel on 5 November 2026. Those dates come from our own scans of the registry and the channel itself, not from anything the publisher announced.

### What licence is the llmshift MCP server under?

llmshift declares the MIT licence, which is OSI-approved. That covers the source only, and says nothing about the cost of any service it calls.

## Links

- PyPI project: https://pypi.org/project/llmshift/
- Socket report: https://socket.dev/pypi/package/llmshift
- Repository: https://github.com/DeChuks-hub/llmshift
- Changelog RSS feed: https://verifymcp.io/servers/dechuks-hub-llmshift/llmshift.xml
- Changelog JSON feed: https://verifymcp.io/servers/dechuks-hub-llmshift/llmshift.json
- HTML version of this page: https://verifymcp.io/servers/dechuks-hub-llmshift/llmshift