vecscope

Audit an embedding file: hubness, anisotropy, collapse, rogue dimensions, norms, metric mismatch, distance concentration, outliers, k-NN graph islands, and optionally ANN recall and query-side findings. Read-only; the file never leaves the machine. `path`: local .npy/.npz/.parquet/.jsonl/.json/.csv with one vector per row, a faiss index file, or a vector store URI (qdrant://, chroma://, milvus://, weaviate://, or their +local forms with a directory). When the server was started with --allow-dir or VECSCOPE_ALLOWED_DIRS, paths outside those directories are refused; network stores need the server started with --allow-network. Store rows are in ascending id order and findings carry `sample_source_ids` next to `sample_ids`. `ann_from_source`: query the store's own index for every stored vector and score it as `ann_path` (needs `sample=0` above the sample size). `qrels_path`: relevance judgments for the queries (.json/.jsonl/.tsv of query row, corpus row, grade); adds `retrieval` with nDCG, recall, MRR and duplicate share of exact search. `ranked_path`: (m x k) corpus row ids per query from the production index, scored with `qrels_path` next to exact search. `ann_path`: optional (n x k) integer array of neighbor ids from an ANN index (requires `sample=0` so row ids line up). `queries_path`: optional (m x dim) file of query embeddings; adds query_gap, query_hubness and the deprecated ood_queries. `query_baseline_path`: optional JSON evidence (this tool's response or `vecscope audit --json`) from an earlier audit of the same corpus with healthy queries; with `queries_path` it adds query_shift, the share of queries farther from the corpus than that reference's median + 3 MAD. `clusters` > 0 attaches a per-cluster map with sample ids (no verdicts); capped at one cluster per 10 rows. `sample`: rows above this are audited on a seeded random subset (0 = audit every row; exact k-NN is quadratic, minutes for 100k rows on a laptop; peak memory about 2 GiB). `sample_repeats`: whe…

Compare two embedding files whose rows are the same items in the same order (two models, or one model before and after a transform). Reports how often the nearest neighbor and the top-k lists agree, the near-duplicate share in each space and in both, whether both spaces pick the same duplicate partner, and the rows that disagree most. No verdicts: duplicates flagged by both spaces with the same partner belong to the items; duplicates flagged by one space only point at that model. Above `sample` rows a seeded subset is compared (0 = all rows). Read `neighbor_scope` before quoting the agreement: "sample" means the neighbors were recomputed inside the sampled rows, so the numbers describe the sample's own neighbor graph and are usually higher than the full corpus's. To compare full-corpus rankings, pass `neighbors_a_path` and `neighbors_b_path` (both): (n x >= k) neighbor ids per row computed on the full corpus in each space, exact or from the production index; the rows may still be sampled but their neighbors are not (`neighbor_scope` = "full", `neighbor_source` = "supplied").

List every finding vecscope can report with its metric key, default (warn, fail) thresholds and direction, likely causes, suggested first action and verification step. Use it to explain a verdict to the user or to decide which thresholds to override.

Sweep HNSW (M, ef_search) and IVF (nprobe) parameters on an embedding file and report recall@k against exact k-NN, build time, queries per second, single-query latency percentiles (p50/p95/p99 ms) and the serialized index size for each setting, plus the fastest configuration that reaches `target_recall`. The recommendation carries `margin` (recall minus target) and is marked `tight` with a `safer_alternative` when the margin is under `margin`, because recall at fixed parameters falls as the corpus grows; relay `caveats` to the user. qps is the median of `timing_repeats` runs. Needs the faiss extra (`pip install "vecscope[index]"`). Read-only; timings are relative to this machine and process, not a serving latency. `device` is "auto" (GPU when the faiss build has one), "cpu" or "gpu"; HNSW is CPU-only. `n_queries` caps the queries timed (sampled from the corpus, or from `queries_path`); pass real query traffic as `queries_path` whenever it exists, because the best setting on corpus-drawn queries can differ. `latency_queries` are searched one at a time per setting for the percentiles (0 = skip). `nlist` fixes the IVF list count; pass the production index's nlist when tuning its nprobe, since the default follows the row count (4*sqrt(n)) and gives other lists.