io.github.KhiipAI/khiip

Capture a URL into the Khiip substrate. Dispatches to the appropriate extractor (X, Reddit, web article, Wikipedia, YouTube, PDF). Stores raw source bytes at Source-tier plus a typed payload in the vault. Submits to Wayback Machine unless disabled. Idempotent by default (dedup by URL hash). Runs as a daemon-side capture job polled to completion, forwarding per-stage progress (extract → enrich → media → preserve → wayback → render → write → embed; media reports item counts) as MCP progress notifications when the client requested them. Args: url: The URL to capture. force_new: If True, skip dedup-by-url and force creation of a new capture (overrides the existing capture's `superseded_by` pointer). Defaults to False. Returns: Capture record (id, source, vault_path, payload, archive_urls, etc.) on success; structured error dict on failure.

Combined daemon health + metadata. Aggregates GET /health (extractor + wayback liveness) and GET /api/v1/meta (version, configured extractors, embedder model + dimension). Useful as a first call when an agent connects, to confirm the daemon is reachable and to learn what sources + capabilities are available. Returns: On success: `{"health": {...}, "meta": {...}, "mcp_server_version": "<khiip package version>"}`. On daemon-unreachable: structured error dict.

Fetch a single capture by id, in the requested format. Prefer `json` for comment-heavy sources (Reddit, YouTube) — the typed JSON payload measures about half the size of the markdown render there. Args: capture_id: The capture ULID. format: One of `json` (default; typed payload as JSON), `markdown` (full vault file — frontmatter + body), `legacy-markdown` (body only, no frontmatter). YouTube captures also accept `srt` / `vtt` (transcript export). Returns: On success: payload in the requested format wrapped in a `{"format": ..., "body": ...}` envelope. On failure: structured error dict.

List captures, newest first. Returns LEAN rows — flat {id, url, source, title, author, recorded_at, vault_path, excerpt, excerpt_from, payload_chars, description}; use get_capture(id) for a full record and `payload_chars` to budget that call. Args: source: Optional filter by source name (x, reddit, web, wikipedia, youtube, pdf). When None, returns all sources. limit: Page size (1-500; default 50). offset: Pagination offset. Returns: On success: {"captures": [lean rows], "count": int}. On failure: structured error dict.

Semantic recall over captured payloads by natural-language query. Ranks captures by cosine similarity over typed-payload embed-text composition (per ADR-0009 §C7). Returns top-k LEAN hits — flat rows {id, url, source, title, author, recorded_at, vault_path, score, excerpt, excerpt_from, payload_chars, description}. Lean hits are the contract; the drill-down for a full record is get_capture(id). Prefer get_capture(..., format="json") for comment-heavy sources (Reddit, YouTube) — the JSON payload measures about half the size of the markdown render there. `payload_chars` is the exact character count of that format="json" body, so you know what you're in for before fetching. `excerpt` is the head (up to 500 chars) of the source's primary text; `description` is the source's own summary when it has one. Args: query: Natural-language recall query. limit: Maximum number of results to return (1-100; default 10). min_score: Optional cosine-similarity floor in [-1.0, 1.0]. Hits scoring below it are dropped. Scores are raw cosine: negative / near-zero means unrelated; roughly 0.3+ is practically relevant for the default MiniLM embedder. Default None = no filtering (all top-k returned). Returns: Recall response with `query`, `embedder_model`, `embedder_dimension`, and `results` (list of lean hits).

Refetch one dimension of an existing capture. Dimensions, per ADR-0010 commitment 6 + ADR-0009 §C2 replay (S42): - `extraction` (default): NETWORK re-fetch → new capture, old marked superseded (can observe a changed/deleted post). - `re-extract`: OFFLINE — re-derive the typed payload from the preserved source bytes with the current extractor (recovers under-extracted fields; survives source deletion). In place; same capture id. - `re-render`: OFFLINE — re-render the markdown body from the existing typed payload (apply an improved renderer / skin). In place. - `media`: re-walk MediaFetcherRegistry on existing payload; updates `Media[].download_status` in place; same capture id. - `wayback`: re-submit canonical URL to Wayback Save Page Now; updates `archive_urls['wayback']` in place; same capture id. Args: capture_id: The capture ULID to refetch. dimension: Which dimension to refetch (extraction | re-extract | re-render | media | wayback). Defaults to `extraction`. force: Overwrite the vault note even if it was modified since khiip last wrote it (the daemon otherwise refuses in-place rewrites with a 409 to protect hand edits); for dimension=media also re-fetches already-successful items. Defaults to False. When a refetch returns a `daemon-409` error, report its `detail` to the user and ask before retrying with force=True — forcing discards their edits. Returns: Updated capture record on success; structured error dict on failure.