io.github.kelvinbiffi/ollos-mcp

Stop a queued or running ollos job by jobId — for example when the user changes their mind about a long transcription, or asked for the wrong file. The running stage is aborted (ffmpeg and model inference stop within a second) and the job is marked cancelled; partial artifacts already written stay on disk and are not reused. A job that already finished is reported with its final status and cancelled: false — not an error. Example: {"jobId":"j_a1b2c3d4e5f6"}.

Split a recording into speaker turns — who talked from when to when — and, if you pass the jobId of a finished ollos_transcribe, label every transcript segment with its speaker. Runs locally: pyannote segmentation, WeSpeaker voice embeddings and agglomerative clustering. For a Zoom local recording folder with one audio file per participant it uses the tracks directly and the result is exact and named. Each speaker gets an 8-second voice clip so a person can rename SPEAKER_00 by ear. Experimental: the similarity threshold (default 0.35) decides whether two voices are one person, and speech with background music (intros, jingles, outros) tends to come out as an extra speaker; pass maxSpeakers when you know the count. Example: {"source":"meeting.mp4","transcriptJobId":"j_…","maxSpeakers":3}.

Return contact sheets or individual frames from a finished ollos_keyframes or ollos_read_screen job as images you can look at. Ask for sheets first (each shows 9 timestamped frames); ask for a single frame only when you need a close-up. Default 6 images per call (maxImages, up to 12) to protect the context window. Example: {"jobId":"j_…","sheets":[1,2]} or {"jobId":"j_…","frames":[14]}.

Get the state of a job started by ollos_transcribe, ollos_keyframes, ollos_read_screen, ollos_review or ollos_diarize: stage, percentage, and — once completed — the formatted result plus resource links, so no second call is needed. Poll every few seconds; the server keeps jobs on disk, so a jobId survives a restart (an interrupted job says so instead of hanging). Example: {"jobId":"j_a1b2c3d4e5f6"}.

Pick the frames of a video that carry information and pack them into 3×3 contact sheets, so you can "watch" an hour of video in a handful of images. Frames are chosen by perceptual-hash change (works on screen recordings, where scene detection sees nothing), hard cuts (window switches, modals), optional anchors, and a floor of one frame every 20 s. Each tile shows its timestamp. Returns sheet resources; use ollos_frames to view them. Set presenterRegion to ignore a webcam overlay when comparing frames. Limitations: video only; about 0.3 s per extracted frame. Example: {"source":"lesson.mp4","sensitivity":"normal","maxFrames":120}.

Read what a file, URL or folder actually is, in under a second: kind (video/audio/image), duration, resolution, aspect ratio and which platforms it fits, codecs, fps, audio channels and track count. Use it first, before any other ollos tool, to decide what to run and to detect a Zoom recording folder with one audio track per participant. Never trusts the file extension; a .mp4 without video is reported as audio. Accepts a local path, an http(s) URL, a video-site link (needs yt-dlp), a data: URI or a Zoom local-recording folder. Example: {"source":"talk.mp4"} or {"source":"https://www.youtube.com/watch?v=…"}.

OCR the text visible in a video or image and scan it for secrets: API keys, JWTs, bearer tokens, .env lines, private deployment URLs, local URLs, e-mails. Frames are selected as in ollos_keyframes, then read in upscaled tiles (small UI text is unreadable otherwise). Secret findings combine three signals — known patterns, high-entropy tokens, and nearby UI words like "API Key Created" — and are ALWAYS masked; the value never leaves this tool. Use it before publishing a screen recording, or to search what was on screen. Slow: roughly 1–3 s per frame depending on CPU, so a 10-minute screencast is a job of a few minutes. Example: {"source":"demo.mp4","detectSecrets":true}.

Check a video before it goes public and return a verdict (ok / warn / block) with findings: loudness vs the platform target (YouTube -14 LUFS) and true peak, silence gaps worth cutting, aspect ratio vs the platform (a 1890×1080 file gets black bars on YouTube), and secrets visible on screen. The secrets check reads frames with OCR and is the slow part; drop it from checks for an instant audio/aspect review. Findings say where (timestamp) and what to do. Example: {"source":"episode.mp4","platform":"youtube","checks":["loudness","silences","aspect","secrets"]}.

Find moments across everything ollos has transcribed or read from screen: "what did we decide about the deadline", "when did the 401 error appear". Hybrid retrieval — BM25 for exact names, acronyms and numbers plus multilingual embeddings for meaning, fused by reciprocal rank — returning up to k passages with timestamp, speaker and source job, never whole transcripts. Scope "all" (default) searches every completed job; scope "job" with jobId searches one. The first call loads a 465 MB embedding model (a few seconds); indexes are built lazily and kept on disk. Example: {"query":"como configurar o webhook","k":5}.

Transcribe the speech in audio or video, locally with Whisper, with timestamps per segment and a heuristic confidence. Silence is skipped with a voice-activity detector and known Whisper hallucinations are removed, so what you get back was actually said. Pass vocabulary with domain terms ("Claude Code", "n8n", "webhook") to fix phonetic confusions. Runs at about 1.7× real time on a 16-core CPU (≈0.6 s of compute per second of audio): an 11-minute video takes ~7 minutes and comes back as a job to poll with ollos_job; a 1-minute clip returns inline. Limitations: needs an audio stream; confidence is heuristic, not a model probability; the reported language label is guessed among pt/en/es — other languages transcribe but are labelled auto. Example: {"source":"talk.mp4","language":"pt","vocabulary":["Claude Code","MCP"]}.