com.framesail/framesail

Switch the project's active script to another saved version (ids come from get_script's version list — each generate_script run creates one; save_script edits the active version in place). Re-run scan_script / rescan_voice_blocks afterwards if the text differs, since downstream artifacts follow the active version.

Add background music to the project from the audio library (find track ids with browse_audio_library, category="music"). Defaults loop the track under the whole video at bed level (volume 0.12 ≈ -18.4 dB under narration — don't raise it without being asked); re-run export_video to hear it.

Attach a sound effect from the audio library to a segment (find track ids via browse_audio_library with category="sfx"). It lands on the segment's scene immediately — no build_scenes needed.

Re-run style analysis (after changing a style's inputs). Async — await_jobs(style_id=...) until the style_analysis job completes.

Animate one segment in a single call: flip it to a generated video shot (keeping its rendered image as the clip's first frame) and START the clip render immediately. BILLS video credits on this call — the segment's image must already be rendered (400 otherwise). A refused generation (out of credits, already running) rolls the flip back, so the segment is either animating or exactly as it was. This is the ONLY way to a generated video (voice=true for a Talking Head) — change_segment_type refuses that target; it owns the other kind switches (real media, overlay scene, back to a still — segment_type "image" with carry_frame=true reverts an animated shot for free). Async — returns {ai_job_id, segment}; await_jobs until the clip completes.

Block (server-side) until the scope has no pending/running jobs, or the timeout passes — use this instead of polling get_workflow_status yourself. Returns {done, jobs}. If done=false the work is still running: just call await_jobs again (a 3-5 minute storyboard takes a few consecutive calls). Keep timeout_seconds <= 50 so the client doesn't time out the tool call.

Browse the audio library for background music and sound effects. category: "music" | "sfx". Returns {tracks} — track ids feed add_music_track / add_segment_sfx. Zero matches also returns the mood and genre tags the library actually carries, so retry with one of those rather than guessing new filter words.

Compile segments + assets + voiceover into the editor/render timeline (scenes). Run after segment assets are complete, before export. Returns a receipt — {scene_count, scenes: [{scene_id, segment_number, duration_frames, status, layer_count}]}; composition detail via list_scenes.

Change what a segment's base visual IS: a generated still ("image"), fetched real media (media_source="real" — a real photo for "image", stock b-roll footage for "video"), or an overlay scene. Generated video is NOT set here — it's the state a rendered still reaches through animate_segment (voice=true for a Talking Head), and a "video" target without media_source="real" is refused with that guidance. segment_type "image" with carry_frame=true reverts an animated shot back to its still for free. media_source="real" turns the shot into fetched media with no start frame and no generation. carry_frame=true reuses the already-rendered image instead of recreating it; ignored for a real target. SFX and overlays always survive a type change. dry_run=true previews what would be kept / staled / recreated / deleted before you commit.

Merge a segment with an adjacent one (segment numbers must be neighbors). keep: "this" | "other" — whose creative data survives: its assets are kept (rendered frame/clip marked stale against the combined narration and its prompts re-derived); the other segment's assets are deleted. Later segments renumber — re-check get_segments before further edits.

Manually add a character/environment/object the scan missed. asset_type: "character" | "environment" | "object". The description is the generation-facing spec of its look — be specific. The scan reads narration and `[SCENE: ...]` directions, so the common miss is anyone NEITHER ever names — a host the video needs but no text mentions, a recurring set the script only implies. Name the asset exactly as the script or directions will refer to it so the storyboard binds the two, then generate_asset_reference it like any scanned asset.

Create a new channel — the container for projects and their reusable styles. Use when the user wants a fresh creative identity rather than adding to an existing channel.

Create a project. The description (the video concept/topic) seeds script generation, so write a meaningful one. Pass video_format='portrait' for a vertical video — every shot, overlay, and the export are then composed for a 9:16 frame. The response's web_url is the project's page in the web app — share it so the user can follow along.

Create a style. Two mutually exclusive paths: References (best): inputs=[{"input_type": "youtube" | "text", "value": "<url or description>"}] — YouTube videos are watched (a channel link or @handle resolves to that channel's newest usable upload) and text directions read; async analysis writes the style's art/narrative/director fields: await_jobs(style_id=...) before using the style. (Image/video FILE references require the multipart REST endpoint POST /styles.) Presets (instant, no analysis): presets={"art_style": id, "narrative_style": id, "director_style": id} — all three axes, ids from list_style_presets.

Delete a project asset (e.g. one the scan over-extracted).

Permanently delete a project and everything in it (script versions, assets, voiceover, segments, renders). Irreversible — confirm with your user first.

Delete a style (e.g. a failed analysis experiment). Don't delete a style that projects still use as their default — rebind them first with set_project_style.

Edit ONE scene with a natural-language note (the same director chat the editor UI uses): move/restyle/add/remove layers and overlays, change motion, etc. Storyboard image, video and voiceover layers can't be removed. Synchronous — returns summary, mutations, skipped_mutations (edits that did NOT land, with reasons — check it) and the scene's layers ({id, name, layerType, segmentAssetId}); list_scenes(include_composition=True) has the full composition. Address the scene by project_id + segment_number (preferred) or a scene_id from a fresh list_scenes call. Scene durations are locked to the narration: don't ask to extend/shorten any but the final scene. remove_segment_asset deletes an SFX or overlay by id without a model in the loop; for whole-video notes, project_director_note.

Render the final MP4 (Remotion). Fetches the current timeline and queues the render. Async — poll get_workflow_status for the video_export job, then call get_video_url.

Render an asset's reference image in the channel's art style — the visual anchor that keeps a character/environment looking identical across every shot. EVERY character, environment, and object asset needs one before generate_voiceover (the server enforces this; fire the jobs for all assets, then one await_jobs). Async — the job writes the image onto the asset row: await_jobs(project_id), then list_assets and view_image the file_path to check likeness.

Generate the project's script from its description/concept and the channel's narrative style. Async — returns {job_id}; poll get_workflow_status.

Render every actionable segment asset (images, video clips, overlays) across the project, in dependency order. THE most expensive call in the pipeline: ALWAYS dry_run=true first, show your user the estimate next to get_credit_balance, and wait for a fresh yes before the real run — prior blanket permission ("do the whole thing") does not cover this spend. The staged flow is cheapest: asset_scope="no_clips" first (images + overlays), review, then animate_segment the shots that deserve motion. Pass segment_numbers to render only a subset — e.g. segments 1-18 for the opening minute before committing to the full video. Safe to re-run: completed and currently-generating assets are skipped, so a second call only picks up new/failed work. Async — one job per asset; await_jobs until all complete.

Plan the full visual storyboard: segments, shot pacing, image/video prompts, overlays, continuation chains — driven by the channel's director and art styles. Requires voiceover to exist (timing comes from it). Plans generated stills + real media only (real media requires the style's @real-media tag); generated video is added per segment after the images render, with animate_segment. Async — returns {job_id}; this is the longest LLM step.

Render one of a style's two template images — a REAL step of style setup, not an optional extra: a style isn't finished until both its character and environment templates are rendered (the app shows them on the style card). Asset reference images render against them (characters → character template; environments and objects → environment template), and segment renders fall back on them when a shot has no asset reference — so finish BOTH before generate_asset_reference. Run once per template_type ("character" | "environment") for every new style; skip types the style already has (get_style's `templates`). A template already exists is a hard stop here — the call refuses unless replace=True, because overwriting one silently re-anchors every future render. Async — await_jobs(style_id=...), then get_style.

Generate TTS audio for the project's voice blocks. Without voice_block_ids it fills gaps: only blocks with no audio yet run, so re-calling it is always safe (already-generated and currently-generating blocks are skipped, never re-billed). Pass voice_block_ids to explicitly REgenerate those blocks (e.g. after changing a block's voice). Speakers must have voices bound first — set_narrator_voice / set_character_voice. Optional editable_sections/settings apply to every selected block (see get_section_template("voice_block") and list_models("voice_block")). On a project that already has a storyboard, segment timings are re-aligned to the new audio automatically when the run finishes (prompts and rendered images untouched) — await_jobs until the project is idle before exporting. Async — returns one job per block.

Current credit balance + plan info. Check before expensive steps (a full segment render can cost hundreds of credits — generate_segments dry_run gives the estimate). Jobs covered by a BYOK provider key bill 0.

THE resume/orientation tool: one call returns every pipeline step's state (script -> scan -> reference_images -> voices -> voiceover -> style_templates -> storyboard -> segment_assets -> scenes -> export), any running jobs, and a next_action telling you exactly what to do next. Call this when picking up an existing project, after any await_jobs, or whenever you're unsure where a video stands — never guess pipeline position. Every step carries a web_url — the page in the Framesail web app where your user can see that step's output. Share it whenever you present a step's results ("review the storyboard here: ...") so they can inspect and edit in the browser; the app and this API act on the same live state.

Fetch a project row — settings, voice config, default style, export URL.

Read the active script's full text + the version list. Use this to show the script to your user for review/feedback before scan_script — the review-edit-resave loop (get_script -> discuss -> save_script) is the expected workflow when the user wants input.

Inspect the prompt sections a generation job exposes for per-call override via editable_sections (jobs: script, script_scan, storyboard, segment_image, segment_video, voice_block, ...). Sections marked locked cannot be overridden.

List one segment's assets (images/video/overlays) including their status, config (prompts, model), and public URLs of rendered files — pass an image's public_url to view_image to actually look at it.

List the storyboard's segments (narration span, type, duration, creative direction). The 1-based segment_number is the handle every segment tool takes (update/split/combine/continuation/regenerate) — you never need a UUID. Returns {total, offset, returned, segments}; on big projects page through with offset/limit rather than fetching all segments in one call. Each segment reports `continues_from_segment`: when set, this shot CONTINUES that earlier segment's frame (same composition, only the delta changes); null is a fresh shot. Adjust with set_segment_continuation. Each segment's segment_assets also lists its overlays (asset_type ui_element, text under config.overlay) — the one call that audits every lower third / title card in the project.

Fetch one style row — its inputs (reference material), analyzed fields (art_style, narrative_style, director_style, script_prompt, ...), and `templates`: {"character": url|null, "environment": url|null}, the two template images. A null there means that template is genuinely missing and needs generate_style_template; a URL means it exists (view_image it) — don't re-render it. Show fields to the user for review; fine-tune with update_style_fields. `inputs_stale: true` means the references changed since the last analysis run — the analyzed fields don't reflect them; offer the user analyze_style (it rewrites those fields).

Download URL for the most recent completed export.

Poll this between steps: returns active + recently-finished AI jobs (scope by project_id, or style_id for style analysis), plus per-segment- asset render statuses for projects. A step is done when its jobs reach status=complete (or error, with a user-readable message). NOTE: finished jobs drop out of `jobs` after ~30s — a short list does NOT mean work was lost; judge render batches by `segment_assets` statuses (or get_pipeline_progress), never by counting jobs. Prefer await_jobs over polling this in a tight loop.

List the project's assets extracted by scan_script — characters, environments, objects. Each has a description (the spec every shot uses to render it — surfaced top-level here; the raw row nests it at ai_output.description), an optional reference image (file_path is a public URL — view_image it), and for characters a voice_id. Review these after scan_script: fix descriptions, then generate_asset_reference for each one (all of them need a reference image before voiceover). asset_type filter: "character" | "environment" | "object".

List your channels. Every project lives in a channel, which owns the reusable styles (art/narrative/director) that drive generation.

List the models allowed for a generation job, with display names, credit estimates, and each model's settings_schema — the valid keys for that tool's `settings` param (e.g. image quality/orientation, video duration). When model is omitted the server picks: the account's saved expert-drawer choice for that job/scope if one exists, else the first entry here. Voice models carry a `provider` field — a voice_block model must match the project's voice_tts_provider or generate_voiceover rejects it.

List the project's background music tracks (volume, loop, timing).

List projects in a channel.

List registered BYOK providers (masked — only the last 4 characters). Returns {keys: [...]}; an empty list means no keys are registered (every job bills platform credits).

List the project's scenes in timeline order. Default rows are light summaries ({scene_id, segment_number, duration_frames, status, layer_count}) — enough to address a scene by number or id; include_composition=True returns the full layer/layout JSON (bulky — page with offset/limit on long projects). Returns {total, offset, returned, scenes}.

The curated preset catalog for the no-AI style creation path, grouped by axis (art_style / narrative_style / director_style). Show the user the labels + descriptions and let THEM pick one per axis — don't choose silently. Art presets include preview image URLs (view_image works on them). Create with create_style(presets={axis: id, ...}) — instant, no analysis job. Full field text lands on the style row (get_style shows it after creation).

List the channel's style rows (variable groups). Styles hold the art_style / narrative_style / director_style / script_prompt fields that drive every generation step, plus any custom @variables.

List the project's voice blocks (per-speaker narration chunks) with their audio status and assigned voices. A block's `scene_direction` is the script's `[SCENE: ...]` direction governing it (never spoken; null when the span carries none). Word-level subtitle timings are stripped unless include_subtitle_data=True.

List available TTS voices (id, label, preview audio URL) for a provider: "minimax" (default engine) or "elevenlabs". Match the project's voice_tts_provider (see get_project) so picked ids work with its engine. Returns {groups: {name: count}, voices}; the ElevenLabs catalogue is 150+ voices, so pass group (names from `groups`) rather than reading it all at once.

Apply a project-WIDE director note ("make the intro punchier", "all captions bigger"). A routing pass picks the scenes the note applies to and edits each — at most 10 scenes per call, so a change touching more needs repeat calls or per-scene director_note. Synchronous — a few seconds per scene. Returns per-scene results (summary, mutation_count).

Regenerate a segment's primary image or video with optional overrides — the API equivalent of the editor's expert drawer. asset_type: "image" | "video" (for a video segment, "image" targets its start frame). Use a different model, override prompt sections (see get_section_template("segment_image")), or tweak settings (e.g. image quality), then re-run just this asset. Async — returns {job_id}.

Remove a music track from the project.

Delete a segment's sound effect or overlay (lower third, title card, callout) — the asset and every scene layer built from it, immediately. These are the only removable assets: the segment's image, video and voiceover are structural. With one SFX/overlay attached, no id needed.