Misata Studio: verified synthetic data

The reference for the engine's full design language (the `blueprint` argument of generate_dataset, start_generation and validate_blueprint): roles, distributions, formulas that read parent columns and draw noise, per-group sequences (seq, lag, cumsum, ar1 drift, random walk), aggregates, event windows, cause-and-effect, lifecycles, with patterns, and an index of chapters. Read it before writing a blueprint. Args: topic: a chapter's id from the index (or several, comma-separated) for that chapter in full, with a world that builds as written: JSON payloads (webhooks, audit trails), threads of messages, one party's duplicate records, running balances, rosters made from demand, a source system's raw drops and schema changes, planted problems for an eval. Empty for the reference.

Stop a running start_generation job. It stops at the next stage boundary and keeps nothing.

The dataset's ground truth: every number it is held to, and the business's standard numbers (MRR, churn, billed revenue, win rate...), by month, worked out when it was made. Use it to test whatever answers questions about the data -- an agent, a dashboard, a cube, a text-to-SQL model -- and to say exactly why a wrong answer is wrong. Call it four ways: 1. dataset_id alone: the key's numbers (`kpis`: id, title, unit, months) and `questions` to ask, drawn from the key ("What is MRR at each month's end for 2025-07?"), each with an id. Their answers are not shown. 2. answers={question_id: number}: each answer graded within its tolerance; a wrong one is named as the usual misreading it equals (voided invoices counted, churn over every account), when it is one. 3. values with kpi: numbers computed anywhere (ClickHouse, a semantic layer, a BI tile), as one number, a {"2025-07": 123.4, ...} series by month, or [[month, value], ...]. Says whether they are that KPI (a tile is read as its latest month, the year to date, ...), and what they are instead when not. 4. sql (and kpi): a query over this dataset's own tables (DuckDB, as query_dataset), run here; a wrong one is told which KPI it computes, or the one change after which it returns the key. The values are plain numbers, so an answer from any engine is checked the same way; export_dataset(format="demo", dialect="clickhouse") hands over the same data and key to load into ClickHouse.

Export a generated dataset as a file. Returns a `download_url` the person can open (or you can fetch, e.g. with curl) for as long as the dataset is held, about 2 hours. Give the person the link rather than pasting file contents into the chat. Args: dataset_id: from a prior generate_dataset call. format: data: csv, parquet, jsonl, json, avro, xlsx, feather, orc, sqlite, duckdb, sql. code and docs: dbt, notebook, dictionary, dbml, mermaid, prisma, sqlalchemy, typescript, jsonschema, expectations, django, openapi, mockapi. `sql` is schema.sql (DDL with keys) + data.sql (COPY/INSERT) — the way to seed a real database: run the returned SQL through your own database connection, since this server never holds a database credential itself. `demo` is the handover for a demo or an eval: the database with a one-step loader, the ER diagram, answer_key.json (every number by month), questions.jsonl and the proof of each request. dialect: for `sql`: postgres, mysql, sqlite, mssql, oracle, bigquery, snowflake, clickhouse (MergeTree tables; above 500k rows, typed Parquet with load.sql). For `demo`: postgres (psql) or clickhouse (Parquet loaded with clickhouse-client). inline: also return the file itself as `base64` (only for files under a few MB). Use it when you must write the file yourself and cannot fetch a URL. Returns: filename, content_type, bytes, download_url, expires_at (unix seconds), and `base64` when `inline` and small enough.

The ready-made datasets Misata publishes: free sample databases (direct download, public domain) and premium datasets with a full answer key (a free preview, then a one-off price). Check this first when someone wants sample, demo, practice or teaching data for a common scenario (retail, e-commerce, SaaS, manufacturing SPC, predictive maintenance, insurance claims, fraud/AML, clinical, network security): handing over a dataset that already exists is instant. If none fits their tables, generate one instead. Args: query: what the person needs, in their words. Only orders the list (closest first); every dataset is still returned, so judge the fit yourself from the tables and summary. Returns: datasets: [{slug, kind (free|premium), title, summary, rows, tables, page_url, and download_url (free) or free_preview_url + buy_url + price_usd (premium)}].

Make a verified relational dataset and wait for it: every foreign key checked, dates correctly ordered, declared aggregates and rates exact, before anything is returned. Deterministic for a seed. Use this when you give a `schema` or `ddl` (seconds). For a plain-English `request` the engine designs the tables with a model, which can take several minutes: call start_generation instead and poll get_status, so the call does not sit open and time out. Args: request: Plain-English description (needs an LLM key unless `schema`/`ddl` is also given). With no key, design it yourself instead: blueprint_guide, then `blueprint` below. schema: A Misata schema dict for exact structural control. No key needed for structure. Its `numbers` (revenue by month, churn, MRR, shares, averages) are held exactly: see the server instructions. A column's `moves_with` ({"column": "customers.income", "strength": 0.6}) pairs it with another number of its row or its parent's, by rank, each keeping its own values. ddl: CREATE TABLE statements. No key needed for structure. Beside it, schema={"numbers": [...], "window": {...}} holds the story's numbers on these tables. seed: Reproducibility seed — the same request/schema and seed always produce the same rows. research: Ground realistic numbers (prices, growth rates) in real published facts via a web search. Off by default (an anonymous caller's request should not trigger external calls unless asked for); needs a key regardless of `schema`/`ddl`. blueprint: The engine's full design language (blueprint_guide has the reference): readings around a parent's nominal, drift, autocorrelation, cause-and-effect, event windows, exact aggregates. Use it whenever the data must behave like the real process. No key need…

The full certificate for a dataset made by generate_dataset: every claim (stated vs actual), every requirement's status and evidence, realism findings, planted defects/anomalies if any, and the pattern charts. This is the whole answer key, not the trimmed form generate_dataset returns.

Where a start_generation job is. While running: the stage it is in and how long it has run. When done: the same answer generate_dataset returns (dataset_id, verification, tables, certificate).

See the tables, sizes and relationships the engine would build, before any rows exist. Free (no rows are made), so it is worth calling before generate_dataset on anything non-trivial: review what it understood and assumed, then adjust your request or schema before spending a real call. Args: request: Plain-English description (needs an LLM key, unless `schema`/`ddl` is also given — then it is still used to ground realism, e.g. locale and what columns mean). schema: A Misata schema dict (see the server instructions for the format). No key needed for structure. ddl: CREATE TABLE statements. No key needed for structure. Returns: route: "chat" (not a dataset request — see `reply`), "design" (the model designed the tables) or "pack" (matched a built-in shape). tables: name, estimated rows, columns, foreign keys for each table the engine would build. understanding: what the engine read the request as (business, archetype, assumptions). requirements: every specific thing the request asked for, so you can see what was understood.

Read-only SQL over a dataset's own tables (one SELECT/WITH statement; every table name is a view over that dataset's own files — nothing else on the server is reachable this way). Args: dataset_id: from a prior generate_dataset call. sql: a single SELECT (or WITH ... SELECT) statement. No semicolons, file paths, or statements that write or read outside the dataset (checked before running). limit: rows returned (capped at 5000). Returns: columns, rows, truncated (whether more rows existed than `limit`).

Start a generation in the background and return a job_id at once. Use it for any plain-English `request` (a model designs the tables, which takes minutes) — then call get_status(job_id) every 20-30 seconds until it says done. Arguments are the same as generate_dataset. Returns: job_id, status "running". get_status gives the stage and, when finished, the dataset.

Check a blueprint before generating it: every error said as what to change, the design traps it falls into (a column that will round away, a copy of a value not made yet, a window that counts nothing), its size against your row cap, and a small preview run (a few thousand rows) with the verifier's findings and sample rows, so you can see the data behave before the real call. Free. Returns: valid, errors (fix these), warnings (read these), planned (sizes the stated numbers set; "keep_size" on a table keeps its row_count), estimated_rows, row_cap, preview: {passed, findings, tables: {name: first rows}} when `preview`.

Which account this connection is, and what it may do: signed in with a Studio MCP key or anonymous, the row cap, and whether a model key is available for plain-English requests.