Hydrata - ANUGA Flood Simulation

Attach a photo to a flood observation as a reference LINK (never a file). MCP never uploads photo bytes. A local path, file:// URL or data: URI is refused: to upload a photo file, open the observation in the map and use the photo strip on its form (JPEG, PNG or HEIC, 25 MB, 10 per observation). The link counts toward the 10.

Add flood observations (Flood mark, Timed level, Gauge record, Dry point) to a layer. POSTs each observation to /projects/<id>/flood-observations/<layer>/observations/ in order. The API validates every field: a row it refuses (400) is listed in `rejected` with its `problems` and the rest still land; a refusal that is not about the row (403 not an Editor, 404, 409) stops the batch and is reported in `stopped`, with `not_sent_count`. Stages are on the LAYER's vertical datum. Photos are not part of an observation: attach a link with add_observation_photo_link.

Attach a GeoJSON you uploaded to GeoNode as the project's boundary, friction, inflow, rainfall, building, mesh_region or culvert layer. The agent moves the bytes: first upload the GeoJSON yourself, with the same agent key, to GeoNode's upload endpoint at the site origin: curl -sS -H 'Authorization: Bearer hk_...' -F "base_file=@/path/rainfall.geojson" https://<site>/api/v2/uploads/upload/ → JSON with `execution_id`. Then call this tool with it. The tool polls GET /api/v2/resource-service/execution-status/<execution_id> (bounded by timeout_seconds; statuses ready → running → finished | failed), reads the new dataset's pk, and PATCHes it onto the project's DEFAULT row of that kind ('Boundary 01' … 'MeshRegion 01' — the six rows the terrain import seeds ~30 s after get_terrain reports ready; if the list is still empty the tool says so: run finalize_terrain_upload / wait for get_terrain first). kind `culvert` (Hydraulic Structures; one two-point line per culvert, Inlet first, then Outlet): no culvert row is seeded, so the tool creates 'Culvert 01' when the project has none and waits for its layer. It reads your culverts back and maps your attribute names onto the culvert fields — name, shape (box|circular), width_m, height_m, diameter_m, upstream_invert_m, downstream_invert_m, barrels (1-20), wall_thickness_m, entrance_type, material (concrete|corrugated_metal), manning_n, blockage_pct (Reduced-area blockage, percent), length_override_m, description — accepting common alternatives (width, diameter, label, us_invert, ds_invert, num_barrels, manning, blockage, length, shape 'pipe' = circular, …) and filling the defaults (barrels 1, wall 0.20 m, square_edge_headwall, concrete, manning_n from the material, blockage 0). Other attributes are kept. If anything changed it re-uploads the mapped culverts and attaches THAT layer (`mapped`: renamed, filled, source_dataset_pk). `culvert_problems` lists what the build will still refuse (typically missing inverts) — the user…

Build a scenario's package (mesh + inputs) after showing what it will cost, and poll until built. Needs a launch-level agent key (a build can start AWS Batch); a read or draft key gets 403 AGENT_LEVEL_INSUFFICIENT — the user creates a launch key on the site's Agent keys page (/agents/). Order of operations, so the number is shown before anything is spent: 1. GET the scenario detail. A record with no `boundary`/`computed_status` keys is a non-member's read of a public project → refused: a build needs the EDITOR role. 2. Re-call checks on `latest_run` (a re-POST is NOT deduplicated after `built`/`complete` — it would dispatch a duplicate build): no run → proceed; run `created`/`building` → resume polling, no POST; run `built`/`queued`/`computing`/`processing`/`complete` with `latest_run_is_valid` not false → return that state, no POST unless rebuild=true; run `error`/`cancelled`, or `latest_run_is_valid` false (the scenario was edited since the build) → proceed. 3. Spend gates, each a refusal with `outcome: "refused"` and no POST: the scenario has no boundary or its boundary row has no features (the server would admit the build and fail it in make_package); the estimate is None (resolution is 0/unset); the estimate is above 100,000 triangles and confirm is not true — the refusal states the number; re-call with confirm=true to proceed. An estimate of 0 over a boundary WITH features is buildable and is reported, not refused. 4. POST /projects/<id>/scenarios/<pk>/build/. Every 4xx comes back verbatim as a normal result with `http_status`: 422 MESH_TOO_LARGE (`error_code`, `estimate`, `ceiling`, `detail` — the server's hard ceiling; coarsen `resolution` or shrink the boundary; no run was created), 409 COMPUTE_TARGET_UNAVAILABLE, 400/403/404. 409 UNREVIEWED_INPUTS (inputs you wrote that the user has not reviewed) answers `outcome` `unreviewed_inputs` with the `unreviewed_inputs` list and a `ui_url` to the s…

Cancel an in-flight simulation run. Needs a launch-level agent key; a read or draft key gets 403 AGENT_LEVEL_INSUFFICIENT. Works on runs in built, queued, or computing status. Cleans up compute resources (terminates EC2 instance, Celery task, or Batch job). Returns 409 if the run is already in a terminal state (complete, cancelled, or error).

Create a flood observation layer in a project and wait for its map layer. A flood observation layer holds real flood evidence (Flood marks, Timed levels, Gauge records, Dry points) that every historic run is compared with (Compare with observations). It is project-level: it is never a scenario input. POSTs /projects/<id>/flood-observations/, then polls the layer until its GeoServer layer exists (`ready`), usually a few seconds; until then add_observations refuses. On `timed_out`, call list_observations later.

Create a new ANUGA project. Returns the project record including its id, base_map and ui_url. The caller's own account becomes the owner. `projection` is the projected CRS every scenario in the project is meshed and run in — pick the UTM zone covering the site. The project's map is created alongside it; `ui_url` opens it — give it to the user. A project you create is unpublished (`is_published: false`): only its owner and members can see it until the user presses Publish in its Sharing panel in the browser — you cannot publish it. Next step for a new project: presign_terrain_upload.

Create a DRAFT scenario (no build, no run) and report its mesh-triangle estimate. POSTs /projects/<id>/scenarios/ with the write fields, then GETs the scenario detail — the create response carries NO estimate; the detail's `mesh_triangle_count_estimate` (+ `_breakdown`) does. Returns a compact record (the same fields get_scenario answers with): id, name, the FK ids as stored, resolution, duration, `computed_status` (`created` = no run yet), the estimate and its breakdown, the price (`compute_cost_estimate` in USD, `vcpu_hours_estimate`), `inflow_anchor_mismatch`, and http_status. Next step: build_scenario (which asks for confirm=true above 100,000 triangles). Nothing is meshed or queued here. Units: `resolution` — on the SCENARIO and on every MeshRegion FEATURE — is a LENGTH in metres; ANUGA maximum_triangle_area = resolution²/2 (FloatField default 100, not nullable; 0 makes the estimate None), and the estimate prices both the same way (TASK-3186). The smallest value becomes the raster cell size. A MeshRegion feature must carry `resolution_units = m` (features drawn in the map are marked automatically); an unmarked one makes the build answer 422 MESH_REGION_UNITS_UNMARKED, naming the conversion an operator runs. The server does NOT validate that the FK ids belong to `project_id` — pass the row ids attach_input_layer / get_terrain returned for THIS project. Every FK is nullable at create time, and the build fails later on a scenario with no terrain, or with neither an inflow nor a rainfall. A `boundary` whose row has no features (the default 'Boundary 01' before attach_input_layer) is accepted here and refused by build_scenario. The estimate is None when resolution is 0; an estimate of 0 with a real boundary is a very coarse mesh, not an error.

Create a time series (rain gauge, hydrograph, tide/stage) in a project. POSTs /projects/<id>/time-series/ with `series_type` and `units` as top-level fields. `series_type` is checked against the four choices and `data` against the {"rowData": [{timestamp, value}, ...]} shape before any request is made — the API answers 500 (not 400) to a malformed row. Returns a compact record — id, name, series_type, units, timezone, row_count, http_status — not the echoed rows; fetch the full row with the REST API (GET /projects/<id>/time-series/<id>/) if you need to round-trip it. A rainfall polygon references its gauge by the series NAME, so create the gauges with the exact names the rainfall GeoJSON's features carry. The units must match the series_type (see `units`), and a hydrograph or hyetograph must END at 0 unless `hold_last_value` is true. The result carries Hydrata's own `summary` of the series (peak, volume or depth, first and last stamp, trailing value): repeat those numbers to the user (`say_to_user` does) rather than computing your own.

Delete one flood observation and its photos (cannot be undone). DELETEs /projects/<id>/flood-observations/<layer>/observations/<fid>/; a missing fid is an error (404), not a silent success.

Step 2 of a terrain import: register the PUT GeoTIFF as a Terrain and start the import. Needs a launch-level agent key (the import can start AWS Batch); a read or draft key gets 403 AGENT_LEVEL_INSUFFICIENT — the user creates a launch key on the site's Agent keys page (/agents/). Call this only after the presigned PUT returned 200. Creates the Terrain row (status `creating`) and queues the import chain — reproject to UTM, publish the layer + hillshade, style — which also seeds the project's six default boundary, friction, inflow, rainfall, building and mesh-region rows. Returns 202 with the terrain record; keep its `id` for get_terrain. A 400 UPLOAD_NOT_FOUND means no object is at `staging_key`: the PUT did not land (check its status code and Content-Type) — do not retry finalize until it has.

Compare a run with the project's flood observations: how well does it match them? Returns `headline` (the figures to tell the user, e.g. "47 observations · 31 within ±0.30 m (66%) · bias -0.12 m · RMSE 0.35 m · 1 excluded"), `headline_numbers` (the same as numbers), `tolerance_m` (used) and `default_tolerance_m` (the project's saved one), `statistics` (the API's summary: n, within, within_pct, bias_m, rmse_m, weighted_rmse_m, mae_m, max_abs_m, excluded, not_scored by reason, gauges), `gauges` (Gauge records: peak error, timing error, NSE), `datum_checks` (warn the user only when `warning` is true: both datums known and different; `not_recorded` = the terrain has no recorded datum, stated in `note`, not a warning) and one row per observation: observed_m / modelled_m / residual_m (modelled - observed, + = model high), within, scored / not_scored_reason, match (`outside_run_window` = not this flood), excluded + exclusion {reason, by, at}. `sampling` counts how the modelled values were read: `centroid` (the peak of the mesh triangle the observation sits in), `nearest_node_fallback` (an older run without that peak: the nearest mesh node's value, tell the user) and `nearest_node` (just outside the mesh); each row's `sample.method` says which. There is no pass or fail: report the figures. A run that is not historic has none (`has_comparison` false) — do not retry it.

Get one ANUGA project: id, name, projection, visibility, your role, its base map — and the link to it. Returns the project record (`id`, `name`, `projection` (EPSG code), `simple_view_config`, `visibility`, `owner_username`, `my_role`, `base_map`) plus `ui_url`: the link that opens the project's map, for the user (`<site>/catalogue/#/map/<base_map>`; null while a new project's map is still being created). Scenarios are NOT listed here — use get_scenario with a scenario id.

Get full details of a simulation run including timing and results. Returns the complete run record: status, progress, timing (start/end timestamps, duration), compute details (backend, instance type, cost), mesh info, error messages, and result log. Use get_run_status for lightweight polling; use this for final results. The presigned `s3_*_url` download links are ELIDED — each is a 6-hour capability and no tool relays one. The results stay viewable through the `gn_layer_*` entries (published WMS layers on the project's map).

Lightweight status check for a simulation run (fast, <50ms). Use this for polling instead of get_run. Returns only: id, status, `phase`, progress_pct (0-100), eta_seconds, error_message, and compute_backend. `phase` is never empty: it is `status`, except `preparing_inputs` while a `computing` run has no progress yet (it is downloading its input package and building the domain — minutes on a big model; this is NOT a stall). Poll every 5-10 seconds. Terminal states: complete, error, cancelled.

Get a scenario's COMPACT state: its `computed_status`, inputs, estimate and price. Returns a compact record, never the raw ~50 KB detail: `scenario_id`, `computed_status`, `mesh_triangle_count_estimate` and its `_breakdown`, `latest_run_is_valid`, the latest run's `run_id`, `run_status`, `error_message`, `user_message`, `status_detail` and `mesh_triangle_count` — plus what the scenario IS and what it will cost: `name`, `description`, `project`, `resolution`, `duration`, the input row ids (`terrain`, `boundary`, `friction`, `inflow`, `rainfall`, `building`, `mesh_region`, `culvert`), `compute_cost_estimate` (USD), `vcpu_hours_estimate`, and `inflow_anchor_mismatch` (null unless an inflow series starts after the model does, whose first value is then held backwards), and `hydrology_check` (when Hydrata sends it: for each inflow and rainfall feature, the series that drives it, its peak and volume, and structured warnings — repeat those numbers to the user, never your own), and `unbound_series` with a `say_to_user` warning when the project holds a rainfall or flow series that no inflow or rainfall feature uses (the model ignores it: tell the user and ask which input it belongs to). The `computed_status` field (there is NO `status` key on a scenario) is derived from the latest run and will be one of: created, building, built, queued, computing, processing, complete, error, or cancelled — `created` also means no run exists yet. A scenario must be `built` before it can be run. `latest_run_is_valid` is false after any edit since the last build. `unreviewed_inputs` lists the inputs written by an agent key that the user has not reviewed at their current content (kind, layer_id, title, feature_count, the grant that wrote it, `reopened` when an edit undid an earlier input review): while it is not empty, build and run are refused UNREVIEWED_INPUTS. Only the user reviews, in the browser — poll this tool every 30-60 s after giving them the review link. A non-member reading a pub…

Poll a terrain's import until it is `ready` (or `error`), bounded by timeout_seconds. Returns `outcome` — exactly one of `ready`, `error`, `timed_out`, `not_found` (the project has no terrain yet; an unknown terrain_id is an API 404 error instead) — plus the last `status` seen (creating → styling → ready | error) and `phase` (that status, or `not_found`; never empty), `polls`, `elapsed_seconds` and the full `terrain` record (its `gn_layer` is the published elevation dataset pk once ready). An import takes minutes to tens of minutes (the SAME 32 MB 1 m DEM measured 4.5 min once and 26 min once — the worker's S3 download speed dominates), so `timed_out` is normal and NOT a failure: keep calling with the same arguments while `status` is still `creating`/`styling`; several calls in a row is expected. `error` is terminal — the import failed and no default input rows were seeded; upload a corrected GeoTIFF as a new terrain. `ready` is written by the import task; the project's default input rows (Boundary 01, Friction 01, Inflow 01, Rainfall 01, Building 01, MeshRegion 01, one GeoNode layer each) are seeded by the NEXT task in the chain and appear over the following ~30 s. Poll the input-layer list until it is non-empty before attaching your own layer to a default row.

List every input layer row of a project, per kind, with whether it has features. For each of boundary, friction, inflow, rainfall, building, mesh_region and culvert: the rows (`row_id`, `title`, `gn_layer` — the dataset pk — and `dataset_alternate`, the `workspace:name` WFS typename) and `has_features`. `has_features` is the API's own answer for every kind. It is null only when a row's record does not carry it: treat null as unknown, not as empty. A row with no `gn_layer` has no layer yet (`has_features` false). Use the row ids with create_scenario; this is the listing primitive — no need to probe files.

List a project's flood observation layers, or one layer's observations. Without layer_id: every layer (layer_id, title, dataset_alternate, vertical_datum, datum_offset_m, has_features). With layer_id: {layer, count, results} where each observation carries its `fid` (the key for update_observation / delete_observation), subtype + subtype_label, stage/depth, sigma, bound, the observed window in UTC, `datum` status (an `unknown` datum is not scored), `photo_count` and `problems` (fields drawn in the map that fail validation, reported, never fixed).

List ANUGA simulation projects accessible to the authenticated user. Returns a paginated list of projects with their names, projections, and base map references.

List all simulation runs across all scenarios in a project. Returns a paginated list of runs. Optionally filter by status to find active, completed, or failed runs. Each row's presigned `s3_*_url` download links are elided (6-hour capabilities).

List a project's time series (rain gauges, hydrographs, tides) — never their rows. Each series: `id`, `name` (a rainfall polygon binds to its gauge by this exact name), `series_type`, `units`, `timezone`, `row_count`, `first_timestamp` and `last_timestamp`. The data rows themselves are dropped (a project's list is megabytes); fetch one series with the REST API (GET /projects/<id>/time-series/<id>/) if you need its values.

Step 1 of a terrain import: get a presigned S3 PUT URL for a GeoTIFF DEM. The import needs a launch-level agent key: a draft key can presign, but finalize_terrain_upload (and the multipart fallback below) answer a read or draft key 403 AGENT_LEVEL_INSUFFICIENT — check the key before moving bytes. The agent moves the bytes itself — no tool accepts file contents. After this call, PUT the file straight to `upload_url`, sending the SAME Content-Type you passed here (it is part of the signature; a mismatch is a 403 SignatureDoesNotMatch before it is anything else): curl -sS -X PUT -H "Content-Type: image/tiff" --upload-file /path/dem.tif "$UPLOAD_URL" then call finalize_terrain_upload with the returned staging_key and process_id. Nothing exists in Hydrata until finalize; the URL expires after `expires_in` seconds (3600). Platform guidance: keep the terrain extent within 5° across and 40,000 km² (200 × 200 km). The API enforces that cap on its bbox-fetch path; an uploaded GeoTIFF is only rejected above 5 GiB, but a larger DEM will not mesh or run well. The GeoTIFF must carry a CRS; the import reprojects it to the site's UTM zone and builds a hillshade. Fallback when a presigned PUT is impossible: multipart-POST the file with the same agent key to /api/v2/anuga/projects/<id>/terrain/upload/ (form field `file`, optional `title`), which creates the terrain in one step — then get_terrain.

Retry a failed simulation run. Needs a launch-level agent key (a retry rebuilds the package and can start AWS Batch); a read or draft key gets 403 AGENT_LEVEL_INSUFFICIENT. Starts a fresh package build of the run's scenario as a NEW run, which becomes the scenario's latest run. The errored run keeps its status and error as history, and the response describes that old run; the new run's ID is get_scenario's `run_id`. Only valid when status is 'error'. Returns 409 for any other state.

Save a project's default tolerance for comparing runs with flood observations. Needs Editor (or above). Every later get_observation_comparison (and the map's Compare with observations tab) uses it unless a tolerance is passed. Only save it when the user asks to change the project's default; to show the figures at another tolerance, pass `tolerance` to get_observation_comparison.

Exclude a flood observation from ONE run's comparison with observations, or include it again. Needs Editor (or above) on the project. The exclusion is logged with the user and reason (nothing is overwritten) and keyed on (run, layer, fid), so it survives a recompute. The figures change at once: `headline_after` is the run's headline with the change applied. To exclude, the user must give the reason: if they have not, ASK them; do not make one up.

Start a flood simulation run for a built scenario. Needs a launch-level agent key (a run spends compute); a read or draft key gets 403 AGENT_LEVEL_INSUFFICIENT — the user creates a launch key on the site's Agent keys page (/agents/). The scenario must be in 'built' status. Returns 202 with the new run. The run transitions through: built → queued → computing → processing → complete. After starting, poll get_run_status to track progress. Returns 409 if the scenario is not in the correct state. The returned run record has its presigned `s3_*_url` links elided (6-hour capabilities). INPUT REVIEW (Hydrata enforces it): while an input you wrote is not reviewed by the user, the run is not started - `outcome` `unreviewed_inputs`, `error_code` UNREVIEWED_INPUTS, a `ui_url` to the scenario's review screen. A run built from inputs other than the ones reviewed now answers `outcome` `run_invalidated` (RUN_INVALIDATED): build_scenario with rebuild=true, then start it again. SPEND CONSENT (Hydrata enforces it; you cannot): a run whose quote does not fit your agent key's monthly spend limit (default $0), or that Hydrata cannot price, is NOT started. The result is then a normal (non-error) answer with `outcome` `consent_required`, `error_code` CONSENT_REQUIRED, `quoted_cost_usd` and `consent_url`: give the user the link and the price (`say_to_user`). Only they can approve, in their browser. Then do NOT call start_simulation again for this run: poll get_run_status(run_id) — the run leaves `built` once they press the button. (With consent_timeout_seconds > 0 this call polls the run itself and answers `outcome` `consented`, with the run, if they approve within that time.)

Edit one flood observation (PATCH; only the fields given change). The API checks the stored observation merged with `changes` and refuses (400, every bad field named) rather than store a half-valid one. Photos are not fields: attach a link with add_observation_photo_link.