hardly

Authentication evidence; sections: quick (default; auth paths, token responses, header names), patterns, credentials, secret_names, cookies. Names, shapes and entry ids only, never values. Example: hardly_auth_report(session_id='S', sections=['credentials','cookies']). quick: a first auth check. patterns: named schemes (bearer+refresh, OIDC+PKCE, SAML POST, double-submit CSRF, signed requests; ``pattern_kinds`` filters). credentials: password fields, session cookies and flags, anti-forgery names, OAuth params, token shapes and a hypothesised login_flow (use before implementing login in a client). secret_names: sensitive header/field/ query NAMES. cookies: per-server timeline of Set-Cookie / Cookie names.

LIVE (confirm-gated): unattended time-boxed browser capture of a URL; analyze=true also discovers its APIs and returns session_id plus a brief. Without confirm=true it returns only the plan. Example: hardly_browser_capture_discover(url='https://example.com', wait_seconds=8, analyze=true, confirm=true). Refuses loopback, private, link-local and metadata start URLs (host_not_allowed; HARDLY_ALLOW_PRIVATE_HOSTS=1 for a local test site). analyze=false: load the URL, wait wait_seconds (default 20), stop; returns the raw HAR (har_output_path) and, with open_session=true, a session. analyze=true (headless only): optional ``steps`` (goto/wait/aria/click/fill/find_click...) then stop, open a session and return a brief (default wait 5). If the brief shows a wall or captcha, STOP and switch to interactive hardly_browser_start(headed=true, channel='chrome') with a person. budget_seconds is a hard per-call budget (env HARDLY_CAPTURE_BUDGET). exclude_noise=true aborts analytics/ad/font/ map-tile/heavy-media requests while loading (default off: blocking can break sites).

Read the running browser tab without navigating; sections: url (default), elements (css/xpath locators), aria (accessibility YAML with [ref=eN] handles). Example: hardly_browser_inspect(sections=['aria'], selector='main'). ``url``: current URL and title (cheap check of where a click landed). ``elements``: visible links/buttons/inputs with locators; ``query`` is a CSS selector override, ``limit`` max rows (default 40, max 100). ``aria``: rendered ARIA tree, ``mode='ai'`` (default) asks for refs, ``selector`` scopes it (default body); the refs feed hardly_browser_interact. For offline HAR HTML bodies use hardly_entry_outline.

Drive the running browser tab: action=goto (needs url), click, fill (needs value) or press (key, default Enter). Prefer ref from hardly_browser_inspect(sections=['aria']). Example: hardly_browser_interact(action='click', ref='e12'). Target one element per call with ``ref``, else ``css`` / ``xpath`` (from hardly_browser_inspect(sections=['elements'])), or ``text`` / ``role`` with ``name`` (click). Never type real credentials yourself: ask the person to use an interactive capture. Omit capture_id for the latest running capture.

Run a scripted list of steps on the running browser tab in one call (cheaper than many single calls). Use once you know the selectors; hardly_session_plan_steps suggests steps from a session. Example: hardly_browser_run_steps(steps=[{'op':'goto','url':'https://example.com'}]). Each step: ``{"op":"goto"|"wait"|"elements"|"click"|"fill"|"press"|"url"|"aria", ...}``. Example steps:: [ {"op":"goto","url":"https://example.com/search"}, {"op":"wait","ms":1000}, {"op":"click","text":"I Accept"}, {"op":"fill","css":"#SearchOnName","value":"EXAMPLE"}, {"op":"click","css":"input[type=submit]"}, {"op":"wait","ms":2000}, {"op":"elements","limit":30} ]

Launch a real browser that records a HAR (needs the capture extra); returns capture_id. headed=true (default) is interactive: ASK THE PERSON to use the window, then call hardly_browser_stop. Example: hardly_browser_start(url='https://example.com', headed=true, channel='chrome'). Refuses loopback, private, link-local and metadata start URLs (host_not_allowed; HARDLY_ALLOW_PRIVATE_HOSTS=1 for a local test site). headed=false is headless: drive it with hardly_browser_inspect / hardly_browser_interact / hardly_browser_run_steps. For a one-shot unattended load prefer hardly_browser_capture_discover. Requires ``pip install -e ".[capture]"`` + ``playwright install chromium`` (see hardly_server_status). Prefer channel='chrome' for bot walls. url_filter is a Playwright glob. profile keeps cookies. Without har_output_path the HAR is ephemeral (memory only). same_tab (default true) forces target=_blank into the current tab. trace=true writes a ``.trace.zip`` beside the HAR. Waits for a capture slot (HARDLY_CAPTURE_SLOTS); the result carries ``slot``.

Stop a running browser capture, flush the HAR and optionally open it as a session. Call when the person says they are done (interactive) or your steps are complete; then continue with hardly_session_site_brief(session_id). Example: hardly_browser_stop(open_session=true). An ephemeral capture (started without har_output_path) lives only in the returned session; save it with hardly_write_session_copy(format='index'). Omit capture_id for the latest running one.

List browser captures (running and persisted) with ids and states, or the status of one capture_id. Poll it while a person drives an interactive capture. Example: hardly_capture_list(capture_id='c1'). Use to find a capture_id after a restart. With capture_id the list holds exactly that capture (state, har_output_path, request counts).

List targets from a content-neutral catalog file (JSON/YAML), filtered; no network. detail='summary' returns counts instead of rows. Tags and roles are defined by your project (docs/catalog.md). Example: hardly_catalog_list(catalog_path='/data/targets.json', detail='summary'). One row per endpoint: target, tags, groups, role, kind, status, redacted url, gate classes, stack names. Filters: ``tag`` (comma-separated, all required), ``group`` (object of key->value), ``role``, ``status``, ``target_id``. hardly assigns no meaning to tags, group keys or roles.

Build a minimal Python (urllib) client sketch from story steps or chosen entry_ids and return it as text; secrets become PLACEHOLDER_* values. Writes nothing (hardly_write_export format=client_python saves it). Example: hardly_client_build(session_id='S', entry_ids=[12, 14]). Use as the last step of SDK work, after hardly_session_story / hardly_session_site_brief.

List ArcGIS REST endpoints (MapServer/FeatureServer/etc.) seen in the capture: layer ids, parameter names, paging evidence. Sends nothing; the live twin is hardly_send_arcgis_explore. Example: hardly_endpoint_arcgis(session_id='S').

Detect GraphQL operations (operationName, query, mutation) behind shared URLs, with evidence entry ids. Use when POSTs share one URL; then hardly_entry_get for a sample. Example: hardly_endpoint_graphql(session_id='S').

List grouped endpoints (METHOD + path template) with counts, paged by limit/offset. The main API surface view for JSON APIs; for server-rendered pages use hardly_session_site_brief or hardly_page_forms. Example: hardly_endpoint_list(session_id='S', host='api.example.com').

Recognise cursor, next-link and Link-header pagination, with evidence entry ids. Shapes only. Use before writing a client loop over results. Example: hardly_endpoint_pagination(session_id='S', host='api.example.com').

Request/response shapes of one endpoint template. Sections: schema (default; JSON field names and types), param_roles (each field static, dynamic or sensitive). Example: hardly_endpoint_schema(session_id='S', method='GET', path_template='/api/items/{id}'). Use after hardly_endpoint_list. param_roles decides which parameters a client must compute versus hard-code. host defaults to the main host. Names and types only.

Summarise non-JSON stream/binary formats: gRPC(-web), protobuf, MessagePack, CSV/TSV, SSE, WebSocket frames, with evidence entry ids. Shapes only. Use when endpoints show odd content types. Example: hardly_endpoint_streams(session_id='S').

Show the requests just before and after one entry in TIME (a click and the XHRs that followed). Use with a navigation or grid-click entry_id to find the API it triggered; inspect rows with positive delta_ms. Example: hardly_entry_around(session_id='S', entry_id=30, after=8). What triggered an entry by the browser's own initiator data is hardly_entry_initiators.

Search inside one large body without loading it: JSONPath-lite ($.a[*].b, ..key) or regex, paged (pass next_offset while has_more). Use instead of hardly_entry_get for big payloads; sensitive values come back as shapes. Example: hardly_entry_body_query(session_id='S', entry_id=12, jsonpath='$.items[*].id').

Build a curl command for one entry and return it as text (not executed); secrets are redacted by default. Use to show a person a single request; for a full client use hardly_client_build. Example: hardly_entry_build_curl(session_id='S', entry_id=12).

Diff headers and JSON body keys between two entries of one session. Use to see what changed between a failing and a working request, or pre/post login. Example: hardly_entry_compare(session_id='S', entry_id=3, other_entry_id=9).

Trace what one request depends on: the earlier response behind each header, cookie, hidden field or value. Returns ordered minimal steps plus inputs the caller must supply. Offline; names and ids only. Example: hardly_entry_dependencies(session_id='S', entry_id=20). Use before writing a client for a multi-step flow; to track one value across the capture use hardly_session_trace_value.

Show one entry with redacted headers and truncated bodies (max_body_chars, default 4000). Use after entry_search/endpoint_list gives an entry_id; for large bodies use hardly_entry_body_query instead of raising max_body_chars. Example: hardly_entry_get(session_id='S', entry_id=12).

Show the initiator parent and children of an entry (HAR _initiator/pageref): what triggered a request and what it triggered. Use to find who called an API; for time order use hardly_entry_around. Example: hardly_entry_initiators(session_id='S', entry_id=30).

Offline outline of an HTML/XML body from a HAR entry. format: markdown (headings/tables/forms/links), tree (tag tree) or aria (YAML); omit for all three. Example: hardly_entry_outline(session_id='S', entry_id=5, format='markdown'). Use instead of reading raw HTML; for the live tab use hardly_browser_inspect(sections=['aria']).

Find entries by host, path, method, status, body text, header or content_kind; returns small paged rows with entry_ids. Use to locate a specific request, then hardly_entry_get. Example: hardly_entry_search(session_id='S', body_contains='token', limit=10). ``content_kind`` uses the classifier: json, jsonl, jsonp, csv, html, html_table (or table), pdf, image, css, javascript, ... Result rows include ``content_kind`` / hints. exclude_noise=false also returns OPTIONS preflights and tracker traffic.

Detect gates in a capture: bot walls, captcha, login, rate limits, 401/403/429 challenges and CDN/WAF bot-protection products. Never evades. Example: hardly_gate_bot_protection(session_id='S', explain=true). Call when requests are blocked, before any retry: every gate is a STOP sign (re-capture interactively with a person). barriers: each barrier (environment_blocked, bot_wall, captcha, proof_of_work, waiting_room, click_through_terms, login, paywall, rate_limit) with evidence names, entry_ids and a policy action (stop | accept_click_through | unknown_rerun). http_challenges: WWW-Authenticate schemes, 429/423/Retry-After lockout signals and captcha widget markup. bot_protection: vendors seen (Cloudflare, Akamai, Imperva, DataDome, HUMAN, ...) with a state each (blocked / challenged / clearance_seen / present). Challenge nonces are never returned; a CDN header on a normal page is informational, not a wall.

Categorized catalog of tools and workflows; pass topic to filter. Use when you do not know which tool fits (topics: portal, tokens, capture, modes, or a tool name); for a task-specific plan use hardly_guide_task_plan. Example: hardly_guide_help(topic='capture').

List the three operating modes (archive, headless, interactive), or get the playbook for one mode, or auto-pick a mode from goal / har_path / url. Example: hardly_guide_mode(har_path='/data/capture.har'). With no argument it lists the modes and when to use each. With ``mode`` it returns that mode's step-by-step playbook; with goal / har_path / url (and no mode) a mode is chosen for you. ``hardly_guide_task_plan`` is the better first call for a full ordered plan.

FIRST CALL for a new task: ordered plan of hardly tool calls with example arguments, environment state and recommended tools for the goal. Example: hardly_guide_task_plan(goal='reverse engineer the login flow', har_path='/data/capture.har'). Pass what you know: goal (free text, e.g. 'build a client SDK'), har_path, url. Small output (~2 KB). For one mode's playbook use hardly_guide_mode, for the tool catalog hardly_guide_help.

Diagnose a HAR file on disk (truncated/omitted bodies, sanitised cookies, clock skew, noise); no session needed. Findings carry a fix_hint such as recapture flags. Example: hardly_har_file_check(har_path='/data/capture.har'). Session-level body counts are hardly_session_body_coverage. Optional ``config`` (object): checks, exclude, severity_overrides, thresholds{max_entry_bytes, truncation_ratio, clock_skew_s, noise_ratio, min_entries}, strict, fail_on, host, ignore_hosts, ignore_paths, max_findings, fix.

Mine endpoints the page embeds. Sections: script_routes (URL path literals in JavaScript), data_attrs (endpoints and config in HTML data-* attributes); default both. Example: hardly_page_embedded_routes(session_id='S', sections=['data_attrs'], entry_id=5). Use when search works but the detail URL is unknown, or on JS-heavy pages to find where the page gets its data; then hardly_entry_around on the click entry. data_attrs reports dataset keys, value kinds, embedded endpoint URLs and JSON config, never free text.

Extract HTML forms and their inputs with signals (hidden fields, tokens, links, handlers). Use to reverse server-rendered pages instead of dumping HTML; entry_id gives the full inventory of one response, omit it to scan the session. Example: hardly_page_forms(session_id='S', entry_id=5). For link/handler/label-first workflows and search links use hardly_page_ui (same scan).

List HAR pageref groups (browser page loads) with document hints. Use to split a capture into navigations. Example: hardly_page_list(session_id='S').

List HTML data tables: headers, row/column counts, column kinds and a masked first row (cell values never returned). Use to understand result grids before scraping or modelling them. Example: hardly_page_tables(session_id='S', entry_id=14).

Inventory UI controls. Sections: links, handlers (onclick/onsubmit), labels (default all three), search_links (ranked links likely to lead to a search page). Example: hardly_page_ui(session_id='S', sections=['search_links'], keywords=['search']). Same scan as hardly_page_forms, projected to the UI parts; handler_functions names feed hardly_page_embedded_routes. entry_id inventories one response, omit it to scan the session. search_links takes ``keywords`` (the site's domain vocabulary): repeat per hop until a form with inputs appears; its next_step feeds hardly_browser_run_steps.

LIVE (confirm-gated): politely explore an ArcGIS REST service or layer URL (GET only, at most 7 requests). Without confirm=true it returns only the plan, so show the person the URL first. For captured traffic use hardly_endpoint_arcgis. Example: hardly_send_arcgis_explore(url='https://host/arcgis/rest/services/X/MapServer', confirm=true). Refuses loopback, private, link-local and metadata hosts, also on redirects (host_not_allowed; local test targets: HARDLY_ALLOW_PRIVATE_HOSTS=1). Returns layers, fields (personal-data-like and id fields FLAGGED), query templates and a one-row sample as field names + masked shapes, never values. Stops on 429 and on token-required (498/499); never guesses tokens.

LIVE (confirm-gated): politely verify catalog endpoints (verified|blocked|dead|needs_browser). Without confirm=true it returns only the plan and sends nothing; write_back=true also records statuses in catalog_path. Example: hardly_send_catalog_verify(catalog_path='/data/targets.json', confirm=true, write_back=true). Refuses loopback, private, link-local and metadata hosts, also on redirects (host_not_allowed; local test targets: HARDLY_ALLOW_PRIVATE_HOSTS=1). Per endpoint it runs one robots-aware, honest-UA fetch and records status, gate classes and stack names, never bodies or tokens. Per-host delay, request budget, resumable (already-checked endpoints are skipped unless force / recheck_after_seconds). Stops a host on any gate per docs/gate-policy.md; never evades.

LIVE (confirm-gated): replay ONE captured request. Without confirm=true it returns only the plan and sends nothing, so tell the person what will be sent first. Sensitive HAR headers are skipped unless given in header_overrides. Example: hardly_send_entry(session_id='S', entry_id=12, confirm=true). Refuses loopback, private, link-local and metadata hosts, also on redirects (host_not_allowed; local test targets: HARDLY_ALLOW_PRIVATE_HOSTS=1). For an ordered series use hardly_send_entry_series; to learn which elements are required hardly_send_entry_ablation. The offline curl text is hardly_entry_build_curl.

LIVE (confirm-gated): find which headers, cookies, params, body fields and prior steps a request truly needs by replaying it with ONE element removed at a time. Without confirm=true it returns only the plan. Example: hardly_send_entry_ablation(session_id='S', entry_ids=[12], confirm=true). Refuses loopback, private, link-local and metadata hosts, also on redirects (host_not_allowed; local test targets: HARDLY_ALLOW_PRIVATE_HOSTS=1). Reports REQUIRED vs OPTIONAL names only, never bodies. entry_ids may be an ordered flow (earlier ids are prior steps, the last is the target). Secrets only via ``overrides`` {"headers":{},"cookies":{},"query":{},"body":{}}; missing ones are listed under needs_override. GET/HEAD only unless allow_unsafe=true. Hard stop on 429 / Retry-After / gate stop; captcha token fields are never sent. Budget-skipped items appear under not_tested.

LIVE (confirm-gated): replay an ordered series and report the first step whose status, content-type or body shape diverges. Without confirm=true it returns the dry-run plan and missing inputs. Example: hardly_send_entry_series(session_id='S', entry_id=20, confirm=true). Refuses loopback, private, link-local and metadata hosts, also on redirects (host_not_allowed; local test targets: HARDLY_ALLOW_PRIVATE_HOSTS=1). Use to verify a client flow works. Secrets only via ``env`` {name: value} or HARDLY_INPUT_<NAME> env vars; values are never printed. GET/HEAD only unless allow_unsafe; halts on 429/Retry-After/ gates.

LIVE (confirm-gated): explain a redirect loop (ERR_TOO_MANY_REDIRECTS) by following the chain by hand with and without cookies. Without confirm=true it returns only the plan. Reports statuses, redacted URLs and cookie names only. Example: hardly_send_redirect_walk(url='https://example.com', confirm=true). Refuses loopback, private, link-local and metadata hosts, also on redirects (host_not_allowed; local test targets: HARDLY_ALLOW_PRIVATE_HOSTS=1). Reports the loop shape: www<->apex or http<->https flips, trailing-slash fights, growing return URLs, cookie-dependent redirects, and whether the alternate host resolves. Redirects already in a capture are hardly_session_redirect_history.

LIVE (confirm-gated): curl-first, robots-aware, polite crawl that finds candidate pages (search forms first). Without confirm=true it returns only the plan. Use when headless Chromium is blocked or static HTML suffices; it stops at gates and never evades them. Example: hardly_send_site_crawl(url='https://example.com', keywords=['search'], confirm=true). Refuses loopback, private, link-local and metadata hosts, also on redirects (host_not_allowed; local test targets: HARDLY_ALLOW_PRIVATE_HOSTS=1). Follows only links found in fetched HTML (never guesses hosts or paths), ranks them with ``keywords`` (nouns), honours robots.txt, waits delay_seconds between requests per host, strips session ids, and stops at gates and on 429/Retry-After. Caps: max_pages <= 40, depth <= 4. External registrable domains are only recorded unless follow_external=true (one hop). Returns candidates, per-page form field NAMES, gate classes and needs_browser pages (with `next` advice when explain=true); no bodies, URLs redacted. For needs_browser pages use hardly_browser_capture_discover with a find_click step.

Server version, feature flags, tool names and browser availability; sections: capabilities (default), browser_setup. Call when a tool seems missing (restart the MCP server) or capture fails. ``capabilities``: version, features, tools, capture_available (small, ~3 KB). ``browser_setup``: Playwright package and browser binaries, with the exact install commands. Example: hardly_server_status(sections=['browser_setup']).

Count entries with usable body text versus empty or truncated. Use before trusting schema/client output, or when bodies look missing. Example: hardly_session_body_coverage(session_id='S'). Useful after Playwright captures that used to omit XHR bodies (size=-1). For problems in the HAR file itself use hardly_har_file_check.

Close an open session to free memory (the session is discarded; files saved with hardly_write_session_copy stay). Example: hardly_session_close(session_id='S').

Compare endpoint templates (and credential maps) between two sessions: routes that appeared and disappeared. Use for logged out vs in, or before and after a site change. Example: hardly_session_compare(session_id='A', other_session_id='B').

Find repeated method+path_template groups (polling, retries). Use to separate noise from real calls before documenting. Example: hardly_session_duplicates(session_id='S', min_count=5).

List capture-quality issues: empty bodies, 4xx/5xx, redirects without Location. Check before trusting other results; for HAR-file problems use hardly_har_file_check. Example: hardly_session_issues(session_id='S').

List open HAR sessions (session_id, har_path, index_path). Use when you lost a session_id. No arguments. Example: hardly_session_list().

Index a HAR file (or open a saved index) into a queryable session and return session_id plus summary counts. Start here for any HAR; never Read the raw HAR. Writes nothing (save an index with hardly_write_session_copy). Example: hardly_session_open(har_path='/data/capture.har'). Opening the same path again returns the same session; force=true re-ingests the HAR.