io.github.ZenRows/zenrows-mcp

Read the stored page of one URL a crawl kept, in the crawl's output_format. Works only for crawls created with an output_format, and only for results whose content_status is fetched. Pass crawl_id and content_id: a content id, or content_url from a crawl_results row. Returns the page as stored — raw HTML for output_format "html", Markdown for "markdown" — cut to max_chars (default 20000); a final note says when it was cut. To read a page the crawl did not store, use scrape on its URL instead.

Start a crawl from one URL (Zenrows Crawl API). Crawl follows the links in each page's HTML up to depth hops (no depth limit when depth is omitted), keeps the URLs that match include_patterns / exclude_patterns, and stays on the start URL's registrable domain (subdomains count). Use crawl when you have one start page (a listing, category, blog index) and need the URLs behind it, optionally with each page's HTML. For a single page, use scrape. Each page fetched is billed as one scrape on this account; max_pages bounds the cost (default 100), max_items bounds how many URLs are kept (default 100). Without depth, the crawl keeps following links until max_pages or max_items stops it or nothing is left to open, so set max_pages to what you are willing to spend. The start page itself is never a result, but its fetch is billed: a crawl whose include_patterns / exclude_patterns match nothing still costs that one scrape. A crawl whose pages held nothing new to follow (no link, only links to other sites, or only links the patterns reject) ends completed with no results and no error. When patterns are set, check them against the URLs the start page links to. Set output_format "html" or "markdown" to also store each kept URL's page, then read it with crawl_content. Set webhook to have signed events posted to your https URL when the crawl starts and when it ends; the account needs a webhook signing key, else CRAWL_INVALID_REQUEST (webhook_signing_key_required). Returns crawl_id, status and the crawl (with coverage) at once; the crawl runs asynchronously. Call crawl_wait, then crawl_results. Set idempotency_key to make a retry safe: the same key returns the crawl the first request created instead of starting another. CRAWL_NOT_ENABLED: Crawl is not enabled for this account; do not retry. CRAWL_TOO_MANY_CRAWLS: the account has reached its limit of active jobs (3 by default), shared with its Batch jobs; retry after retry_after seconds.

Save every result of a crawl to one local file, as JSON lines: url, content_status and, once fetched, content (the page). Returns the file's path and line count, not the results, so a crawl of any size costs one small reply. Read the file with your own tools. The file goes to path when given (a relative path inside ~/.zenrows/crawls; an absolute path or one that leaves that folder fails with INVALID_PATH), else ~/.zenrows/crawls/<crawl_id>.jsonl. An existing file is kept and the call fails with FILE_EXISTS unless overwrite is true. On a running crawl the file holds only what was kept so far: download again once crawl_wait says it ended. After crawl_stop, pages in flight can still add lines for up to 10 minutes: the reply says so with partial: true. On the hosted server no file is written: the reply gives the download URL and a command to run where your API key is set.

Get a crawl and one page of the URLs it kept: crawl_id, status (running, completed, stopped, failed), the crawl (with coverage: pages_fetched, pages_failed, items_found; webhook when set), results and next_cursor. completed with stop_reason max_items / max_pages means a limit ended it; stopped with stop_reason user means crawl_stop ended it; stopped with no stop_reason and error.code insufficient_credits or api_key_cap_reached means the account ran out of credits or the API key hit a credit cap (the URLs kept before that stay readable); completed with no results and no error means the crawl's pages held nothing new to follow; when include_patterns / exclude_patterns are set, check them against the URLs the start page links to. failed carries error.code and error.detail: seed_unreachable means the start URL could not be fetched or returned an empty page; no_items_found means the crawl read the start page but found no data it could use, so it had nothing to follow; start from a listing or product page. A failed crawl, and a stopped one with an error, answer ok: false: error.code is CRAWL_INSUFFICIENT_CREDITS or CRAWL_KEY_CAP_REACHED (the codes crawl_create gets on a 402) when credits ended it, failed or stopped, else CRAWL_FAILED; error.server_code is the crawl's error.code. Pass next_cursor as cursor for the next page; while the crawl runs, next_cursor is never null. To read many pages at once, use crawl_results.

List this account's crawls, newest first, with status and coverage (no results). Pass next_cursor as cursor for the next page; next_cursor is absent on the last page.

List the URLs a crawl kept, in the order it kept them, following the API's pages up to limit. Each result has url and, when the crawl has output_format, content_status (pending, fetched, failed) and, once fetched, content_url: pass it to crawl_content as content. While the crawl runs the list is partial (partial=true): call again later with the returned next_cursor to get only the URLs kept since. A stopped crawl can still add URLs from pages fetched before the stop for up to 10 minutes, so next_cursor stays set during that window: call again later. next_cursor is null once the crawl ended and every URL was read.

Stop a running crawl. No page still waiting is fetched or billed; pages already in flight finish. The URLs kept so far stay readable with crawl_results. A stopped crawl cannot resume. Idempotent: a crawl that already ended answers with its final status. A 429, 502, 503 or 504 is retried up to 3 times, waiting at most 50 seconds in all; a 503 crawl_busy means the stop was not saved yet, and if it persists, CRAWL_BUSY says when to call again. Pages in flight still finish, so coverage and results can keep growing for up to 10 minutes after the stop; read them with crawl_get and crawl_results.

Poll a crawl until it ends (completed, stopped, or failed) and return its status, its counts and the first 20 URLs (first_urls). For every result, call crawl_download (one file) or crawl_results (pages). A crawl that ends completed with no results and no error finished; it is not a failure (see crawl_get). If the wait runs out first, it returns the crawl with status running and a note: call crawl_wait again. A call that carries a progressToken gets a progress notification after each poll, with the status and counts.

Extract structured data from a webpage via Zenrows. Returns JSON fields (products, articles, listings) in "data" rather than a full page body. Set include_html to also get the page HTML. Modes: - auto (default): extract=auto, site-tailored Extract (currently free, billing may apply later). Works on any prepared domain, with any API key. Error REQS007 says why in "reason": - domain_not_prepared: call prepare_domain with the domain and 1 to 5 example URLs (one works; we recommend 3 or more of the same page type; a few minutes, no credits), poll prepared_domain_status about every 30 seconds until it says ready, then call extract again. - page_not_recognised: the domain is prepared, but this page is not one Extract extracts (a page with nothing to extract, such as a home page). Try a product, article or listing page instead; if this page should work, retry it once in a few minutes; otherwise use mode=css. Do not prepare the domain again. - domain_not_enabled: preparing does not help; use mode=css. - autoparse: deprecated general-purpose structured JSON on any domain; use only when asked for - css: css_extractor with an explicit selector map Requests use automatic mode (mode=auto) by default: Zenrows enables JS rendering and premium proxies only when the site needs them, and charges only for the configuration that succeeds. Pass just the URL. Set js_render or premium_proxy only to force a fixed configuration; that turns automatic mode off and bills every request at that configuration's cost. Check "empty" in the result: true means no field came back with a value. "noDataExtracted": true means Extract processed the page and found nothing to extract (not charged); retrying the same request returns the same answer. Full-page markdown, HTML and screenshots come from scrape.

Prepare a domain so extract (mode=auto) works on it. Use this when extract returns error REQS007 with reason domain_not_prepared. Send the domain (the host, e.g. shop.example.com; a leading www. is dropped, and each host is prepared separately) and 1 to 5 example URLs of real pages you want extracted. One works; we recommend 3 or more of the same page type, so pass 3 or more when you can, up to 5 (e.g. 3 product pages: https://www.example.com/products/a, /products/b and /products/c), because Extract compares pages of one type to check the fields it found and fill in fields one page is missing. Several page types can share the 5 (e.g. 3 product pages + 2 search-result pages). It returns at once with status "preparing" and never waits; preparation takes a few minutes and costs no credits. Then call prepared_domain_status about every 30 seconds until status is "ready", and call extract again. Each new domain uses one slot of the account's monthly allowance (prepared_domain_status shows it). A domain already being prepared returns its current status; a domain already prepared returns "ready" at once with already_prepared: true and uses no slot. Do not call this for REQS007 with reason page_not_recognised: the domain is already prepared and preparing again does not change that answer.

Check a domain submitted with prepare_domain, and the account's monthly allowance. Returns status (preparing, ready or failed), state (the current step), roles (one per page kind), a failure code and detail when it failed, the allowance (used, limit, remaining) and "next": what to do now. Poll about every 30 seconds; preparation takes a few minutes. Call extract again once status is "ready". Costs no credits.

Scrape any webpage and return its content using Zenrows (Fetch). Returns full-page content (markdown/HTML/PDF/screenshot). For structured JSON fields (products, articles, listings), the extract tool returns parsed fields instead of a full page body. By default the request uses automatic mode (mode=auto): Zenrows adds JS rendering or premium proxies per page only when a plain request does not return the page, and charges only for the configuration that succeeds. Pass just the URL. Set js_render or premium_proxy only to force a fixed configuration. Doing so switches off automatic mode, and every request is billed at that configuration's cost (premium_proxy with js_render is 25x a basic request). Examples: Default: { url: "https://example.com" } Geo: { url: "https://example.com", proxy_country: "US" } Forced: { url: "https://spa.com", js_render: true }