Crawlberg
OCI · GHCR.IO/XBERG-IO/CRAWLBERG
Scrape, crawl, and map websites to Markdown or JSON via local CLI.
Available components
Last scanned (1 hour ago)
Scores higher than 70% of OCI servers
Category Web Scraping
How this component scores in each security and reliability category. Every signal is checked automatically from public evidence about the published package, including repeated runs of it in an isolated sandbox, and we only credit what we can confirm. How we score → Why this is hard to score →
Supply Chain Security0
- No malware scan is available for this kind of package: the supply-chain vendors we use do not cover it. This is a permanent gap in our coverage, not a finding about the package.Unverified
- Known CVEs could not be checked: this artifact ships no SBOM or dependency manifest, so there is no dependency list to read.Unverified
- Install-script risk not yet assessed.Unverified
- Dependency health could not be checked: this artifact ships no SBOM or dependency manifest, so there is no dependency list to read.Unverified
Provenance & Transparency48
- Source repository is publicly reachable at the declared URL. View diagnostics → Pass
- Provenance check failed: no build-provenance attestation is published. See how to fix → View diagnostics → Fail
- Clear OSI-approved license (MIT).Pass
- Actively maintained (last published 1 days ago).Pass
- Publishes a security disclosure policy (SECURITY.md).Pass
Schema Quality & AI Usability76
- AI-judged instruction clarity (good).Pass
- Tool/resource definitions use about 730 tokens (~81/item across 9 items; 9 tools + 0 resources), lean.Pass
- Usage-examples check failed: none of the tools include examples. See how to fix → Fail
Stability & Change Management47
- Stability observed for 14 of 30 days with no destabilising changes; credit accrues until the full window elapses.Partial
Tool Coverage100
- 100% of tools have a non-trivial description (not blank, and not just the tool's name).Pass
- 100% of tool parameters carry a description.Pass
- Structured output schemas are declared (100% of tools); any adoption earns full credit.Pass
Tool Safety100
- No prompt-injection markers were found in the server instructions, tool names or descriptions we captured.Pass
- All 1 tool(s) whose name or description implies an irreversible operation declare an MCP destructiveHint annotation.Pass
- An AI judge read all 10 captured unit(s) of tool text and found none that tries to manipulate the model reading it.Pass
Capabilities100
- Implements a current MCP spec version (2026-07-28).Pass
Unverified: 1 category
A category scored 0 because we could not verify it: a data source with nothing on this package, evidence we could not reach, or a check we could not run. We only credit what we can confirm.
Every change we have recorded for this component, newest first. Security-relevant changes are always shown. ▲ marks a change for the better, ▼ a change for the worse; unmarked changes are neutral.
- 17 Oct 26 +1
No change was recorded against any check on this day. Stability & Change Management went from 43 to 47. That category is still filling its 30-day observation window: 13 days of observed history at the previous scan, 14 at this one. The score rises as the window fills, whether or not the server changes.
- 15 Oct 26 +1
- Stability: 0.37 → unverified ▼ security
- Tool safety: pass → unverified ▼ security
- Capabilities: pass → unverified ▼ functional
- Tool coverage: 100 → unverified ▼ functional
- Package version: 1.10.2 → 1.11.0 functional
- 14 Oct 26 +36
- Injection markers: unverified → pass ▲ security
- Tool coverage: unverified → 100 ▲ functional
- Stability: unverified → 0.37 ▲ functional
- MCP protocol: unverified → pass ▲ functional
- Package version: 1.9.0 → 1.10.2 functional
- 12 Oct 26 0
- Package version: 1.9.0 → 1.10.2 functional
- 7 Oct 26 0
- Package version: 1.9.0 → 1.10.2 functional
- 6 Oct 26 −31
- Tool safety: pass → unverified ▼ security
- Stability: Stability not yet verified: we do not have a sandbox capture of the MCP schema this version of the package serves yet. security
- Capabilities: pass → unverified ▼ functional
- Tool coverage: 100 → unverified ▼ functional
- First check of Schema quality: unverified functional
- Package version: 1.9.0 → 1.10.1 functional
- 3 Oct 26 41
First indexed and scored.
Diagnostic detail from the automated scan of this channel: what the scanner observed at each step, so you can see exactly where a check passed or failed. It is informational only and never changes the trust score.
Captured 17 Oct 2026 · Analysed oci/ghcr.io/xberg-io/crawlberg@1.11.0
Provenance No attestation
The registry publishes no build provenance for this version, so there is nothing to verify.
| Result | No attestation |
|---|---|
| Ecosystem | oci |
| Reason | No attestation published |
Background: How many MCP packages publish verified provenance →
The tools this component advertises to a client, with an estimated token cost for each. Expand a tool to see its parameters and schema. The per-tool counts are indicative and are not scored directly; the schema's total context footprint is one signal in Schema Quality & AI Usability. A tool's description is untrusted text the model reads on every call, which is what makes this list a security surface and not just an inventory: how tool poisoning works →
batch_crawl ~119
Crawl multiple seed URLs concurrently. Returns crawl results for all seeds.
| Name | Type | Req | Description |
|---|---|---|---|
| concurrency | integer|null | – | Maximum number of concurrent requests |
| format | string|null | – | Output format: "markdown" (default) or "json" |
| max_depth | integer|null | – | Maximum link depth from each seed URL |
| max_pages | integer|null | – | Maximum number of pages to crawl per seed |
| stay_on_domain | boolean|null | – | Whether to restrict crawling to the same domain |
| urls | array | yes | List of seed URLs to crawl |
Structured output declared, but exposes no named fields.
No examples provided.
batch_scrape ~66
Scrape multiple URLs concurrently. Returns results for all URLs.
| Name | Type | Req | Description |
|---|---|---|---|
| concurrency | integer|null | – | Maximum number of concurrent requests |
| format | string|null | – | Output format: "markdown" (default) or "json" |
| urls | array | yes | List of URLs to scrape |
Structured output declared, but exposes no named fields.
No examples provided.
crawl ~128
Crawl a website following links. Returns content for all discovered pages up to max_depth/max_pages.
| Name | Type | Req | Description |
|---|---|---|---|
| format | string|null | – | Output format: "markdown" (default) or "json" |
| max_depth | integer|null | – | Maximum link depth from the start URL |
| max_pages | integer|null | – | Maximum number of pages to crawl |
| respect_robots_txt | boolean|null | – | Whether to respect robots.txt directives and nofollow hints |
| stay_on_domain | boolean|null | – | Whether to restrict crawling to the same domain |
| url | string | yes | Starting URL for the crawl |
| Name | Type | Req | Description |
|---|---|---|---|
| browser_used | boolean | yes | Whether the browser fallback was used for any page in this crawl. |
| cookies | array | yes | Cookies collected during the crawl. |
| error | string|null | – | An error message, if the crawl encountered an issue. |
| final_url | string | yes | The final URL after following redirects. |
| pages | array | yes | The list of crawled pages. |
| redirect_count | integer | yes | The number of redirects followed. |
| stayed_on_domain | boolean | yes | Whether all crawled pages stayed on the same domain as the start URL. |
| was_skipped | boolean | yes | Whether any page was skipped during crawling. |
No examples provided.
download ~45
Download a document from a URL. Returns metadata about the downloaded file.
| Name | Type | Req | Description |
|---|---|---|---|
| max_size | integer|null | – | Maximum document size in bytes |
| url | string | yes | URL to download |
| Name | Type | Req | Description |
|---|---|---|---|
| body_size | integer|null | – | Response body size when the URL returned HTML rather than a document. |
| content_hash | string|null | – | SHA-256 hex digest of the downloaded document. |
| content_type | string|null | – | Content-Type when the URL returned HTML rather than a document. |
| filename | string|null | – | Filename of the downloaded document. |
| mime_type | string|null | – | MIME type of the downloaded document. |
| note | string|null | – | Explanatory note when the URL returned HTML rather than a document. |
| size | integer|null | – | Size of the downloaded document in bytes. |
| status_code | integer|null | – | HTTP status code when the URL returned HTML rather than a document. |
| url | string | yes | The URL that was fetched. |
No examples provided.
generate_citations ~38
Convert markdown links into numbered citations with an appended reference list.
| Name | Type | Req | Description |
|---|---|---|---|
| markdown | string | yes | Markdown text whose inline links are converted to numbered citations |
| Name | Type | Req | Description |
|---|---|---|---|
| content | string | yes | Markdown with links replaced by numbered citations. |
| references | array | yes | Numbered reference list: (index, url, text). |
No examples provided.
get_version ~16
Get the current crawlberg library version.
Input schema present but exposes no named parameters.
| Name | Type | Req | Description |
|---|---|---|---|
| version | string | yes | The crawlberg library version. |
No examples provided.
interact ~57
Execute browser actions on a page. Actions may mutate page or application state.
| Name | Type | Req | Description |
|---|---|---|---|
| actions | array | yes | Sequence of browser actions to execute (click, type, scroll, etc.) |
| url | string | yes | URL to navigate to before executing actions |
| Name | Type | Req | Description |
|---|---|---|---|
| action_results | array | yes | Results from each executed action. |
| final_html | string | yes | Final page HTML after all actions completed. |
| final_url | string | yes | Final page URL (may have changed due to navigation). |
| screenshot_base64 | string|null | – | Base64-encoded PNG screenshot taken after all actions. Populated only when a `PageAction::Screenshot` action actually ran, so callers that never request a screenshot do not pay the encoding cost. |
| ssrf_refused_urls | array | – | URLs of the requests the page sent during the session that the SSRF policy refused, including during the extra wait, without their credentials, each listed once. |
No examples provided.
map ~87
Discover all pages on a website via links and sitemaps. Returns a list of discovered URLs.
| Name | Type | Req | Description |
|---|---|---|---|
| limit | integer|null | – | Maximum number of URLs to return |
| respect_robots_txt | boolean|null | – | Whether to respect robots.txt directives |
| search | string|null | – | Case-insensitive substring filter for discovered URLs |
| url | string | yes | URL of the website to map |
| Name | Type | Req | Description |
|---|---|---|---|
| urls | array | yes | The list of discovered URLs. |
No examples provided.
scrape ~97
Scrape a single URL and extract content as markdown or JSON. Returns page content, metadata, links, and images.
| Name | Type | Req | Description |
|---|---|---|---|
| format | string|null | – | Output format: "markdown" (default) or "json" |
| only_main_content | boolean|null | – | Whether to extract only the main content of the page |
| url | string | yes | URL to scrape |
| use_browser | boolean|null | – | Force browser rendering instead of HTTP fetch |
| Name | Type | Req | Description |
|---|---|---|---|
| assets | array | yes | Downloaded assets from the page. |
| auth_header_sent | boolean | yes | Whether an authentication header was sent with the request. |
| body_size | integer | yes | The size of the response body in bytes. |
| browser | – | – | Browser-specific extras (eval result, network events, cookies). Only populated when `BrowserBackend::Native` was used for this request. |
| browser_used | boolean | yes | Whether the browser fallback was used to fetch this page. |
| content_type | string | yes | The Content-Type header value. |
| crawl_delay | integer|null | – | The crawl delay from robots.txt, in seconds. |
| detected_charset | string|null | – | The detected character set encoding. |
| downloaded_document | – | – | Downloaded non-HTML document (PDF, DOCX, image, code, etc.). |
| extracted_data | – | – | Structured data extracted by LLM. Populated when extraction is configured. |
| extraction_meta | – | – | Metadata about the LLM extraction pass (cost, tokens, model). |
| feeds | array | yes | Feed links found on the page. |
| final_url | string | yes | The final URL after following all redirects. |
| html | string | yes | The HTML body of the response. |
| images | array | yes | Images found on the page. |
| is_allowed | boolean | yes | Whether the URL is allowed by robots.txt. |
| is_pdf | boolean | yes | Whether the content is a PDF. |
| js_render_hint | boolean | yes | Whether the page content suggests JavaScript rendering is needed. |
| json_ld | array | yes | JSON-LD entries found on the page. |
| links | array | yes | Links found on the page. |
| markdown | – | – | Markdown conversion of the page content. |
| metadata | – | yes | Extracted metadata from the page. |
| nofollow_detected | boolean | yes | Whether a nofollow directive was detected. |
| noindex_detected | boolean | yes | Whether a noindex directive was detected. |
| response_meta | – | – | Response metadata extracted from HTTP headers. |
| screenshot_base64 | string|null | – | Base64-encoded PNG screenshot of the page. Populated only when `CrawlConfig.capture_screenshot` was enabled for this request, so callers that never requested a screenshot do not pay the encoding cos… |
| ssrf_refused_urls | array | – | URLs of the requests the page sent in browser mode that the SSRF policy refused, without their credentials, each listed once. The page is kept; only the refused requests failed. |
| status_code | integer | yes | The HTTP status code of the response. |
| was_skipped | boolean | yes | Whether the page was skipped (binary or PDF content). |
| x_robots_tag | string|null | – | The X-Robots-Tag header values, joined with `, ` when the response sent more than one. |
No examples provided.
How do I install the Crawlberg MCP server?
Crawlberg runs locally as a container image, launched with docker run --rm -i ghcr.io/xberg-io/crawlberg:1.11.0. Ready-made configuration for Claude, Cursor, VS Code, Codex and 3 more is on this page, copied from each client's own documentation.
oci · ghcr.io/xberg-io/crawlberg
claude mcp add xberg-io-crawlberg -- docker run --rm -i ghcr.io/xberg-io/crawlberg:1.11.0
What is the Crawlberg MCP server?
Crawlberg is an MCP server listed in the public MCP registry as io.github.xberg-io/crawlberg. Scrape, crawl, and map websites to Markdown or JSON via local CLI. This page covers its container image (ghcr.io/xberg-io/crawlberg).
Is the Crawlberg MCP server safe to use?
Crawlberg scores 48 out of 100 on VerifyMCP. That is a record of what we were able to check automatically, not an endorsement. The category breakdown on this page shows every signal behind the number, including the ones we could not confirm.
What tools does the Crawlberg MCP server expose?
Crawlberg exposes 9 tools: batch_crawl, batch_scrape, crawl, download, generate_citations, and 4 more. Their descriptions and schemas cost roughly 653 tokens of context every time the server is loaded.
Is the Crawlberg MCP server still maintained?
Crawlberg is still listed as active in the MCP registry. We last reached this channel on 17 October 2026. Those dates come from our own scans of the registry and the channel itself, not from anything the publisher announced.
What licence is the Crawlberg MCP server under?
Crawlberg declares the MIT licence, which is OSI-approved. That covers the source only, and says nothing about the cost of any service it calls.