mcp-toolbelt

Check a site's robots.txt for explicit directives targeting known AI crawlers (GPTBot, ClaudeBot, CCBot, PerplexityBot, Google-Extended, Bytespider, Amazonbot, and others used for LLM training or AI search/answer products), and check for an llms.txt file. Useful for publishers deciding whether their content policy toward AI crawlers matches their intent, or for auditing a competitor's stance.

Crawl a site starting from a URL (same-origin pages only, bounded by maxPages) and check every linked URL for broken status codes. Returns broken links with the page(s) they were found on. Note: some external sites (e.g. social platforms) block automated HEAD/GET requests and may show up as false positives.

TCP-connect scan a host for open ports. Defaults to a list of ~20 common service ports (SSH, HTTP/S, mail, DBs, etc.) if none are given. For checking your own infrastructure's exposure — capped at 100 ports per call.

Check whether a site has a valid robots.txt and sitemap.xml, and return their raw contents (truncated).

Send a probe request with a foreign Origin header to a URL and report its CORS response headers. Flags wildcard-origin + credentials combinations and arbitrary-origin reflection, both common CORS misconfigurations.

Query a DNS record for a domain against several major public resolvers (Google, Cloudflare, Quad9, OpenDNS) in parallel and compare the answers. Flags mismatches, which usually mean propagation is still in progress after a DNS change.

Check whether a domain is registered, plus scan common typo-squat variants (adjacent-key substitution, letter omission/doubling, transposition) across popular TLDs (.com, .net, .org, .io, .co, .ai, .app, .dev) for brand-protection or domain-flipping research. WHOIS-based; capped at 40 variants checked per call for latency.

Check a domain's registration expiry (via WHOIS) and DNS health: nameservers, A/AAAA, MX, SPF, and DMARC records. Flags common misconfigurations.

Bundle: runs seo_audit, domain_health_check (WHOIS + DNS: nameservers, A/AAAA, MX, SPF, DMARC), and email_deliverability_check (DKIM, SPF lookup-count, DMARC strength, DNSBL) against a domain in one call, one charge. Cheaper than calling the three tools separately. Each sub-check reports independently, so a failure in one doesn't void the others.

Validate a single email address: RFC syntax check, MX record lookup on the domain, disposable/temporary-email-provider detection, and role-account detection (info@, admin@, etc.). Per-address check — different from email_deliverability_check, which audits a whole domain's sending reputation (SPF/DKIM/DMARC/PTR/DNSBL).

Deep-dive email deliverability check for a domain: MX records + reverse-DNS (PTR) on each MX host, common DKIM selector probing, SPF lookup-count (RFC 7208 caps at 10), DMARC policy strength, and DNSBL blacklist lookups (Spamhaus Zen, SpamCop, Barracuda) on MX IPs. Note: public-resolver DNSBL queries are frequently rate-limited or blocked by Spamhaus, so a `listed: null` result means "unknown", not "clean" — treat null results as inconclusive, not as a clean bill of health.

Check a site for favicon, apple-touch-icon, web app manifest, and theme-color presence — a quick completeness check for browser/OS chrome and PWA metadata.

Check a GitHub repository's health signals: stars, forks, open issues, license, archived/disabled status, and days since last push. Flags likely-abandoned or unlicensed repos. Useful before depending on a repo.

Resolve a hostname to its IPv4 addresses and look up each one's ASN, network prefix, country code, and network owner via Team Cymru's DNS-based WHOIS service (no API key). Country-level only — not city/street geolocation.

Fetch a URL, extract every JSON-LD (<script type="application/ld+json">) block, and validate basic structure — @context/@type presence plus required fields for common schema.org types (Article, Product, Organization, WebSite, LocalBusiness, BreadcrumbList, FAQPage). Reports per-block errors rather than failing the whole call on one bad block.

Look up a package (optionally pinned to a version) against OSV.dev's aggregated vulnerability database (GitHub Advisories, PyPA, RustSec, Go vuln DB, etc.) for known CVEs/advisories. Supports npm, PyPI, crates.io, RubyGems, Go, Maven, NuGet, and Packagist ecosystems. Useful before adding a dependency.

Fetch a URL and measure time-to-first-byte, total fetch time, and response size. Flags missing compression, missing Cache-Control, oversized payloads, and slow TTFB.

Run Google's real PageSpeed Insights (Lighthouse + Chrome UX Report) against a URL: performance score, Core Web Vitals (LCP, CLS, INP/TBT), and real-user field data where available. Authoritative version of a local timing check — hits Google's own infrastructure. Works without an API key at low volume; set GOOGLE_PAGESPEED_API_KEY server-side for higher throughput.

Follow a URL through every HTTP redirect hop and report the full chain, final destination, and issues like redirect loops, too many hops, or HTTPS-to-HTTP downgrades.

Check a URL against Google Safe Browsing's malware/phishing/unwanted-software/PUA blocklists. Requires GOOGLE_SAFE_BROWSING_API_KEY to be configured server-side (free Google Cloud API key).

Fetch a URL and audit its response for security-relevant HTTP headers (HSTS, CSP, X-Frame-Options, X-Content-Type-Options, Referrer-Policy, Permissions-Policy). Flags missing/misconfigured headers with a score.

Fetch a URL and run an SEO audit: title/meta description length, canonical tag, Open Graph + Twitter Card tags, html lang attribute, viewport meta, heading structure, image alt-text coverage, internal/external link counts and generic-anchor-text detection, robots meta (noindex/nofollow), structured data (JSON-LD) presence, and robots.txt/sitemap.xml presence.

Parse a site's sitemap.xml (following one level of sitemap-index nesting) and check the HTTP status of every listed URL. Concurrency-limited, capped at 200 URLs checked per call. Reports broken/redirecting URLs found in the sitemap.

Connect to a host over TLS and report its certificate's expiry date, days remaining, issuer, and subject.

Full SSL Labs-style TLS assessment: overall letter grade, protocol support (TLS 1.0-1.3), cipher strength, certificate chain issues, and known vulnerabilities (Heartbleed, POODLE, etc.) for each endpoint. Slower than ssl_cert_check (can take up to ~90s on a cold cache; SSL Labs caches results for 24h server-side).

Fetch a URL and extract structured data deterministically: JSON-LD blocks, OpenGraph/meta tags, and optional caller-supplied CSS-selector fields (e.g. { price: '.product-price', title: 'h1' }). No LLM involved — pure HTML parsing via CSS selectors, so results are exact matches only, not summarized or inferred.

Fetch a URL and fingerprint its likely tech stack from response headers (server, x-powered-by, x-generator) and HTML markers (generator meta tag, framework/CMS-specific script or class patterns). Best-effort — not exhaustive, no additional paths are probed.

Fetch a URL and return its main text content as clean, readable plain text/markdown-ish output. Strips scripts, styles, and HTML tags.

Test a WebSocket endpoint (ws:// or wss://): attempts the handshake, reports success/failure, time-to-open in ms, and close code/reason. Useful for verifying a WebSocket server is reachable and completes its upgrade handshake before you wire real traffic to it.