IA-QA — 152 QA & Dev Tools for AI Agents
REMOTE · WWW.IA-QA.COM
152 free QA tools for agents: injection scan, evals, RAG, web audits. Smaller packs: /mcp/{pack}.
Available components
Last scanned (9 hours ago)
Scores higher than 97% of remote servers
Reachable. Last reached 5 Nov 2026.
Category Testing and QA
How this component scores in each security and reliability category. Every signal is checked automatically against the live server, and we only credit what we can confirm. How we score → Why this is hard to score →
Endpoint Security83
- The endpoint's TLS certificate is valid, in date, and uses a strong key. View diagnostics → Pass
- No authorisation is required to call this server. Nothing on it is marked destructive, but we can't show that an anonymous caller can only read: full credit needs the server to list at least one tool and every tool to be verifiably read-only. We use a best-effort check of tool names and descriptions to tell read-only tools from tools that change things, so some genuinely read-only tools may not get full credit. Names that start with a read verb (get, list, read, search and similar) and accurate readOnlyHint annotations help. Meet that to earn full credit, or require authorisation. See how to fix → View diagnostics → Partial
- HTTPS is enforced; there's no plaintext access path. View diagnostics → Pass
- The HSTS (Strict-Transport-Security) header is present. View diagnostics → Pass
- DNSSEC is configured correctly; the domain's records validate against the full chain to the root. View diagnostics → Pass
Transport & Reachability100
- Verified streamable-http transport via a live MCP handshake. View diagnostics → Pass
Schema Quality & AI Usability86
- AI-judged instruction clarity (excellent).Pass
- Context-footprint check failed: tool/resource definitions use about 22973 tokens (~151/item across 152 items; 152 tools + 0 resources), over budget; trim descriptions and params. See how to fix → Fail
- Tools include usage examples.Pass
Stability & Change Management100
- Stability check failed: schema churn in the 30 days we've observed: 0 tool removals, 1 breaking changes, 0 auth/transport breaks, 0 additions. See how to fix → Fail
Tool Coverage99
- 100% of tools have a non-trivial description (not blank, and not just the tool's name).Pass
- 97% of tool parameters carry a description.Partial
- Structured output schemas are declared (100% of tools); any adoption earns full credit.Pass
Tool Safety75
- No prompt-injection markers were found in the server instructions, tool names or descriptions we captured.Pass
- 0 of 5 tool(s) whose name or description implies an irreversible operation declare an MCP destructiveHint annotation; "run_eval_contract" implies "eval" and declares readOnlyHint instead, contradicting what its own name says it does. See how to fix → Fail
- An AI judge read all 153 captured unit(s) of tool text and found none that tries to manipulate the model reading it.Pass
Capabilities100
- Implements a current MCP spec version (2026-07-28).Pass
Checks we ran (13)
- TLS certificate
- Authorization
- HTTPS
- HSTS header
- DNSSEC
- Transport
- Schema quality
- Stability
- Tool coverage
- Injection markers
- Destructive annotations
- Judged manipulation
- MCP protocol
Does not apply to this hosted endpoint (2)
- Supply Chain SecurityScored on package channels only. This is a hosted endpoint, so there is nothing to install or inspect.
- Provenance & TransparencyScored on package channels only. This is a hosted endpoint, so there is nothing to install or inspect.
The score does not measure everything. What the score does not measure →
Every change we have recorded for this component, newest first. Security-relevant changes are always shown. ▲ marks a change for the better, ▼ a change for the worse; unmarked changes are neutral.
- 5 Nov 26 0
- The server rewrote its instructions, which are the text every model session reads security
- Tool “cors_test” rewrote its description, which is the text the model reads security
- Tool “mcp_server_evaluate” rewrote its description, which is the text the model reads security
- MCP protocol: Implements a current MCP spec version (2026-07-28). functional
- MCP protocol version: 2025-11-25 → 2026-07-28 functional
- “mcp_server_evaluate” added an optional parameter “test_tool_args” cosmetic
- “mcp_server_evaluate” reworded the description of “test_tool_name” cosmetic
- 4 Nov 26 0
- Tool “find_tool” rewrote its description, which is the text the model reads security
- Tool “rate_tool” rewrote its description, which is the text the model reads security
- Tool “run_eval_contract” rewrote its description, which is the text the model reads security
- “run_eval_contract” reworded the description of “inline_contract” cosmetic
- 3 Nov 26 0
- Tool “compare_models” rewrote its description, which is the text the model reads security
- Tool “estimate_llm_cost” rewrote its description, which is the text the model reads security
- Tool “llm_fit_finder” rewrote its description, which is the text the model reads security
- Tool “llm_generate” rewrote its description, which is the text the model reads security
- Tool “token_budget_calculator” rewrote its description, which is the text the model reads security
- “compare_models” reworded the description of “models” cosmetic
- “context_window_check” reworded the description of “model” cosmetic
- “llm_fit_finder” reworded the description of “top_n” cosmetic
- “llm_fit_finder” reworded the description of “use_case” cosmetic
- “llm_generate” reworded the description of “model” cosmetic
- “shield_analyze” reworded the description of “model” cosmetic
- “token_budget_calculator” reworded the description of “model” cosmetic
- “compare_models” dropped the optional parameter “use_case” cosmetic
- 1 Nov 26 +2
- Stability: pass → fail ▼ security
- A breaking change shipped without a version bump: still 1.0.0 ▼ security
- The server rewrote its instructions, which are the text every model session reads security
- Tool “detect_secrets” rewrote its description, which is the text the model reads security
- Tool “generate_ci_workflow” rewrote its description, which is the text the model reads security
- Tool “jira_to_test_suite” rewrote its description, which is the text the model reads security
- Tool “mcp_schema_lint” rewrote its description, which is the text the model reads security
- Tool “metamorphic_check” rewrote its description, which is the text the model reads security
- Tool “multimodal_eval_guide” rewrote its description, which is the text the model reads security
- Tool “prompt_injection_scan” rewrote its description, which is the text the model reads security
- Tool “rerank_evaluate” rewrote its description, which is the text the model reads security
- Tool “run_eval_contract” rewrote its description, which is the text the model reads security
- Tool “run_vlm_test_suite” rewrote its description, which is the text the model reads security
- Tool “run_vlm_test_suite_batch” rewrote its description, which is the text the model reads security
- Tool “secret_scan” rewrote its description, which is the text the model reads security
- Tool “shield_analyze” rewrote its description, which is the text the model reads security
- Tool “test_skill” rewrote its description, which is the text the model reads security
- Tool “validate_agent_trajectory” rewrote its description, which is the text the model reads security
- “secret_scan” changed the type of “input”: string → string|array ▼ functional
- Schema quality: fail → pass ▲ functional
- “hallucination_check” added an optional parameter “cases” cosmetic
- “json_schema_generate” added an optional parameter “samples” cosmetic
- “generate_ci_workflow” reworded the description of “app_base_url” cosmetic
- “generate_ci_workflow” reworded the description of “app_start_command” cosmetic
- “generate_ci_workflow” reworded the description of “cli_tools” cosmetic
- “generate_ci_workflow” reworded the description of “contract_path” cosmetic
- “generate_ci_workflow” reworded the description of “cron” cosmetic
- “generate_ci_workflow” reworded the description of “fail_on_fail” cosmetic
- “generate_ci_workflow” reworded the description of “gate” cosmetic
- “generate_ci_workflow” reworded the description of “min_score” cosmetic
- “generate_ci_workflow” reworded the description of “node_version” cosmetic
- “generate_ci_workflow” reworded the description of “provider” cosmetic
- “generate_ci_workflow” reworded the description of “triggers” cosmetic
- “generate_ci_workflow” reworded the description of “workflow_name” cosmetic
- “hallucination_check” reworded the description of “answer” cosmetic
- “hallucination_check” reworded the description of “context” cosmetic
- “hallucination_check” reworded the description of “strict” cosmetic
- “jira_to_test_suite” reworded the description of “api_key” cosmetic
- “jira_to_test_suite” reworded the description of “confluence_pages” cosmetic
- “jira_to_test_suite” reworded the description of “issue” cosmetic
- “jira_to_test_suite” reworded the description of “issue_key” cosmetic
- “jira_to_test_suite” reworded the description of “jira_base_url” cosmetic
- “jira_to_test_suite” reworded the description of “jira_email” cosmetic
- “jira_to_test_suite” reworded the description of “jira_token” cosmetic
- “jira_to_test_suite” reworded the description of “max_tokens” cosmetic
- “jira_to_test_suite” reworded the description of “model” cosmetic
- “json_schema_generate” reworded the description of “required_all” cosmetic
- “metamorphic_check” reworded the description of “api_key” cosmetic
- “metamorphic_check” reworded the description of “base” cosmetic
- “metamorphic_check” reworded the description of “baseline_guard” cosmetic
- “metamorphic_check” reworded the description of “mode” cosmetic
- “metamorphic_check” reworded the description of “require_all” cosmetic
- “metamorphic_check” reworded the description of “thresholds” cosmetic
- “metamorphic_check” reworded the description of “variants” cosmetic
- “multimodal_eval_guide” reworded the description of “action” cosmetic
- “multimodal_eval_guide” reworded the description of “api_key” cosmetic
- “multimodal_eval_guide” reworded the description of “clip” cosmetic
- “multimodal_eval_guide” reworded the description of “fid” cosmetic
- “multimodal_eval_guide” reworded the description of “generated_images” cosmetic
- “multimodal_eval_guide” reworded the description of “image_base64” cosmetic
- “multimodal_eval_guide” reworded the description of “image_mime_type” cosmetic
- “multimodal_eval_guide” reworded the description of “image_url” cosmetic
- “multimodal_eval_guide” reworded the description of “metric” cosmetic
- “multimodal_eval_guide” reworded the description of “model” cosmetic
- “multimodal_eval_guide” reworded the description of “real_images” cosmetic
- “multimodal_eval_guide” reworded the description of “score” cosmetic
- “multimodal_eval_guide” reworded the description of “system_prompt” cosmetic
- “multimodal_eval_guide” reworded the description of “test_cases” cosmetic
- “multimodal_eval_guide” reworded the description of “text” cosmetic
- “multimodal_eval_guide” reworded the description of “vqa” cosmetic
- “rerank_evaluate” reworded the description of “api_key” cosmetic
- “rerank_evaluate” reworded the description of “passages” cosmetic
- “rerank_evaluate” reworded the description of “query” cosmetic
- “rerank_evaluate” reworded the description of “threshold” cosmetic
- “rerank_evaluate” reworded the description of “top_k” cosmetic
- “response_quality_score” reworded the description of “expected_keywords” cosmetic
- “run_eval_contract” reworded the description of “api_keys” cosmetic
- “run_eval_contract” reworded the description of “contract_path” cosmetic
- “run_eval_contract” reworded the description of “inline_contract” cosmetic
- “run_vlm_test_suite” reworded the description of “api_key” cosmetic
- “run_vlm_test_suite” reworded the description of “image_base64” cosmetic
- “run_vlm_test_suite” reworded the description of “image_mime_type” cosmetic
- “run_vlm_test_suite” reworded the description of “image_url” cosmetic
- “run_vlm_test_suite” reworded the description of “model” cosmetic
- “run_vlm_test_suite” reworded the description of “system_prompt” cosmetic
- “run_vlm_test_suite” reworded the description of “test_cases” cosmetic
- “run_vlm_test_suite” reworded the description of “threshold” cosmetic
- “run_vlm_test_suite_batch” reworded the description of “api_keys” cosmetic
- “run_vlm_test_suite_batch” reworded the description of “image_base64” cosmetic
- “run_vlm_test_suite_batch” reworded the description of “image_mime_type” cosmetic
- “run_vlm_test_suite_batch” reworded the description of “image_url” cosmetic
- “run_vlm_test_suite_batch” reworded the description of “models” cosmetic
- “run_vlm_test_suite_batch” reworded the description of “system_prompt” cosmetic
- “run_vlm_test_suite_batch” reworded the description of “test_cases” cosmetic
- “run_vlm_test_suite_batch” reworded the description of “threshold” cosmetic
- “secret_scan” reworded the description of “input” cosmetic
- “secret_scan” reworded the description of “types” cosmetic
- “shield_analyze” reworded the description of “model” cosmetic
- “shield_analyze” reworded the description of “prompt” cosmetic
- “shield_analyze” reworded the description of “response” cosmetic
- “shield_analyze” reworded the description of “rules” cosmetic
- “shield_analyze” reworded the description of “source” cosmetic
- “test_skill” reworded the description of “api_key” cosmetic
- “test_skill” reworded the description of “dir_name” cosmetic
- “test_skill” reworded the description of “lint_only” cosmetic
- “test_skill” reworded the description of “model” cosmetic
- “test_skill” reworded the description of “other_skills” cosmetic
- “test_skill” reworded the description of “scenario_count” cosmetic
- “test_skill” reworded the description of “scenarios” cosmetic
- “validate_agent_trajectory” reworded the description of “assertions” cosmetic
- “validate_agent_trajectory” reworded the description of “format” cosmetic
- “validate_agent_trajectory” reworded the description of “trace” cosmetic
- “hallucination_check” made “answer” optional cosmetic
- “hallucination_check” made “context” optional cosmetic
- 29 Oct 26 0
- Tool “llm_generate” rewrote its description, which is the text the model reads security
- “llm_generate” reworded the description of “model” cosmetic
- 28 Oct 26 0
- Tool “ab_test_report” rewrote its description, which is the text the model reads security
- Tool “function_call_validate” rewrote its description, which is the text the model reads security
- Tool “json_schema_validate” rewrote its description, which is the text the model reads security
- Tool “prompt_injection_scan” rewrote its description, which is the text the model reads security
- Tool “rate_tool” rewrote its description, which is the text the model reads security
- Tool “redact_pii” rewrote its description, which is the text the model reads security
- Tool “toxicity_scan” rewrote its description, which is the text the model reads security
- Tool “validate_agent_trajectory” rewrote its description, which is the text the model reads security
- “function_call_validate” reworded the description of “allowed_functions” cosmetic
- “redact_pii” reworded the description of “types” cosmetic
- 26 Oct 26 0
- “compare_models” reworded the description of “models” cosmetic
- “jira_to_test_suite” reworded the description of “model” cosmetic
- “llm_generate” reworded the description of “model” cosmetic
- “model_info” reworded the description of “model” cosmetic
- “token_budget_calculator” reworded the description of “model” cosmetic
5 cosmetic changes on this day. Switch on “Show cosmetic changes” to see them.
- 24 Oct 26 0
- Tool “mcp_server_evaluate” rewrote its description, which is the text the model reads security
- Tool “mcp_server_health_check” rewrote its description, which is the text the model reads security
- “mcp_server_evaluate” reworded the description of “test_tool_name” cosmetic
- “mcp_server_evaluate” reworded the description of “url” cosmetic
- “mcp_server_health_check” reworded the description of “manifest” cosmetic
- “mcp_server_health_check” reworded the description of “strict” cosmetic
Diagnostic detail from the automated scan of this channel: what the scanner observed at each step, so you can see exactly where a check passed or failed. It is informational only and never changes the trust score.
Captured 5 Nov 2026 · Probed https://www.ia-qa.com/mcp
TLS valid
Negotiated TLS 1.3 with TLS_AES_128_GCM_SHA256 .
| Subject | Issuer | Valid from | Valid until | Key | Signature | Serial |
|---|---|---|---|---|---|---|
| CN=www.ia-qa.com | CN=YR2,O=Let's Encrypt,C=US | 10 Oct 2026 | 8 Jan 2027 | RSA 2048 | SHA256-RSA | 660e1d6981514e1320131b13dbf4a910775 |
| SANs: www.ia-qa.com | ||||||
| CN=YR2,O=Let's Encrypt,C=US (CA) | CN=Root YR,O=ISRG,C=US | 3 Sept 2025 | 2 Sept 2028 | RSA 2048 | SHA256-RSA | 4ebd24947e24d394802d84a52fd5b319 |
| CN=Root YR,O=ISRG,C=US (CA) | CN=ISRG Root X1,O=Internet Security Research Group,C=US | 13 May 2026 | 2 Sept 2032 | RSA 4096 | SHA256-RSA | f24b6d17f9d9ad7cb1c9fea78782699f |
Background: What to check on a remote MCP endpoint →
DNSSEC secure
Validation of www.ia-qa.com. — Secure
| Zone | DS | Keys | Algorithms | Outcome |
|---|---|---|---|---|
| . | Trust anchor | 20326, 38696 | 8, 8 | Verified |
| com. | DS published | 19718 | 13 | Verified |
| ia-qa.com. | DS published | 52852 | 8 | Verified |
| www.ia-qa.com. | Verified address RRset verified with the apex keys |
Authentication No authorisation required
The endpoint answered without asking for a token. Anyone who knows the URL can reach it.
| Result | No authorisation required |
|---|---|
| HTTP status | 200 |
| Header | Value |
|---|---|
| strict-transport-security | max-age=63072000; includeSubDomains; preload |
| content-security-policy | default-src 'self';script-src 'self' 'unsafe-inline' 'unsafe-eval' https://www.googletagmanager.com;style-src 'self' 'unsafe-inline' https://fonts.googleapis.com;img-src 'self' data: blob: https:;font-src 'self' https://fonts.gstatic.com data:;connect-src 'self' https: wss: http://localhost:11434 data:;media-src 'self' blob:;object-src 'none';frame-ancestors 'self';base-uri 'self';form-action 'self';script-src-attr 'none';upgrade-insecure-requests |
| x-content-type-options | nosniff |
| x-frame-options | SAMEORIGIN |
| referrer-policy | no-referrer |
| permissions-policy | camera=(), microphone=(), geolocation=(), payment=(), usb=(), display-capture=() |
Background: How OAuth 2.1 works in the 2026 MCP spec →
Transports 2 probes
| Transport | URL | Outcome | Status | Location |
|---|---|---|---|---|
| streamable-http | https://www.ia-qa.com/mcp | Verified | 200 | |
| http (plaintext) | http://www.ia-qa.com/mcp | HTTPS enforced | 301 | https://www.ia-qa.com/mcp |
The tools this component advertises to a client, with an estimated token cost for each. Expand a tool to see its parameters and schema. The per-tool counts are indicative and are not scored directly; the schema's total context footprint is one signal in Schema Quality & AI Usability. A tool's description is untrusted text the model reads on every call, which is what makes this list a security surface and not just an inventory: how tool poisoning works →
mcp_server_health_check ~287
Check an MCP server's tool definitions without contacting it — for a server not deployed yet, a stdio server, or a CI gate. Paste the answer to tools/list (raw JSON-RPC or its result), a bare array of tools, one tool definition, or a manifest carrying name/description; input_shape says which one was read. Validates every tool against the MCP spec (name 1-128 chars of A-Z a-z 0-9 _ - .; camelCase and a tool with no parameters are valid; inputSchema of type object; required fields listed in properties) and, with strict, documentation (description ≥ 20 chars, every parameter with a description and a type). verdict: FAIL when a tool breaks a rule or no tool is found, WARN when only a manifest's server name/description is missing, PASS otherwise. A tools/list carries no server name, so those two checks are not scored on it (the name comes from initialize). Gate on verdict or on checks[tool_quality].pass.
| Name | Type | Req | Description |
|---|---|---|---|
| manifest | string | yes | JSON text: the answer to tools/list (raw JSON-RPC or its result), an array of tools, one tool definition, or a manifest with name/description/tools. |
| strict | boolean | – | Also require documentation: every description ≥ 20 characters, every parameter with a description and a type. |
| Name | Type | Req | Description |
|---|---|---|---|
| checks | array | – | – |
| failed | number | – | – |
| input_shape | string | – | – |
| passed | number | – | – |
| stats | object | – | – |
| toolIssues | – | – | – |
| total | number | – | – |
| verdict | string | – | – |
{"manifest":"{\"tools\":[{\"name\":\"get_weather\",\"description\":\"Return the current weather for a city. Use it when the user asks about the weather now.\",\"inputSchema\":{\"type\":\"object\",\"required\":[\"city\"],\"properties\":{\"city\":{\"type\":\"string\",\"description\":\"City name, e.g. Paris\"}}}}]}"}merge_json ~120
Deep merge two JSON objects. Supports three array strategies: replace (default), concat, or unique (dedup concat). Nested objects are recursively merged — override takes precedence for primitives.
| Name | Type | Req | Description |
|---|---|---|---|
| array_strategy | string | – | Array merge strategy: replace (default), concat, or unique |
| base | string|object|array | yes | The base JSON object (merged into) — a JSON string, or the object itself. |
| override | string|object|array | yes | The override JSON object (takes precedence) — a JSON string, or the object itself. |
| Name | Type | Req | Description |
|---|---|---|---|
| merged | – | – | – |
| new_keys | array | – | – |
| overridden_keys | array | – | – |
| total_keys | number | – | – |
{"base":{"a":{"x":1,"y":2}},"override":{"a":{"y":3}}}metamorphic_check ~299
Reference-free stability check: does an assistant's answer stay the same when the QUESTION is transformed (case, typo, paraphrase, reorder, translation)? You bring the outputs; no model is called (tfidf mode is deterministic and free). Default thresholds: case .95, typo .90, paraphrase .80, reorder .80, translation .75 (embeddings only), specialization .60 (advisory, never gated). Returns PASS / FAIL / INVALID — INVALID when the BASE answer is a refusal or too short, so a model that refuses everything never scores 1.0. Pair with run_semantic_tests for correctness.
| Name | Type | Req | Description |
|---|---|---|---|
| api_key | string | – | OpenAI key, embeddings mode only. |
| base | object | yes | The original run. |
| baseline_guard | object | – | Floor on the BASE answer; failing it returns INVALID. |
| mode | string | – | tfidf (default, lexical: gate on case/typo, read paraphrase as a trend) or embeddings (OpenAI, needs api_key; required for translation). |
| require_all | boolean | – | Default true: every gated variant must pass. Keep it for a gate — false lets one easy typo row hold the run at PASS while a paraphrase fails. |
| thresholds | object | – | Per-relation overrides, 0-1. Calibrate on your own data before gating. |
| variants | array | yes | Answers to transformed versions of the question. |
| Name | Type | Req | Description |
|---|---|---|---|
| baseline | object | – | – |
| mode | string | – | – |
| summary | object | – | – |
| thresholds | object | – | – |
| variants | array | – | – |
| verdict | string | – | – |
| warnings | array | – | – |
| weakest | object | – | – |
{"base":{"output":"Paris is the capital and largest city of France."},"variants":[{"output":"Paris is the capital and largest city of France.","relation":"case"}]}minify_js ~77
Minify a JavaScript snippet, function, class, or module up to 50 KB using Terser. Returns minified code and byte savings. Use when embedding scripts in HTML templates, report payloads, or injecting inline code programmatically.
| Name | Type | Req | Description |
|---|---|---|---|
| code | string | yes | JavaScript code to minify (max 50kb) |
| Name | Type | Req | Description |
|---|---|---|---|
| minified | – | – | – |
{"code":"function add(a, b) {\n return a + b;\n}"}mock_from_schema ~136
Generate realistic mock data from a JSON Schema. Supports all common types (string, number, integer, boolean, array, object, null), format hints (email, date, date-time, uri, uuid), enum, const, and nested schemas. Perfect for testing MCP tools with realistic data.
| Name | Type | Req | Description |
|---|---|---|---|
| count | number | – | Number of mock objects to generate (default: 1, max: 20) |
| schema | string|object|array | yes | The JSON Schema to generate from — a JSON string, or the schema object itself. |
| seed | string | – | Optional seed string for deterministic output (uses first char codes) |
| Name | Type | Req | Description |
|---|---|---|---|
| count | number | – | – |
| results | – | – | – |
{"count":20,"schema":{"properties":{"email":{"format":"email","type":"string"},"id":{"maximum":9,"minimum":5,"type":"integer"},"tag":{"enum":["a","b"]}},"required":["id","email"],"type":"object"}}model_info ~110
Get detailed specs for an AI model: context window, pricing per 1K tokens, knowledge cutoff, provider, multimodal support, reasoning capabilities, and feature list. Covers 30+ models from OpenAI, Anthropic, Google, DeepSeek, Meta, Mistral, Cohere, xAI.
| Name | Type | Req | Description |
|---|---|---|---|
| model | string | yes | Model name (e.g. "gpt-4o", "claude-3.5-sonnet", "gemini-3.1-pro-preview") |
| Name | Type | Req | Description |
|---|---|---|---|
| model | – | – | – |
| pricing_per_1k | object | – | – |
{"model":"gpt-4o"}multimodal_eval_guide ~302
Multimodal evaluation: action=guide (default) interprets a CLIP/FID/VQA score against reference thresholds; clip_score, fid_score, vqa_accuracy compute the metric (HuggingFace, or your VLM key for VQA); pipeline runs all three.
| Name | Type | Req | Description |
|---|---|---|---|
| action | string | – | – |
| api_key | string | – | [vqa_accuracy] Key for the model's provider. |
| clip | object | – | [pipeline] {image_url, text} |
| fid | object | – | [pipeline] {real_images, generated_images} |
| generated_images | array | – | [fid_score] URLs. |
| image_base64 | string | – | [clip_score, vqa_accuracy] |
| image_mime_type | string | – | [clip_score, vqa_accuracy] |
| image_url | string | – | [clip_score, vqa_accuracy] Public image URL. |
| metric | string | – | [guide] |
| model | string | – | [vqa_accuracy] Default gpt-4o. |
| real_images | array | – | [fid_score] URLs. |
| score | number | – | [guide] A score to interpret. |
| system_prompt | string | – | [vqa_accuracy] |
| test_cases | array | – | [vqa_accuracy] |
| text | string | – | [clip_score] Text to compare with the image. |
| vqa | object | – | [pipeline] Same inputs as vqa_accuracy. |
| Name | Type | Req | Description |
|---|---|---|---|
| best_practices | array | – | – |
| comparison_table | array | – | – |
| errors | – | – | – |
| metrics | – | – | – |
| results | – | – | – |
| score_interpretation | – | – | – |
| web_tool | string | – | – |
{"metric":"clip_score","score":72}needle_haystack_generate ~171
Generate a "needle in a haystack" test: embeds a target fact into a large block of filler text at a specified position. Use this to test LLM context window retrieval accuracy. Returns the full haystack, the question to ask, and metadata. No API key needed.
| Name | Type | Req | Description |
|---|---|---|---|
| needle | string | yes | The fact to hide (e.g. "The secret code is ALPHA-42") |
| position | string | – | Where to insert the needle: "start", "middle", "end", "random" (default: "middle") |
| question | string | yes | The question to ask the LLM (e.g. "What is the secret code?") |
| tokens | integer | – | Target haystack size in tokens (default: 5000, max: 100000) |
| Name | Type | Req | Description |
|---|---|---|---|
| estimated_tokens | – | – | – |
| haystack | – | – | – |
| insert_block | – | – | – |
| needle | – | – | – |
| position | – | – | – |
| question | – | – | – |
| total_blocks | number | – | – |
{"needle":"The code is 4242.","position":"middle","question":"What is the code?","tokens":2000}normalize_vector ~79
L2-normalize a float vector (produce a unit vector with norm=1). Required by many vector DBs (Pinecone, Qdrant cosine). Supports batch normalization of up to 1000 vectors.
| Name | Type | Req | Description |
|---|---|---|---|
| batch | array | – | Batch of vectors to normalize (overrides vector) |
| vector | array | – | Single vector to normalize |
| Name | Type | Req | Description |
|---|---|---|---|
| count | number | – | – |
| dimension | number | – | – |
| index | – | – | – |
| mode | string | – | – |
| norm | number | – | – |
| norm_after | number | – | – |
| norm_before | number | – | – |
| normalized | – | – | – |
| results | – | – | – |
| vector | – | – | – |
{"vector":[3,4]}normalize_whitespace ~181
Normalize whitespace: trim trailing spaces, collapse blank lines, normalize line endings (LF/CRLF), convert tabs to spaces. Useful for cleaning code, configs, and text before processing.
| Name | Type | Req | Description |
|---|---|---|---|
| collapse_blanks | boolean | – | Collapse runs of blank lines down to max_blank_lines (default: true) |
| input | string | yes | Text to normalize |
| line_ending | string | – | "lf" (default), "crlf", or "cr" |
| max_blank_lines | number | – | Blank lines to keep when collapsing, 0-10 (default: 2) |
| tab_to_spaces | number | – | Convert tabs to N spaces (omit to keep tabs) |
| trim_file | boolean | – | Trim leading/trailing blank lines (default: true) |
| trim_lines | boolean | – | Trim trailing whitespace from each line (default: true) |
| Name | Type | Req | Description |
|---|---|---|---|
| line_ending | – | – | – |
| normalized_length | number | – | – |
| original_length | number | – | – |
| result | – | – | – |
{"collapse_blanks":true,"input":"a \r\nb\t\r\n\r\n\r\n\r\nc","line_ending":"lf","max_blank_lines":1,"trim_lines":true}number_base_convert ~123
Convert numbers between bases: decimal, binary, octal, hexadecimal, or any base 2–36. Auto-detects 0x, 0b, 0o prefixes.
| Name | Type | Req | Description |
|---|---|---|---|
| from_base | number | – | Source base 2–36 (auto-detects prefix if omitted) |
| input | string | yes | Number to convert (e.g., "255", "0xFF", "0b1010", "0o77") |
| to_base | number | – | Target base 2–36 (omit to get all common bases) |
| Name | Type | Req | Description |
|---|---|---|---|
| binary | string | – | – |
| decimal | – | – | – |
| from_base | – | – | – |
| hexadecimal | string | – | – |
| octal | string | – | – |
| result | string | – | – |
| to_base | – | – | – |
{"from_base":16,"input":"ff","to_base":10}openapi_validate ~139
Validate the structure of an OpenAPI 3.x specification (JSON or YAML). Checks required top-level fields (openapi, info.title, info.version, paths), validates each operation (responses, operationId uniqueness), detects undeclared $ref components, and flags missing 2xx responses. Returns a PASS/FAIL verdict, a 0–100 compliance score, and a list of errors and warnings with JSON-pointer locations. Use before publishing an API spec or generating SDK code.
| Name | Type | Req | Description |
|---|---|---|---|
| input | string|object|array | yes | The OpenAPI 3.x spec — a JSON string, a YAML string, or the already-parsed spec object. |
| Name | Type | Req | Description |
|---|---|---|---|
| errors | – | – | – |
| score | number | – | – |
| stats | object | – | – |
| verdict | – | – | – |
| warnings | – | – | – |
{"input":{"info":{"title":"t","version":"1"},"openapi":"3.0.3","paths":{"/a":{"get":{"operationId":"getA","responses":{"200":{"description":"ok"}}}}}}}optimize_prompt_tokens ~74
Compress an LLM prompt by removing filler words, verbose phrases, duplicate sentences, and unnecessary whitespace. Returns optimized text with token savings breakdown. 100% deterministic, no API key needed.
| Name | Type | Req | Description |
|---|---|---|---|
| options | object | – | Toggle optimization steps (all true by default) |
| text | string | yes | The prompt text to optimize |
| Name | Type | Req | Description |
|---|---|---|---|
| optimized | – | – | – |
| percent_saved | string | – | – |
| steps | – | – | – |
| tokens_after | – | – | – |
| tokens_before | – | – | – |
| tokens_saved | – | – | – |
{"text":"Please kindly make sure that you basically summarize the following text."}parse_csv ~142
Parse a CSV string into a JSON array of objects (or raw arrays). Full RFC 4180: quoted fields may contain the delimiter, embedded newlines (the Excel/Sheets multi-line cell), and doubled quotes. Custom delimiters supported. An unterminated quote is rejected with its position rather than parsed into corrupted rows. Use when processing spreadsheet exports, data imports, or structured text pipelines where the source is CSV. Supports up to 200 KB.
| Name | Type | Req | Description |
|---|---|---|---|
| delimiter | string | – | Field delimiter character (default: ",") |
| header | boolean | – | Treat the first row as headers (default: true) |
| input | string | yes | CSV content to parse |
| Name | Type | Req | Description |
|---|---|---|---|
| columns | number | – | – |
| headers | array | – | – |
| row_count | number | – | – |
| rows | array | – | – |
{"delimiter":";","input":"a;b\n1;2"}parse_http_headers ~88
Parse a raw HTTP headers block into a structured JSON object. Detects multi-value headers, masks Authorization values, and optionally audits for missing security headers (HSTS, CSP, X-Frame-Options, etc.).
| Name | Type | Req | Description |
|---|---|---|---|
| analyze_security | boolean | – | Audit for missing security headers (default: true) |
| headers | string | yes | Raw HTTP headers (one "Name: Value" per line) |
| Name | Type | Req | Description |
|---|---|---|---|
| header_count | number | – | – |
| parsed | object | – | – |
| security | object | – | – |
{"headers":"Content-Type: text/html\nSet-Cookie: a=1\nSet-Cookie: b=2"}post_jira_comment ~141
Post the output of jira_to_test_suite as a formatted comment on the source Jira ticket. Converts Gherkin, E2E steps, API tests, and ambiguities into Atlassian Document Format (ADF). STATEFUL — creates a comment on the issue.
| Name | Type | Req | Description |
|---|---|---|---|
| issue_key | string | yes | Jira issue key, e.g. "PROJ-123" |
| jira_base_url | string | yes | Atlassian base URL |
| jira_email | string | yes | Atlassian account email |
| jira_token | string | yes | Atlassian API token |
| test_suite | object | yes | The test_suite object from jira_to_test_suite result |
| Name | Type | Req | Description |
|---|---|---|---|
| comment_id | string | – | – |
| comment_url | string | – | – |
| success | boolean | – | – |
{"issue_key":"PROJ-1","jira_base_url":"https://yourcompany.atlassian.net","jira_email":"you@example.com","jira_token":"\u003cyour-atlassian-token\u003e","test_suite":{"acceptance_criteria":["A reset link expires 30 minutes after it is sent."]}}pr_gatekeeper ~162
Compound quality gate for pull requests. Runs three sequential checks: (1) secret detection — scans diff for API keys, tokens, passwords matching 16 regex patterns; (2) bug analysis — heuristic scan for eval(), innerHTML, empty catch, console.log, TODO/FIXME; (3) commit message linting against Conventional Commits spec. Returns gate verdict (PASS/WARN/BLOCK), blockers, and actionable warnings. Use before merging any code change.
| Name | Type | Req | Description |
|---|---|---|---|
| commit_message | string | yes | The commit message to lint (e.g. "feat(auth): add OAuth2 login") |
| context | string | – | Optional: PR title or description for richer bug analysis |
| diff | string | yes | Unified git diff (output of `git diff HEAD`) |
| Name | Type | Req | Description |
|---|---|---|---|
| checks | object | – | – |
| flags | array | – | – |
| score | number | – | – |
| verdict | string | – | – |
{"commit_message":"fix(pay): accept a zero amount","diff":"diff --git a/pay.js b/pay.js\n--- a/pay.js\n+++ b/pay.js\n@@ -1 +1 @@\n-if (amount \u003e 0) charge(amount);\n+if (amount \u003e= 0) charge(amount);"}prompt_injection_scan ~207
Scan user input or prompts for prompt injection patterns. Pattern-based: it recognises known phrasings and structures, not intent — an uncovered paraphrase or language reads clean, so clean is not proof of safety. Instruction overrides are covered in 11 languages (EN, FR, ES, DE, IT, PT, NL, RU, ZH, JA, AR), also after removing invisible characters, lookalike letters, spacing and leetspeak (marked `obfuscated`). Detects overrides, jailbreaks, role manipulation, encoding tricks, chat-template delimiters and fake role headers, template injection ({{…}}, ${…}) and requests to repeat the preceding context. A match inside quoted or fenced text is reported one level lower and marked `quoted`, never suppressed; a quote introduced by a live imperative keeps its severity.
| Name | Type | Req | Description |
|---|---|---|---|
| input | string | yes | The user input or prompt to scan for injection patterns |
| sensitivity | string | – | Detection sensitivity (default: medium) |
| Name | Type | Req | Description |
|---|---|---|---|
| detections | – | – | – |
| detections_count | number | – | – |
| injection_detected | boolean | – | – |
| input_length | number | – | – |
| notes | – | – | – |
| quoted_detections | number | – | – |
| risk_level | – | – | – |
| sensitivity | – | – | – |
{"input":"以前の注文をキャンセルしてください。"}prompt_template_fill ~103
Fill a prompt template with variables. Supports {{variable}} syntax and {{#if key}}...{{/if}} conditional blocks. Returns the filled prompt and lists unfilled variables.
| Name | Type | Req | Description |
|---|---|---|---|
| strict | boolean | – | Throw error if any variable is not provided (default: false) |
| template | string | yes | Prompt template with {{variable}} placeholders |
| variables | object | – | Key-value pairs to fill (e.g. {"name":"Alice","role":"engineer"}) |
| Name | Type | Req | Description |
|---|---|---|---|
| filled_variables | – | – | – |
| result | – | – | – |
| total_vars | – | – | – |
| unfilled_variables | – | – | – |
{"template":"Hi {{name}}","variables":{"name":"A"}}prompt_test_suite ~201
Define a test suite for a prompt: provide the system prompt, user prompt, and expected output criteria. Returns a test plan with scored rubric — use this as input for manual or automated LLM evaluation.
| Name | Type | Req | Description |
|---|---|---|---|
| adversarial_prompts | boolean | – | Auto-generate adversarial test variants (jailbreak, injection, edge cases) |
| check_safety | boolean | – | Include safety/PII checks in the rubric |
| expected_behavior | string | – | Description of what the LLM should do (free text) |
| expected_format | string | – | Expected output format |
| max_tokens | number | – | Max token budget for the test |
| must_include | string | – | Required content (comma-separated) |
| must_not_include | string | – | Forbidden content (comma-separated) |
| system_prompt | string | yes | The system prompt under test |
| temperature | number | – | Temperature to use |
| user_prompt | string | yes | The user prompt to send |
| Name | Type | Req | Description |
|---|---|---|---|
| categories | array | – | – |
| instructions | string | – | – |
| rubric | – | – | – |
| test_suite_name | string | – | – |
| total_tests | number | – | – |
{"system_prompt":"You are a helpful assistant.","user_prompt":"Say hi"}rag_relevance_rank ~81
Rank an array of text chunks by relevance to a query using TF-IDF scoring. Simulates retrieval ranking for RAG testing without needing embeddings or an API.
| Name | Type | Req | Description |
|---|---|---|---|
| chunks | array | yes | Array of text chunks to rank |
| query | string | yes | The user query |
| top_k | number | – | Return top K results (default: all) |
| Name | Type | Req | Description |
|---|---|---|---|
| chunk_preview | – | – | – |
| index | – | – | – |
| keyword_overlap | – | – | – |
| query | – | – | – |
| rank | number | – | – |
| results | – | – | – |
| returned | number | – | – |
| score | string | – | – |
| total_chunks | number | – | – |
{"chunks":["a b c","d e f"],"query":"quantum"}rate_tool ~308
Give honest usage feedback on an IA-QA MCP tool. Give a score (1-5) and a comment. Rate low (1-2) if the tool was wrong, irrelevant, or a poor fit; rate high (4-5) only if it genuinely solved your need. Maintainers read ratings, one vote per agent and tool a month: comments and per-tool scores are not published, only the overall average and vote count. Comments are screened: one carrying a credential is not stored, personal data becomes placeholders — describe what happened, not what a model should do. Skip routine successes: signal, not praise. Example: rate_tool({ tool_name: "format_json", score: 2, comment: "Tried to pretty-print a JSON5 file, it rejected trailing commas — not usable for my case." })
| Name | Type | Req | Description |
|---|---|---|---|
| comment | string | – | Strongly encouraged — explain what you were trying to do and whether the tool got you there. Be specific about what was missing, wrong, or a poor fit. This is the most valuable part of the rating. Up… |
| score | number | yes | Rating from 1 (poor) to 5 (excellent) |
| tool_name | string | yes | Name of the MCP tool to rate (e.g. "format_json", "shield_analyze") |
| Name | Type | Req | Description |
|---|---|---|---|
| comment | – | – | – |
| message | string | – | – |
| ok | boolean | – | – |
| rated_at | string | – | – |
| score | – | – | – |
| stored_chars | number | – | – |
| tool_name | – | – | – |
| truncated | boolean | – | – |
{"comment":"Formatted a 2 MB payload correctly.","score":4,"tool_name":"format_json"}redact_pii ~145
Automatically detect and redact Personally Identifiable Information (PII) from text. Replaces emails, phone numbers, SSNs, IBANs, credit cards, IP addresses, and JWT tokens with [REDACTED_TYPE] placeholders. Safe to use before logging or sending to an LLM.
| Name | Type | Req | Description |
|---|---|---|---|
| input | string | yes | Text to redact PII from |
| marker | string | – | Custom replacement marker (default: "REDACTED"). Result: [REDACTED_EMAIL] |
| types | string | – | Comma-separated types to redact (default: all). Options: email, phone, ssn, iban, credit_card, ip_address, jwt |
| Name | Type | Req | Description |
|---|---|---|---|
| clean | boolean | – | – |
| pii_found | – | – | – |
| redacted_text | – | – | – |
| replacements | – | – | – |
| total_redactions | – | – | – |
{"input":"order #12345678"}regex_test ~183
Test a regular expression pattern against an input string and return all matches with their index positions and named capture groups. Use for validating user inputs, extracting structured data from text, or debugging regex patterns. Supports flags g, i, m, s, u, y. The match runs in an isolated thread with a 500 ms budget: a pattern that blows up (catastrophic backtracking, e.g. "(a+)+$") comes back as redos_detected:true — a real ReDoS verdict on your pattern — instead of hanging.
| Name | Type | Req | Description |
|---|---|---|---|
| flags | string | – | Regex flags: g (global), i (case-insensitive), m (multiline), s (dotAll) — default: "" |
| input | string | yes | The string to test against (max 50 KB) |
| pattern | string | yes | Regular expression pattern (without delimiters) |
| Name | Type | Req | Description |
|---|---|---|---|
| elapsed_ms | number | – | – |
| flags | – | – | – |
| match_count | number | – | – |
| matched | – | – | – |
| matches | – | – | – |
| note | – | – | – |
| pattern | – | – | – |
| redos_detected | boolean | – | – |
| verdict | – | – | – |
{"flags":"g","input":"a 12-34 b 5-6","pattern":"\\d+-\\d+"}rerank_evaluate ~195
Evaluate RAG retrieval: rank passages against a query, compute Precision@k / Recall@k and a PASS/FAIL verdict from your relevance labels. Modes: (1) give every passage the `score` YOUR reranker produced — deterministic, offline, the mode to gate CI on; (2) no scores, no key — a local BM25 keyword baseline, never a judgement of a neural reranker; (3) api_key for an NVIDIA account that still has reranking (NVIDIA retired the hosted endpoints on 2026-05-18).
| Name | Type | Req | Description |
|---|---|---|---|
| api_key | string | – | NVIDIA key, used only when no passage has a score; never stored. |
| passages | array | yes | 2 to 20 passages. |
| query | string | yes | – |
| threshold | number | – | Minimum Precision@k to PASS (default 0.5). |
| top_k | integer | – | Default 3. |
| Name | Type | Req | Description |
|---|---|---|---|
| has_ground_truth | boolean | – | – |
| k | number | – | – |
| latency_ms | number | – | – |
| mode | string | – | – |
| model | string | – | – |
| precision_at_k | number | – | – |
| ranked | array | – | – |
| recall_at_k | number | – | – |
| threshold | number | – | – |
| total_passages | number | – | – |
| verdict | string | – | – |
{"api_key":"\u003cyour-api-key\u003e","passages":[{"relevant":true,"text":"Paris is the capital of France."},{"relevant":false,"text":"Bananas are rich in potassium."}],"query":"capital of France"}response_quality_score ~189
Score an LLM response against the criteria you pass: coverage of expected_keywords and compliance with max_length. Returns a 0-100 score over the criteria actually measured — or total_score null with the reason when none is given, because relevance and correctness cannot be read off word overlap (a correct one-word answer shares no words with its question). For meaning against a reference answer, use run_semantic_tests. Also returns unscored signals: question-term overlap, average sentence length, markdown.
| Name | Type | Req | Description |
|---|---|---|---|
| expected_keywords | array | – | Keywords a correct answer must contain, as whole words (case-insensitive; a plural in -s/-es counts). Scored as coverage. |
| max_length | number | – | Maximum character length. Scored as a proportional penalty beyond it. |
| question | string | yes | The original question/prompt |
| response | string | yes | The LLM response to score |
| Name | Type | Req | Description |
|---|---|---|---|
| breakdown | object | – | – |
| grade | – | – | – |
| max_score | number | – | – |
| not_measured | array | – | – |
| reason | string | – | – |
| signals | object | – | – |
| stats | object | – | – |
| total_score | – | – | – |
{"question":"q","response":"anything"}run_eval_contract ~244
Run an IA-EVAL test suite (.ia-eval.yaml contract): call the model per scenario; return passed (true, false, null if undecided) and a verdict (PASS, FAIL, PARTIAL). Expectations are assertions that fail a scenario whatever its score; evaluators give the score. Hosted: pass inline_contract. validate_only calls no model.
| Name | Type | Req | Description |
|---|---|---|---|
| api_keys | object | – | Your keys per provider. Only Groq falls back to the server key; any other provider needs yours. |
| contract_path | string | – | A .ia-eval.yaml under the server's working directory (local use). |
| inline_contract | object | – | The contract as an object: "metadata" ({name, version, model?, provider?}), "expectations", "evaluators" and "scenarios" ([{id, input, expectations?}]). generate_eval_yaml scaffolds one. |
| overrides | object | – | Override contract defaults |
| validate_only | boolean | – | Check the contract against the IA-EVAL schema and stop: no model call, no key, deterministic. Returns valid + issues instead of throwing. Gate a PR that edits a contract on valid. |
| Name | Type | Req | Description |
|---|---|---|---|
| contract_path | – | – | – |
| issues | – | – | – |
| metadata | object | – | – |
| scenario_results | – | – | – |
| summary | object | – | – |
| valid | boolean | – | – |
| warnings | – | – | – |
{"inline_contract":{"expectations":{"contains":["Paris"],"min_score":70},"metadata":{"name":"smoke","version":"1.0"},"scenarios":[{"ground_truth":"Paris","id":"capital","input":"Capital of France? One word."}]}}run_pr_gate_pipeline ~183
Review triage for a pull request. Takes a unified git diff (`git diff HEAD`) and returns: diff-lint findings with the lines that produced them, regression impact areas, a risk score 0–100 with the factors that built it (churn, files touched, sensitive paths, whether any test file changed, lint severities, impacted risk areas), generated test cases, and a PASS / CONDITIONAL / BLOCK recommendation. Advisory: the score measures properties of the diff, not the correctness of the change — it does not read the code semantically and does not replace a reviewer or a static analyser. See notAnalysed in the response.
| Name | Type | Req | Description |
|---|---|---|---|
| context | string | – | Optional PR title or description for richer analysis |
| git_diff | string | yes | Unified git diff (output of `git diff HEAD` or copied from GitHub diff view) |
| Name | Type | Req | Description |
|---|---|---|---|
| bugsFound | number | – | – |
| changedFiles | – | – | – |
| critical | – | – | – |
| disclaimer | string | – | – |
| high | – | – | – |
| impactAreas | array | – | – |
| inputFormat | string | – | – |
| mergeRecommendation | – | – | – |
| notAnalysed | array | – | – |
| riskFactors | array | – | – |
| riskLevel | – | – | – |
| riskScore | – | – | – |
| severityLevel | – | – | – |
| sla | – | – | – |
| testCasesGenerated | – | – | – |
| topBugs | array | – | – |
{"git_diff":"diff --git a/pay.js b/pay.js\n--- a/pay.js\n+++ b/pay.js\n@@ -1 +1 @@\n-if (amount \u003e 0) charge(amount);\n+if (amount \u003e= 0) charge(amount);"}run_semantic_tests ~216
Semantic assertion primitive: compare actual vs expected text pairs using cosine similarity + ROUGE-L. Two modes: tfidf (default, free, no API key) or embeddings (OpenAI text-embedding-3-small, BYOK, true semantic similarity). Returns per-case PASS/FAIL verdicts and an overall verdict. CI-ready: pipe the JSON verdict field to gate a build.
| Name | Type | Req | Description |
|---|---|---|---|
| api_key | string | – | OpenAI API key — required only when mode is embeddings. |
| cases | array | yes | Array of (actual, expected) pairs to evaluate. |
| mode | string | – | tfidf (default): fast, free, lexical. embeddings: OpenAI text-embedding-3-small, true semantic similarity, requires api_key. |
| require_all | boolean | – | If true (default), all cases must pass for overall PASS. If false, at least one case passing returns PASS. |
| thresholds | object | – | Pass/fail thresholds (defaults: cosine 0.75, rouge_l 0.5). |
| Name | Type | Req | Description |
|---|---|---|---|
| failed | number | – | – |
| mode | string | – | – |
| passed | number | – | – |
| results | array | – | – |
| total | number | – | – |
| verdict | string | – | – |
{"cases":[{"actual":"Paris is the capital of France","expected":"The capital of France is Paris"}]}run_vlm_test_suite ~209
Test a vision model (OpenAI, Anthropic or Gemini) on one image: each test case asks a question and asserts on the answer (contains, not_contains, json_format, min_length, max_length, semantic_contains = TF-IDF cosine ≥ 0.4). Returns per-case PASS/FAIL, the pass rate, an overall PASS/WARNING/FAIL against threshold, and latency. BYOK: your key for the model's provider; the image is fetched only from a public address.
| Name | Type | Req | Description |
|---|---|---|---|
| api_key | string | yes | Key for the model's provider. |
| image_base64 | string | – | – |
| image_mime_type | string | – | For image_base64 (default image/jpeg). |
| image_url | string | – | Public image URL (or image_base64). |
| model | string | yes | – |
| system_prompt | string | – | – |
| test_cases | array | yes | – |
| threshold | number | – | Pass rate for an overall PASS (default 80). |
| Name | Type | Req | Description |
|---|---|---|---|
| failed | number | – | – |
| model | string | – | – |
| passed | number | – | – |
| results | array | – | – |
| total | number | – | – |
| verdict | string | – | – |
{"api_key":"\u003cyour-api-key\u003e","image_url":"https://www.ia-qa.com/media/discover-report-poster.png","model":"gemini-2.5-flash","test_cases":[{"assertion_type":"contains","assertion_value":"yes","question":"Does the image contain text? Answer yes or no."}]}run_vlm_test_suite_batch ~156
Compare several vision models on the same image and test cases, in parallel (same assertions as run_vlm_test_suite). Returns per-model PASS/FAIL, pass rates, latency and a comparison table. BYOK: one key per model.
| Name | Type | Req | Description |
|---|---|---|---|
| api_keys | object | yes | Model id → API key. |
| image_base64 | string | – | – |
| image_mime_type | string | – | For image_base64 (default image/jpeg). |
| image_url | string | – | Public image URL (or image_base64). |
| models | array | yes | – |
| system_prompt | string | – | – |
| test_cases | array | yes | – |
| threshold | number | – | Pass rate for an overall PASS (default 80). |
| Name | Type | Req | Description |
|---|---|---|---|
| suites | array | – | – |
| total_failed | number | – | – |
| total_passed | number | – | – |
| verdict | string | – | – |
{"api_keys":{"gemini-2.5-flash":"\u003cyour-google-key\u003e","gpt-4o-mini":"\u003cyour-openai-key\u003e"},"image_url":"https://www.ia-qa.com/media/discover-report-poster.png","models":["gpt-4o-mini","gemini-2.5-flash"],"test_cases":[{"question":"Does the image contain text? Answer yes or no."}]}sandbox_scenario ~301
Get a ready-made selector-drift test case with a known-correct answer, for testing this MCP server or an agent workflow end to end. Each scenario is a real DOM capture of a deliberately breakable app, taken before and after one specific UI change (a renamed label, two swapped buttons, a duplicated locator, an element moved behind a menu…), plus the verdict those two contracts MUST produce. Call with no arguments to list the scenarios; call with a scenario id to get "baseline" and "current" mappings. THE LOOP: pass baseline and current to diff_mappings, then compare its "verdict" and "counts" to this tool's "expected" — they must match exactly. A mismatch means this server's diff engine has drifted, not that your inputs are wrong. Deterministic and offline: the captures are committed fixtures, identical on every call. Try it live at https://www.ia-qa.com/devtools/sandbox
| Name | Type | Req | Description |
|---|---|---|---|
| include_html | boolean | – | Include the generated HTML of the mutated page (default false). Only useful if you want to render or re-capture it yourself; the loop does not need it. |
| scenario | string | – | Scenario id. Omit to list every available scenario with its expected verdict. Ids: no-change, swap-label, add-testid, duplicate-role-name, remove-element, insert-sibling, rename-label, counter-label,… |
| Name | Type | Req | Description |
|---|---|---|---|
| baseline | object | – | – |
| blurb | – | – | – |
| count | number | – | – |
| current | object | – | – |
| expected | object | – | – |
| how_to_run_the_loop | – | – | – |
| html | – | – | – |
| mutations | array | – | – |
| page | – | – | – |
| scenario | – | – | – |
| scenarios | array | – | – |
| teaches | – | – | – |
| title | – | – | – |
{"scenario":"swap-label"}score_geo_signals ~89
Analyze a webpage <head> HTML (or full HTML) for GEO (Generative Engine Optimization) signals. Returns a score /60 with per-check results and improvement tips. GEO = optimizing pages for AI-powered search engines (ChatGPT Search, Perplexity, etc.).
| Name | Type | Req | Description |
|---|---|---|---|
| head_html | string | yes | Raw HTML of the <head> section (or full page HTML) to analyze |
| Name | Type | Req | Description |
|---|---|---|---|
| checks | – | – | – |
| grade | – | – | – |
| max_score | number | – | – |
| passed | number | – | – |
| score | – | – | – |
| total_checks | number | – | – |
{"head_html":"\u003chead\u003e\u003ctitle\u003eRefund policy | Shop\u003c/title\u003e\u003cmeta name=\"description\" content=\"How refunds work at Shop.\"\u003e\u003c/head\u003e"}search_jira_issues ~200
Search Jira using JQL (Jira Query Language). Returns matching issues with key fields. Ideal for finding open bugs, sprint tickets, or issues by label/assignee/component. BYOK — credentials transit in-memory only, never stored.
| Name | Type | Req | Description |
|---|---|---|---|
| fields | array | – | Fields per issue. Default: summary, status, assignee, priority, issuetype, labels, created, updated |
| jira_base_url | string | yes | Atlassian base URL, e.g. "https://mycompany.atlassian.net" |
| jira_email | string | yes | Atlassian account email |
| jira_token | string | yes | Atlassian API token |
| jql | string | yes | JQL query string, e.g. "project = PROJ AND status = Open AND assignee = currentUser() ORDER BY priority DESC" |
| max_results | number | – | Max issues to return (default: 10, max: 50) |
| Name | Type | Req | Description |
|---|---|---|---|
| issues | array | – | – |
| jql | string | – | – |
| returned | number | – | – |
| total | number | – | – |
{"jira_base_url":"https://yourcompany.atlassian.net","jira_email":"you@example.com","jira_token":"\u003cyour-atlassian-token\u003e","jql":"project = PROJ AND status = \"To Do\""}secret_scan ~245
Scan text or code for leaked secrets: API keys (AWS, GCP, Azure, OpenAI, Anthropic, Stripe, GitHub, GitLab, Slack, Twilio, SendGrid, HuggingFace), Slack / Teams / Discord webhook URLs, private keys, JWTs, database connection strings, Bearer and Basic auth headers, and hard-coded values assigned to password/secret/token names (a value read at run time — Cypress.env(), os.getenv() — is not flagged). Returns findings with type, severity, line and a redacted preview. Regex-based, no network.
| Name | Type | Req | Description |
|---|---|---|---|
| input | string|array | yes | Text to scan — or an array of texts / {id, text} for a batch (one result each, worst gate on top). |
| types | string | – | Comma-separated families (default all): aws, gcp, azure, openai, anthropic, huggingface, github, gitlab, stripe, slack, teams, discord, cookie, twilio, sendgrid, jwt, private_key, connection_string,… |
| Name | Type | Req | Description |
|---|---|---|---|
| findings | – | – | – |
| findings_count | number | – | – |
| input_lines | number | – | – |
| risk_level | – | – | – |
| scanned_types | – | – | – |
| secrets_found | – | – | – |
| summary | string | – | – |
{"input":"redis://cache:6379"}security_headers_check ~265
Analyse the HTTP security headers of a public URL OR of raw response headers you paste in. Grades each header (A–F) for: Strict-Transport-Security, Content-Security-Policy, X-Frame-Options, X-Content-Type-Options, Referrer-Policy, Permissions-Policy, X-XSS-Protection, Cross-Origin-Opener-Policy, Cross-Origin-Resource-Policy, and Cross-Origin-Embedder-Policy. Returns an overall score (0–100), per-header grades, missing headers, and fix snippets for Express, Nginx, and Apache. For localhost/private targets the remote server cannot reach, pass the `headers` parameter instead of `url`.
| Name | Type | Req | Description |
|---|---|---|---|
| headers | – | – | Optional, and sufficient on its own (no url needed). The response headers to grade, either as an object {"strict-transport-security": "max-age=...", ...} or as the raw header block pasted as a string… |
| url | string | – | Optional. Full public URL to check (e.g. https://example.com). Omit it entirely when using `headers`. The server cannot reach localhost/private IPs. |
| Name | Type | Req | Description |
|---|---|---|---|
| details | – | – | – |
| fix | – | – | – |
| grade | – | – | – |
| header | – | – | – |
| headers_checked | number | – | – |
| key | – | – | – |
| missing | – | – | – |
| missing_count | number | – | – |
| overall_grade | – | – | – |
| score | – | – | – |
| source | – | – | – |
| url | – | – | – |
| value | – | – | – |
| weak | – | – | – |
| weak_count | number | – | – |
| weight | – | – | – |
{"url":"https://www.ia-qa.com"}shield_analyze ~237
One-call safety gate for an LLM response: grounding (with source), prompt injection on the response AND the prompt (checks.injection / checks.injection_prompt), toxicity, PII, guardrail rules, quality, plus an LLM review on Groq (server key; model "none" skips it and sends nothing out). Returns PASS/FIX/BLOCK, a 0-100 safety score, per-check results and fixes. PII gates the verdict: SSN or card → BLOCK, email → FIX, phone/IP flagged only.
| Name | Type | Req | Description |
|---|---|---|---|
| model | string | – | LLM review model: openai/gpt-oss-20b, openai/gpt-oss-120b, qwen/qwen3.8-27b (default openai/gpt-oss-20b), or "none". |
| prompt | string | – | The original prompt — also scanned for injection; pass it whenever you have it. |
| response | string | yes | The LLM response to check. |
| rules | array | – | Same format as guardrail_test. |
| source | string | – | Reference text for the grounding check. |
| Name | Type | Req | Description |
|---|---|---|---|
| checks | object | – | – |
| flags | – | – | – |
| grade | – | – | – |
| score | – | – | – |
| verdict | – | – | – |
{"model":"none","prompt":"When will my order arrive?","response":"Your order #1234 will arrive tomorrow."}similarity_score ~206
Compute text similarity between reference and hypothesis using multiple metrics: Cosine (BoW, TF-IDF), Jaccard, ROUGE-1, ROUGE-2, ROUGE-L, and BLEU. No API key needed. Ideal for LLM eval (expected vs actual), RAG quality checks, and NLG benchmarking. Supports batch mode.
| Name | Type | Req | Description |
|---|---|---|---|
| batch | array | – | Batch mode: array of {reference, hypothesis} pairs. |
| hypothesis | string | – | Hypothesis / actual text (LLM output) |
| metrics | array | – | Metrics to compute (default: all). Options: "cosine_bow", "cosine_tfidf", "jaccard", "rouge1", "rouge2", "rougeL", "bleu" |
| reference | string | – | Reference / expected text (ground truth) |
| threshold | number | – | Optional pass/fail threshold (0-1). Applies to ROUGE-L F1 score. |
| Name | Type | Req | Description |
|---|---|---|---|
| count | number | – | – |
| f1 | number | – | – |
| mode | string | – | – |
| precision | number | – | – |
| recall | number | – | – |
| results | – | – | – |
{"hypothesis":"the cat sat on the mat","reference":"the cat sat on the mat","threshold":0.5}How do I install the IA-QA — 152 QA & Dev Tools for AI Agents MCP server?
IA-QA — 152 QA & Dev Tools for AI Agents is a hosted endpoint at https://www.ia-qa.com/mcp, so there is nothing to install locally. Ready-made configuration for Claude, Cursor, VS Code, Codex and 5 more is on this page, copied from each client's own documentation.
remote · www.ia-qa.com
claude mcp add --transport http jcjamet-ia-qa-toolbox 'https://www.ia-qa.com/mcp'
What is the IA-QA — 152 QA & Dev Tools for AI Agents MCP server?
IA-QA — 152 QA & Dev Tools for AI Agents is an MCP server listed in the public MCP registry as io.github.JcJamet/ia-qa-toolbox. 152 free QA tools for agents: injection scan, evals, RAG, web audits. Smaller packs: /mcp/{pack}. This page covers its hosted endpoint (https://www.ia-qa.com/mcp).
Is the IA-QA — 152 QA & Dev Tools for AI Agents MCP server safe to use?
IA-QA — 152 QA & Dev Tools for AI Agents scores 89 out of 100 on VerifyMCP. That is a record of what we were able to check automatically, not an endorsement. The category breakdown on this page shows every signal behind the number, including the ones we could not confirm.
What tools does the IA-QA — 152 QA & Dev Tools for AI Agents MCP server expose?
IA-QA — 152 QA & Dev Tools for AI Agents exposes 152 tools: get_testing_guidelines, find_tool, generate_test_cases, jira_to_test_suite, analyze_diff_bugs, and 147 more. Their descriptions and schemas cost roughly 22,457 tokens of context every time the server is loaded.
Does the IA-QA — 152 QA & Dev Tools for AI Agents MCP server require authentication?
No. We connected to IA-QA — 152 QA & Dev Tools for AI Agents without credentials and it answered, so anything it exposes is reachable by anyone who knows the address.
Is the IA-QA — 152 QA & Dev Tools for AI Agents MCP server still maintained?
IA-QA — 152 QA & Dev Tools for AI Agents is still listed as active in the MCP registry. We last reached this channel on 5 November 2026. Those dates come from our own scans of the registry and the channel itself, not from anything the publisher announced.
What did VerifyMCP test on the IA-QA — 152 QA & Dev Tools for AI Agents MCP server?
VerifyMCP ran 13 checks on IA-QA — 152 QA & Dev Tools for AI Agents, covering TLS certificate, Authorization, HTTPS, HSTS header and 9 more. Supply Chain Security and Provenance & Transparency do not apply to a hosted endpoint, so they were not tested. The score does not measure everything, and the scoring documentation lists what it leaves out.