IA-QA — 152 QA & Dev Tools for AI Agents
REMOTE · WWW.IA-QA.COM
152 free QA tools for agents: injection scan, evals, RAG, web audits. Smaller packs: /mcp/{pack}.
Available components
Last scanned (7 hours ago)
Scores higher than 97% of remote servers
Reachable. Last reached 5 Nov 2026.
Category Testing and QA
How this component scores in each security and reliability category. Every signal is checked automatically against the live server, and we only credit what we can confirm. How we score → Why this is hard to score →
Endpoint Security83
- The endpoint's TLS certificate is valid, in date, and uses a strong key. View diagnostics → Pass
- No authorisation is required to call this server. Nothing on it is marked destructive, but we can't show that an anonymous caller can only read: full credit needs the server to list at least one tool and every tool to be verifiably read-only. We use a best-effort check of tool names and descriptions to tell read-only tools from tools that change things, so some genuinely read-only tools may not get full credit. Names that start with a read verb (get, list, read, search and similar) and accurate readOnlyHint annotations help. Meet that to earn full credit, or require authorisation. See how to fix → View diagnostics → Partial
- HTTPS is enforced; there's no plaintext access path. View diagnostics → Pass
- The HSTS (Strict-Transport-Security) header is present. View diagnostics → Pass
- DNSSEC is configured correctly; the domain's records validate against the full chain to the root. View diagnostics → Pass
Transport & Reachability100
- Verified streamable-http transport via a live MCP handshake. View diagnostics → Pass
Schema Quality & AI Usability86
- AI-judged instruction clarity (excellent).Pass
- Context-footprint check failed: tool/resource definitions use about 22973 tokens (~151/item across 152 items; 152 tools + 0 resources), over budget; trim descriptions and params. See how to fix → Fail
- Tools include usage examples.Pass
Stability & Change Management100
- Stability check failed: schema churn in the 30 days we've observed: 0 tool removals, 1 breaking changes, 0 auth/transport breaks, 0 additions. See how to fix → Fail
Tool Coverage99
- 100% of tools have a non-trivial description (not blank, and not just the tool's name).Pass
- 97% of tool parameters carry a description.Partial
- Structured output schemas are declared (100% of tools); any adoption earns full credit.Pass
Tool Safety75
- No prompt-injection markers were found in the server instructions, tool names or descriptions we captured.Pass
- 0 of 5 tool(s) whose name or description implies an irreversible operation declare an MCP destructiveHint annotation; "run_eval_contract" implies "eval" and declares readOnlyHint instead, contradicting what its own name says it does. See how to fix → Fail
- An AI judge read all 153 captured unit(s) of tool text and found none that tries to manipulate the model reading it.Pass
Capabilities100
- Implements a current MCP spec version (2026-07-28).Pass
Checks we ran (13)
- TLS certificate
- Authorization
- HTTPS
- HSTS header
- DNSSEC
- Transport
- Schema quality
- Stability
- Tool coverage
- Injection markers
- Destructive annotations
- Judged manipulation
- MCP protocol
Does not apply to this hosted endpoint (2)
- Supply Chain SecurityScored on package channels only. This is a hosted endpoint, so there is nothing to install or inspect.
- Provenance & TransparencyScored on package channels only. This is a hosted endpoint, so there is nothing to install or inspect.
The score does not measure everything. What the score does not measure →
Every change we have recorded for this component, newest first. Security-relevant changes are always shown. ▲ marks a change for the better, ▼ a change for the worse; unmarked changes are neutral.
- 5 Nov 26 0
- The server rewrote its instructions, which are the text every model session reads security
- Tool “cors_test” rewrote its description, which is the text the model reads security
- Tool “mcp_server_evaluate” rewrote its description, which is the text the model reads security
- MCP protocol: Implements a current MCP spec version (2026-07-28). functional
- MCP protocol version: 2025-11-25 → 2026-07-28 functional
- “mcp_server_evaluate” added an optional parameter “test_tool_args” cosmetic
- “mcp_server_evaluate” reworded the description of “test_tool_name” cosmetic
- 4 Nov 26 0
- Tool “find_tool” rewrote its description, which is the text the model reads security
- Tool “rate_tool” rewrote its description, which is the text the model reads security
- Tool “run_eval_contract” rewrote its description, which is the text the model reads security
- “run_eval_contract” reworded the description of “inline_contract” cosmetic
- 3 Nov 26 0
- Tool “compare_models” rewrote its description, which is the text the model reads security
- Tool “estimate_llm_cost” rewrote its description, which is the text the model reads security
- Tool “llm_fit_finder” rewrote its description, which is the text the model reads security
- Tool “llm_generate” rewrote its description, which is the text the model reads security
- Tool “token_budget_calculator” rewrote its description, which is the text the model reads security
- “compare_models” reworded the description of “models” cosmetic
- “context_window_check” reworded the description of “model” cosmetic
- “llm_fit_finder” reworded the description of “top_n” cosmetic
- “llm_fit_finder” reworded the description of “use_case” cosmetic
- “llm_generate” reworded the description of “model” cosmetic
- “shield_analyze” reworded the description of “model” cosmetic
- “token_budget_calculator” reworded the description of “model” cosmetic
- “compare_models” dropped the optional parameter “use_case” cosmetic
- 1 Nov 26 +2
- Stability: pass → fail ▼ security
- A breaking change shipped without a version bump: still 1.0.0 ▼ security
- The server rewrote its instructions, which are the text every model session reads security
- Tool “detect_secrets” rewrote its description, which is the text the model reads security
- Tool “generate_ci_workflow” rewrote its description, which is the text the model reads security
- Tool “jira_to_test_suite” rewrote its description, which is the text the model reads security
- Tool “mcp_schema_lint” rewrote its description, which is the text the model reads security
- Tool “metamorphic_check” rewrote its description, which is the text the model reads security
- Tool “multimodal_eval_guide” rewrote its description, which is the text the model reads security
- Tool “prompt_injection_scan” rewrote its description, which is the text the model reads security
- Tool “rerank_evaluate” rewrote its description, which is the text the model reads security
- Tool “run_eval_contract” rewrote its description, which is the text the model reads security
- Tool “run_vlm_test_suite” rewrote its description, which is the text the model reads security
- Tool “run_vlm_test_suite_batch” rewrote its description, which is the text the model reads security
- Tool “secret_scan” rewrote its description, which is the text the model reads security
- Tool “shield_analyze” rewrote its description, which is the text the model reads security
- Tool “test_skill” rewrote its description, which is the text the model reads security
- Tool “validate_agent_trajectory” rewrote its description, which is the text the model reads security
- “secret_scan” changed the type of “input”: string → string|array ▼ functional
- Schema quality: fail → pass ▲ functional
- “hallucination_check” added an optional parameter “cases” cosmetic
- “json_schema_generate” added an optional parameter “samples” cosmetic
- “generate_ci_workflow” reworded the description of “app_base_url” cosmetic
- “generate_ci_workflow” reworded the description of “app_start_command” cosmetic
- “generate_ci_workflow” reworded the description of “cli_tools” cosmetic
- “generate_ci_workflow” reworded the description of “contract_path” cosmetic
- “generate_ci_workflow” reworded the description of “cron” cosmetic
- “generate_ci_workflow” reworded the description of “fail_on_fail” cosmetic
- “generate_ci_workflow” reworded the description of “gate” cosmetic
- “generate_ci_workflow” reworded the description of “min_score” cosmetic
- “generate_ci_workflow” reworded the description of “node_version” cosmetic
- “generate_ci_workflow” reworded the description of “provider” cosmetic
- “generate_ci_workflow” reworded the description of “triggers” cosmetic
- “generate_ci_workflow” reworded the description of “workflow_name” cosmetic
- “hallucination_check” reworded the description of “answer” cosmetic
- “hallucination_check” reworded the description of “context” cosmetic
- “hallucination_check” reworded the description of “strict” cosmetic
- “jira_to_test_suite” reworded the description of “api_key” cosmetic
- “jira_to_test_suite” reworded the description of “confluence_pages” cosmetic
- “jira_to_test_suite” reworded the description of “issue” cosmetic
- “jira_to_test_suite” reworded the description of “issue_key” cosmetic
- “jira_to_test_suite” reworded the description of “jira_base_url” cosmetic
- “jira_to_test_suite” reworded the description of “jira_email” cosmetic
- “jira_to_test_suite” reworded the description of “jira_token” cosmetic
- “jira_to_test_suite” reworded the description of “max_tokens” cosmetic
- “jira_to_test_suite” reworded the description of “model” cosmetic
- “json_schema_generate” reworded the description of “required_all” cosmetic
- “metamorphic_check” reworded the description of “api_key” cosmetic
- “metamorphic_check” reworded the description of “base” cosmetic
- “metamorphic_check” reworded the description of “baseline_guard” cosmetic
- “metamorphic_check” reworded the description of “mode” cosmetic
- “metamorphic_check” reworded the description of “require_all” cosmetic
- “metamorphic_check” reworded the description of “thresholds” cosmetic
- “metamorphic_check” reworded the description of “variants” cosmetic
- “multimodal_eval_guide” reworded the description of “action” cosmetic
- “multimodal_eval_guide” reworded the description of “api_key” cosmetic
- “multimodal_eval_guide” reworded the description of “clip” cosmetic
- “multimodal_eval_guide” reworded the description of “fid” cosmetic
- “multimodal_eval_guide” reworded the description of “generated_images” cosmetic
- “multimodal_eval_guide” reworded the description of “image_base64” cosmetic
- “multimodal_eval_guide” reworded the description of “image_mime_type” cosmetic
- “multimodal_eval_guide” reworded the description of “image_url” cosmetic
- “multimodal_eval_guide” reworded the description of “metric” cosmetic
- “multimodal_eval_guide” reworded the description of “model” cosmetic
- “multimodal_eval_guide” reworded the description of “real_images” cosmetic
- “multimodal_eval_guide” reworded the description of “score” cosmetic
- “multimodal_eval_guide” reworded the description of “system_prompt” cosmetic
- “multimodal_eval_guide” reworded the description of “test_cases” cosmetic
- “multimodal_eval_guide” reworded the description of “text” cosmetic
- “multimodal_eval_guide” reworded the description of “vqa” cosmetic
- “rerank_evaluate” reworded the description of “api_key” cosmetic
- “rerank_evaluate” reworded the description of “passages” cosmetic
- “rerank_evaluate” reworded the description of “query” cosmetic
- “rerank_evaluate” reworded the description of “threshold” cosmetic
- “rerank_evaluate” reworded the description of “top_k” cosmetic
- “response_quality_score” reworded the description of “expected_keywords” cosmetic
- “run_eval_contract” reworded the description of “api_keys” cosmetic
- “run_eval_contract” reworded the description of “contract_path” cosmetic
- “run_eval_contract” reworded the description of “inline_contract” cosmetic
- “run_vlm_test_suite” reworded the description of “api_key” cosmetic
- “run_vlm_test_suite” reworded the description of “image_base64” cosmetic
- “run_vlm_test_suite” reworded the description of “image_mime_type” cosmetic
- “run_vlm_test_suite” reworded the description of “image_url” cosmetic
- “run_vlm_test_suite” reworded the description of “model” cosmetic
- “run_vlm_test_suite” reworded the description of “system_prompt” cosmetic
- “run_vlm_test_suite” reworded the description of “test_cases” cosmetic
- “run_vlm_test_suite” reworded the description of “threshold” cosmetic
- “run_vlm_test_suite_batch” reworded the description of “api_keys” cosmetic
- “run_vlm_test_suite_batch” reworded the description of “image_base64” cosmetic
- “run_vlm_test_suite_batch” reworded the description of “image_mime_type” cosmetic
- “run_vlm_test_suite_batch” reworded the description of “image_url” cosmetic
- “run_vlm_test_suite_batch” reworded the description of “models” cosmetic
- “run_vlm_test_suite_batch” reworded the description of “system_prompt” cosmetic
- “run_vlm_test_suite_batch” reworded the description of “test_cases” cosmetic
- “run_vlm_test_suite_batch” reworded the description of “threshold” cosmetic
- “secret_scan” reworded the description of “input” cosmetic
- “secret_scan” reworded the description of “types” cosmetic
- “shield_analyze” reworded the description of “model” cosmetic
- “shield_analyze” reworded the description of “prompt” cosmetic
- “shield_analyze” reworded the description of “response” cosmetic
- “shield_analyze” reworded the description of “rules” cosmetic
- “shield_analyze” reworded the description of “source” cosmetic
- “test_skill” reworded the description of “api_key” cosmetic
- “test_skill” reworded the description of “dir_name” cosmetic
- “test_skill” reworded the description of “lint_only” cosmetic
- “test_skill” reworded the description of “model” cosmetic
- “test_skill” reworded the description of “other_skills” cosmetic
- “test_skill” reworded the description of “scenario_count” cosmetic
- “test_skill” reworded the description of “scenarios” cosmetic
- “validate_agent_trajectory” reworded the description of “assertions” cosmetic
- “validate_agent_trajectory” reworded the description of “format” cosmetic
- “validate_agent_trajectory” reworded the description of “trace” cosmetic
- “hallucination_check” made “answer” optional cosmetic
- “hallucination_check” made “context” optional cosmetic
- 29 Oct 26 0
- Tool “llm_generate” rewrote its description, which is the text the model reads security
- “llm_generate” reworded the description of “model” cosmetic
- 28 Oct 26 0
- Tool “ab_test_report” rewrote its description, which is the text the model reads security
- Tool “function_call_validate” rewrote its description, which is the text the model reads security
- Tool “json_schema_validate” rewrote its description, which is the text the model reads security
- Tool “prompt_injection_scan” rewrote its description, which is the text the model reads security
- Tool “rate_tool” rewrote its description, which is the text the model reads security
- Tool “redact_pii” rewrote its description, which is the text the model reads security
- Tool “toxicity_scan” rewrote its description, which is the text the model reads security
- Tool “validate_agent_trajectory” rewrote its description, which is the text the model reads security
- “function_call_validate” reworded the description of “allowed_functions” cosmetic
- “redact_pii” reworded the description of “types” cosmetic
- 26 Oct 26 0
- “compare_models” reworded the description of “models” cosmetic
- “jira_to_test_suite” reworded the description of “model” cosmetic
- “llm_generate” reworded the description of “model” cosmetic
- “model_info” reworded the description of “model” cosmetic
- “token_budget_calculator” reworded the description of “model” cosmetic
5 cosmetic changes on this day. Switch on “Show cosmetic changes” to see them.
- 24 Oct 26 0
- Tool “mcp_server_evaluate” rewrote its description, which is the text the model reads security
- Tool “mcp_server_health_check” rewrote its description, which is the text the model reads security
- “mcp_server_evaluate” reworded the description of “test_tool_name” cosmetic
- “mcp_server_evaluate” reworded the description of “url” cosmetic
- “mcp_server_health_check” reworded the description of “manifest” cosmetic
- “mcp_server_health_check” reworded the description of “strict” cosmetic
Diagnostic detail from the automated scan of this channel: what the scanner observed at each step, so you can see exactly where a check passed or failed. It is informational only and never changes the trust score.
Captured 5 Nov 2026 · Probed https://www.ia-qa.com/mcp
TLS valid
Negotiated TLS 1.3 with TLS_AES_128_GCM_SHA256 .
| Subject | Issuer | Valid from | Valid until | Key | Signature | Serial |
|---|---|---|---|---|---|---|
| CN=www.ia-qa.com | CN=YR2,O=Let's Encrypt,C=US | 10 Oct 2026 | 8 Jan 2027 | RSA 2048 | SHA256-RSA | 660e1d6981514e1320131b13dbf4a910775 |
| SANs: www.ia-qa.com | ||||||
| CN=YR2,O=Let's Encrypt,C=US (CA) | CN=Root YR,O=ISRG,C=US | 3 Sept 2025 | 2 Sept 2028 | RSA 2048 | SHA256-RSA | 4ebd24947e24d394802d84a52fd5b319 |
| CN=Root YR,O=ISRG,C=US (CA) | CN=ISRG Root X1,O=Internet Security Research Group,C=US | 13 May 2026 | 2 Sept 2032 | RSA 4096 | SHA256-RSA | f24b6d17f9d9ad7cb1c9fea78782699f |
Background: What to check on a remote MCP endpoint →
DNSSEC secure
Validation of www.ia-qa.com. — Secure
| Zone | DS | Keys | Algorithms | Outcome |
|---|---|---|---|---|
| . | Trust anchor | 20326, 38696 | 8, 8 | Verified |
| com. | DS published | 19718 | 13 | Verified |
| ia-qa.com. | DS published | 52852 | 8 | Verified |
| www.ia-qa.com. | Verified address RRset verified with the apex keys |
Authentication No authorisation required
The endpoint answered without asking for a token. Anyone who knows the URL can reach it.
| Result | No authorisation required |
|---|---|
| HTTP status | 200 |
| Header | Value |
|---|---|
| strict-transport-security | max-age=63072000; includeSubDomains; preload |
| content-security-policy | default-src 'self';script-src 'self' 'unsafe-inline' 'unsafe-eval' https://www.googletagmanager.com;style-src 'self' 'unsafe-inline' https://fonts.googleapis.com;img-src 'self' data: blob: https:;font-src 'self' https://fonts.gstatic.com data:;connect-src 'self' https: wss: http://localhost:11434 data:;media-src 'self' blob:;object-src 'none';frame-ancestors 'self';base-uri 'self';form-action 'self';script-src-attr 'none';upgrade-insecure-requests |
| x-content-type-options | nosniff |
| x-frame-options | SAMEORIGIN |
| referrer-policy | no-referrer |
| permissions-policy | camera=(), microphone=(), geolocation=(), payment=(), usb=(), display-capture=() |
Background: How OAuth 2.1 works in the 2026 MCP spec →
Transports 2 probes
| Transport | URL | Outcome | Status | Location |
|---|---|---|---|---|
| streamable-http | https://www.ia-qa.com/mcp | Verified | 200 | |
| http (plaintext) | http://www.ia-qa.com/mcp | HTTPS enforced | 301 | https://www.ia-qa.com/mcp |
The tools this component advertises to a client, with an estimated token cost for each. Expand a tool to see its parameters and schema. The per-tool counts are indicative and are not scored directly; the schema's total context footprint is one signal in Schema Quality & AI Usability. A tool's description is untrusted text the model reads on every call, which is what makes this list a security surface and not just an inventory: how tool poisoning works →
find_tool ~194
Search available MCP tools by keyword or category before calling them. Returns matching tool names, descriptions, and optionally their inputSchemas. Queries are logged 90 days, screened for credentials and personal data. Categories: data, encoding, text, llm, qa, rag, dev, security, web.
| Name | Type | Req | Description |
|---|---|---|---|
| category | string | – | Optional: filter by category — data | encoding | text | llm | qa | rag | dev | security | web |
| max_results | number | – | Maximum tools to return (default 10, max 50). Results are ranked by IDF-weighted relevance, so common words like "test" do not inflate the list. |
| query | string | yes | Keyword(s) to search in tool name and description (e.g. "cors", "token", "vector", "json") |
| with_schema | boolean | – | Set true to include inputSchema in results (default: false) |
| Name | Type | Req | Description |
|---|---|---|---|
| category | – | – | – |
| count | number | – | – |
| hint | – | – | – |
| query | – | – | – |
| score | – | – | – |
| tool | – | – | – |
| tools | array | – | – |
| total_matches | number | – | – |
| truncated | boolean | – | – |
{"query":"leaked api key"}fix_gherkin ~290
Fix Gherkin syntax warnings from a jira_to_test_suite result. Takes the current gherkin text and the _gherkin_warnings array, calls your LLM to fix ONLY the flagged issues (adds missing Given/When/Then steps, etc.), and returns the corrected Gherkin. Lightweight — uses ~300-500 tokens vs ~5k for a full regeneration. Requires BYOK LLM key.
| Name | Type | Req | Description |
|---|---|---|---|
| api_key | string | yes | Your own LLM provider API key (BYOK) — OpenAI "sk-…", Anthropic "sk-ant-…", Google "AIzaSy…", or Groq "gsk_…". There is no server-side key for this tool: if you do not have one, do not call it and do… |
| gherkin | string | yes | The current Gherkin text from the jira_to_test_suite result (test_suite.gherkin). |
| model | string | yes | LLM model to use for the fix, e.g. "gpt-4o-mini". Must belong to the provider whose key you passed in api_key. |
| warnings | array | yes | The _gherkin_warnings array from the jira_to_test_suite result. |
| Name | Type | Req | Description |
|---|---|---|---|
| fixed_gherkin | string | – | – |
| latency_ms | number | – | – |
| model_used | string | – | – |
| remaining_warnings | array | – | – |
| warnings_after | number | – | – |
| warnings_before | number | – | – |
{"api_key":"\u003cyour-api-key\u003e","gherkin":"Feature: Login\n Scenario: ok\n given I open the page\n Then I see the form","model":"gpt-4o-mini","warnings":["Step keyword \"given\" should be capitalised"]}flatten_json ~112
Flatten a nested JSON object to single-level dot-notation keys (e.g. {"a":{"b":1}} → {"a.b":1}), or unflatten dot-notation keys back to a nested object. Supports custom separators.
| Name | Type | Req | Description |
|---|---|---|---|
| input | string|object|array | yes | The JSON to flatten or unflatten — a JSON string, or the object itself. |
| mode | string | – | "flatten" (default) or "unflatten" |
| separator | string | – | Key separator (default: ".") |
| Name | Type | Req | Description |
|---|---|---|---|
| key_count | number | – | – |
| max_depth | array | – | – |
| result | – | – | – |
{"input":{"a":{"b":1,"c":[1,2]}}}format_bytes ~124
Convert raw byte counts to human-readable sizes in SI (KB=1000) or IEC (KiB=1024) units, or parse size strings back to bytes. Covers B, KB/KiB, MB/MiB, GB/GiB, TB/TiB, PB/PiB.
| Name | Type | Req | Description |
|---|---|---|---|
| bytes | number | – | Number of bytes to format |
| size_string | string | – | Size string to parse to bytes (e.g. "1.5 GB", "512 MiB") |
| standard | string | – | Output standard (default: both) |
| Name | Type | Req | Description |
|---|---|---|---|
| bytes | number | – | – |
| original | – | – | – |
{"bytes":1536,"standard":"iec"}format_json ~148
Validate and pretty-print a string that is ALREADY valid JSON. Strict by design — it is a validity gate: valid JSON comes back formatted, anything else is rejected with the exact parse error. It never repairs, completes, or guesses. NOT for: plain text or prose (will fail), JSON embedded in markdown/prose (use extract_json_from_text first), JS objects (JSON.stringify them first), YAML (use yaml_to_json).
| Name | Type | Req | Description |
|---|---|---|---|
| indent | number | – | Indent size (default: 2) |
| input | string | yes | A raw JSON string, e.g. '{"key":"value"}'. Must already parse as JSON — plain text or truncated JSON is rejected, not repaired. |
| Name | Type | Req | Description |
|---|---|---|---|
| formatted | string | – | – |
| valid | boolean | – | – |
{"input":"{\"a\":1,\"b\":{\"a\":2}}"}format_table ~108
Convert a JSON array of objects into a Markdown table. Automatically detects columns, aligns headers, and fills missing keys with empty cells. Use when an agent needs to present structured data — tool results, model comparisons, test reports — as a readable table in a response or document.
| Name | Type | Req | Description |
|---|---|---|---|
| columns | array | – | Column names and order (default: all keys from first row) |
| input | string|object|array | yes | The array of objects to render — a JSON string, or the array itself. |
| Name | Type | Req | Description |
|---|---|---|---|
| columns | number | – | – |
| rows | number | – | – |
| table | string | – | – |
{"input":[{"a":1}]}function_call_validate ~186
Validate an LLM function call / tool_use output: check that function name is in allowed list, arguments match expected schema (types, enums, ranges), no extra/missing args. Reads the definitions as OpenAI (`parameters`), MCP (`inputSchema`) or Anthropic (`input_schema`) write them, or as plain `required_args`/`optional_args`; reads the call as `arguments` (object or JSON string), `input` (Anthropic tool_use) or an OpenAI `{function:{…}}` wrapper.
| Name | Type | Req | Description |
|---|---|---|---|
| allowed_functions | array | yes | List of allowed function definitions, in OpenAI, MCP or Anthropic form, or as {name, required_args, optional_args} |
| function_call | object | yes | The function call object from LLM (e.g. { "name": "get_weather", "arguments": {"city":"Paris"} }) |
| Name | Type | Req | Description |
|---|---|---|---|
| error_count | number | – | – |
| errors | array | – | – |
| function_name | – | – | – |
| provided_args | – | – | – |
| required_args | – | – | – |
| valid | boolean | – | – |
{"allowed_functions":[{"name":"get_weather","required_args":["city"]}],"function_call":{"arguments":{"city":"Paris"},"name":"get_weather"}}generate_ci_workflow ~380
Generate a ready-to-commit GitHub Actions workflow that gates a build on IA-QA: "eval_contract" runs a .ia-eval.yaml through ia-qa-com/eval-action@v1 (needs a provider key as a repo secret), "cli_checks" runs deterministic tools via npx @ia-qa/cli and fails the build on their verdict, "selector_drift" gates E2E selectors with @ia-qa/self-healing. Deterministic template, no LLM. Returns the YAML, the secrets to create and the next steps. Pair with generate_eval_yaml for the contract.
| Name | Type | Req | Description |
|---|---|---|---|
| app_base_url | string | – | [selector_drift] Default http://127.0.0.1:3000; must match .ia-qa/config.json. |
| app_start_command | string | – | [selector_drift] Boots the app, e.g. "npm run start:ci". Never guessed: omitted, the step is a TODO. |
| cli_tools | array | – | Tools to gate on, e.g. ["secret_scan","prompt_injection_scan"]. |
| contract_path | string | – | Default evals/smoke.ia-eval.yaml. |
| cron | string | – | For schedule (default '0 6 * * 1'). |
| fail_on_fail | boolean | – | Default true; false reports without gating. |
| gate | string | – | Default eval_contract. both = CLI checks + eval; all = CLI checks, drift, eval. |
| min_score | number | – | Override the contract min_score (0-100). |
| node_version | string | – | Default "20". |
| provider | string | – | Decides which repo secret is wired (default groq). |
| triggers | array | – | Default push + pull_request. |
| workflow_name | string | – | – |
| Name | Type | Req | Description |
|---|---|---|---|
| gate | string | – | – |
| next_steps | array | – | – |
| notes | array | – | – |
| path | string | – | – |
| secrets_required | array | – | – |
| yaml | string | – | – |
{}generate_curl ~154
Generate a curl command from request parameters. Supports GET/POST/PUT/DELETE, custom headers, JSON body, and form data. Useful for documentation, sharing, and debugging API calls.
| Name | Type | Req | Description |
|---|---|---|---|
| body | string | – | Raw request body string |
| body_json | object | – | JSON body (auto-adds Content-Type: application/json) |
| follow_redirects | boolean | – | Follow redirects with -L flag (default: true) |
| headers | object | – | Request headers as key-value object |
| method | string | – | HTTP method (default: GET) |
| url | string | yes | Request URL (must be http/https) |
| verbose | boolean | – | Add -v for verbose output (default: false) |
| Name | Type | Req | Description |
|---|---|---|---|
| curl | string | – | – |
| header_count | number | – | – |
| method | – | – | – |
| url | – | – | – |
{"method":"DELETE","url":"https://x.com"}generate_eval_yaml ~192
Generate a complete .ia-eval.yaml evaluation contract from a plain-language description of what your LLM should do. Uses Groq openai/gpt-oss-20b (server-side, no API key needed). Returns ready-to-run YAML for the LLM Test Runner (run_eval_contract). Picks appropriate evaluators (cosine_similarity, contains_check, hallucination_check, etc.) based on the task type.
| Name | Type | Req | Description |
|---|---|---|---|
| description | string | yes | Plain-language description of what the LLM under test should do. Be specific: describe inputs, expected behaviour, and constraints. |
| scenario_count | number | – | Number of scenarios to generate (default: 5). Covers happy path + edge cases + adversarial. |
| system_prompt | string | – | Optional system prompt of the LLM under test. Helps generate more accurate test cases. |
| task_type | string | – | Optional task type hint to guide evaluator selection. |
| Name | Type | Req | Description |
|---|---|---|---|
| model_used | string | – | – |
| scenario_count | number | – | – |
| task_type | string | – | – |
| yaml | string | – | – |
{"description":"Summarize a customer support ticket in one sentence."}generate_hmac ~117
Compute an HMAC signature for a message using a secret key. Supports SHA-256 (default), SHA-512, SHA-1, and MD5. Used for API request signing, webhook verification (GitHub, Stripe, Twilio), and JWT validation.
| Name | Type | Req | Description |
|---|---|---|---|
| algorithm | string | – | Hash algorithm: sha256 (default), sha512, sha1, md5 |
| encoding | string | – | Output encoding (default: hex) |
| message | string | yes | Message to sign |
| secret | string | yes | Secret key |
| Name | Type | Req | Description |
|---|---|---|---|
| algorithm | – | – | – |
| encoding | – | – | – |
| hmac | – | – | – |
| message_length | number | – | – |
{"algorithm":"sha256","message":"msg","secret":"k"}generate_html_report ~79
Convert a run_eval_contract() LLM Test Runner JSON result into a fully self-contained dark-themed HTML report with Pass/Fail badges, side-by-side Input/Output/Ground-Truth panels, evaluator score bars, and a radar chart. Returns the HTML as a string.
| Name | Type | Req | Description |
|---|---|---|---|
| results | object | yes | The JSON object returned by run_eval_contract() |
| Name | Type | Req | Description |
|---|---|---|---|
| html | – | – | – |
{"results":{}}generate_json_ld ~138
Generate a ready-to-paste <script type="application/ld+json"> snippet for GEO / structured data optimization. Supported types: WebSite, FAQPage, Article, Person, Organization, SoftwareApplication, HowTo.
| Name | Type | Req | Description |
|---|---|---|---|
| faq_items | array | – | For FAQPage/HowTo: array of { question, answer } objects |
| fields | object | – | Schema fields as key-value pairs (name, url, description, author, datePublished, etc.) |
| type | string | yes | Schema @type: "WebSite", "FAQPage", "Article", "Person", "Organization", "SoftwareApplication", "HowTo" |
| Name | Type | Req | Description |
|---|---|---|---|
| acceptedAnswer | object | – | – |
| name | string | – | – |
| schema | – | – | – |
| snippet | string | – | – |
{"faq_items":[{"answer":"A.","question":"Q?"}],"type":"FAQPage"}generate_password ~114
Generate a cryptographically secure random password using crypto.randomBytes. Configurable length (4–128), uppercase letters, digits, and symbols. Use when resetting user passwords, seeding test accounts, or generating API secrets.
| Name | Type | Req | Description |
|---|---|---|---|
| length | number | – | Password length (4–128, default: 16) |
| numbers | boolean | – | Include digits (default: true) |
| symbols | boolean | – | Include symbols like !@#$ (default: false) |
| uppercase | boolean | – | Include uppercase letters (default: true) |
| Name | Type | Req | Description |
|---|---|---|---|
| charset_size | number | – | – |
| length | number | – | – |
| password | – | – | – |
{"length":24,"numbers":true,"symbols":true,"uppercase":true}generate_slug ~84
Convert any string into a URL-friendly slug: lowercase, ASCII-normalized (é→e), special characters removed, spaces replaced with hyphens. Use for generating SEO-friendly URL paths, file names, or identifier keys from user-provided titles or labels.
| Name | Type | Req | Description |
|---|---|---|---|
| input | string | yes | String to slugify |
| separator | string | – | Separator character (default: "-") |
| Name | Type | Req | Description |
|---|---|---|---|
| slug | – | – | – |
{"input":"Héllo Wörld 2026"}generate_test_cases ~257
Generate a set of test cases (valid, edge, invalid, pairwise) for a given feature description. Declared constraints drive the boundaries: a length or numeric bound ("[8-64]", "min 8 chars", "at least 8 characters" — read from inputs, and from the feature prose when a sentence names exactly one field) yields the last accepted value AND the first rejected one; a format (email/url/uuid, from the type, the field name or the wording) yields malformed-value cases. A bound nobody declared is labelled as this tool's assumption, not as expected behaviour. gherkinFormat renders every case (cap 200, stated in the output) and gherkinScenarioCount lets you check it against totalCases.
| Name | Type | Req | Description |
|---|---|---|---|
| feature | string | yes | Feature or function to test. Be specific: describe inputs, expected behaviour, context. Constraints stated here ("password must be at least 8 characters") are used when the sentence names exactly one… |
| inputs | string | – | Optional: list of input parameters (one per line, e.g. "email: string [required]", "password: string [required, min 8 chars]", "age: number [18-99]") |
| Name | Type | Req | Description |
|---|---|---|---|
| feature | string | – | – |
| gherkinFormat | string | – | – |
| gherkinScenarioCount | number | – | – |
| parsedInputs | array | – | – |
| test_cases | array | – | – |
{"feature":"Login form with email and password"}generate_uuid ~80
Generate one or more cryptographically random UUID v4 identifiers. Use this when you need unique IDs for test fixtures, database records, session tokens, or any scenario requiring a guaranteed-unique string. Returns up to 100 UUIDs in one call.
| Name | Type | Req | Description |
|---|---|---|---|
| count | number | – | Number of UUIDs to generate (1–100, default: 1) |
| Name | Type | Req | Description |
|---|---|---|---|
| count | number | – | – |
| uuids | – | – | – |
{"count":5}get_testing_guidelines ~227
Query the IA-QA methodology knowledge base. Returns structured testing guidelines, assertion strategies, thresholds, best practices, and relevant MCP tools for a given topic. Call without a topic to list all available topics. Topics: llm-unit-testing, rag-pipeline, prompt-stability, prompt-ab-testing, embedding-quality, eval-framework, semantic-testing, auto-testing, security, api-testing, ci-cd, multimodal, llm-data-security, agent-observability, pro-tips, learning-paths, golden-dataset, selector-drift, qa-recipes, playbooks. Not sure where to start testing an LLM, RAG pipeline or agent? Call without a topic (or with "start-here"): it maps what you are testing to the tools to call and the output field to gate CI on. A plain question such as "how do I test my RAG" also resolves to the right topic.
| Name | Type | Req | Description |
|---|---|---|---|
| topic | string | – | The testing topic to retrieve guidelines for. Omit to get the start-here map and the full list of available topics. |
| Name | Type | Req | Description |
|---|---|---|---|
| available_topics | array | – | – |
| keywords | array | – | – |
| start_here | object | – | – |
| tip | string | – | – |
| topic | – | – | – |
| usage | string | – | – |
{"topic":"rag-pipeline"}guardrail_test ~80
Test an LLM response against a set of guardrail rules: must-include, must-not-include, max length, required format, language, forbidden patterns, and custom regex. Returns pass/fail per rule.
| Name | Type | Req | Description |
|---|---|---|---|
| response | string | yes | The LLM response to test |
| rules | array | yes | Array of guardrail rules to check |
| Name | Type | Req | Description |
|---|---|---|---|
| all_passed | boolean | – | – |
| detail | string | – | – |
| failed | – | – | – |
| label | – | – | – |
| pass | boolean | – | – |
| passed | – | – | – |
| results | – | – | – |
| rule | – | – | – |
| total_rules | number | – | – |
| value | – | – | – |
{"response":"Your refund is approved.","rules":[{"type":"must_include","value":"refund"},"no_pii"]}hallucination_check ~247
Lexical hallucination check: verifies an LLM answer's words, numbers and polarity against the provided source/context. Fast, deterministic, no API key needed. Each answer sentence is aligned to its best-matching source sentence, so a number only counts as support when it sits on the SAME statement ("founded in 1998" is not grounded by "sold 1998 units"), and a negation or antonym flip against that sentence returns verdict "contradicted" — the corrupted-fact hallucination that reuses source vocabulary. Limitations: still lexical — it cannot follow a paraphrase, a synonym, or multi-sentence reasoning, so a "well_grounded" verdict means "nothing lexical found", never "verified true". For entailment use run_semantic_tests (NLI/embedding) or a calibrated judge.
| Name | Type | Req | Description |
|---|---|---|---|
| answer | string | – | The LLM answer to verify (with context). |
| cases | array | – | Batch instead of answer + context: one result per case, and the worst gate overall. |
| context | string | – | The source text that should ground it. |
| strict | boolean | – | Every sentence must be supported (default false). |
| Name | Type | Req | Description |
|---|---|---|---|
| analysis | – | – | – |
| contradictions | – | – | – |
| detail | string | – | – |
| entities | – | – | – |
| grounded | boolean | – | – |
| grounded_count | number | – | – |
| grounding_score | – | – | – |
| matched_words | number | – | – |
| message | string | – | – |
| numbers | – | – | – |
| overlap | number | – | – |
| sentence | – | – | – |
| total_sentences | number | – | – |
| total_words | number | – | – |
| unbound_claims | – | – | – |
| ungrounded_count | number | – | – |
| unsupported_claims | – | – | – |
| verdict | string | – | – |
{"answer":"Le remboursement prend plus de 7 jours ouvrés.","context":"Le remboursement prend plus de 7 jours ouvrés."}hash_text ~102
Compute a cryptographic hash of a text string. Use when you need to verify data integrity, generate content fingerprints, hash passwords (prefer SHA-256+), or produce a fixed-length digest of any input. Supports SHA-256 (default), SHA-512, SHA-1, and MD5.
| Name | Type | Req | Description |
|---|---|---|---|
| algorithm | string | – | Hash algorithm: sha256 (default), sha512, sha1, md5 |
| input | string | yes | Text to hash |
| Name | Type | Req | Description |
|---|---|---|---|
| algorithm | – | – | – |
| hash | – | – | – |
| input_length | number | – | – |
{"algorithm":"md5","input":"héllo wörld 🌍"}html_to_markdown ~81
Convert HTML to clean Markdown. Strips scripts, styles, nav, ads, and comments. Converts headings, lists, links, images, code blocks. Ideal for preparing web content as LLM context.
| Name | Type | Req | Description |
|---|---|---|---|
| input | string | yes | HTML string to convert |
| strip_links | boolean | – | Strip link URLs, keep text only (default: false) |
| Name | Type | Req | Description |
|---|---|---|---|
| markdown | – | – | – |
| markdown_length | number | – | – |
| original_length | number | – | – |
{"input":"\u003ch1\u003eT\u003c/h1\u003e\u003cp\u003ea \u003cstrong\u003eb\u003c/strong\u003e \u003ca href=\"https://x.com\"\u003ec\u003c/a\u003e\u003c/p\u003e\u003cul\u003e\u003cli\u003ei\u003c/li\u003e\u003c/ul\u003e"}http_status_lookup ~142
Look up detailed information about any HTTP status code: class, name, description, cacheability, typical causes, and handling best practices. Covers every code in the IANA HTTP Status Code Registry (1xx-5xx, including 226, 425, 451, 508, 511 and the WebDAV codes) with its defining RFC; anything outside the registry is reported as `registered: false` rather than described. `cacheable` means heuristically cacheable by default per RFC 9110 §15.1.
| Name | Type | Req | Description |
|---|---|---|---|
| code | number | yes | HTTP status code (e.g. 200, 404, 429, 503) |
| Name | Type | Req | Description |
|---|---|---|---|
| cacheable | – | – | – |
| class | – | – | – |
| code | – | – | – |
| desc | string | – | – |
| description | – | – | – |
| name | string | – | – |
| registered | boolean | – | – |
{"code":418}identify_caller ~71
Returns what the server knows about the current MCP client: clientInfo captured during initialize, User-Agent, and any _meta fields sent with this request. Useful for debugging caller identification.
| Name | Type | Req | Description |
|---|---|---|---|
| _meta | object | – | Optional self-identification. Keys: agent (string), model (string), version (string). |
| Name | Type | Req | Description |
|---|---|---|---|
| effective_agent | string | – | – |
| meta_override | object | – | – |
| note | string | – | – |
| session | object | – | – |
{}jira_to_test_suite ~267
Turn a Jira ticket into a test suite: Gherkin scenarios, E2E steps, API tests, test data and the ambiguities to clarify. Pass the issue object, or Jira credentials to fetch it. It never invents: a thin ticket returns an empty suite with verdict INCOMPLETE_SPEC and the questions to ask. test_suite._gherkin_warnings is a deterministic syntax check. BYOK: your LLM key.
| Name | Type | Req | Description |
|---|---|---|---|
| api_key | string | yes | Your LLM provider key. |
| confluence_pages | array | – | Pages from fetch_confluence_page, as documentation. |
| issue | object | – | The ticket: key, summary, description, and optionally status, issue_type, priority, labels, comments (fetch_jira_issue returns this shape). |
| issue_key | string | – | e.g. "PROJ-123", to fetch with the jira_* fields. |
| jira_base_url | string | – | – |
| jira_email | string | – | – |
| jira_token | string | – | – |
| max_tokens | integer | – | Default 8192. |
| model | string | yes | e.g. "gpt-4o-mini", "claude-haiku-4-5-20251001", "gemini-3.5-flash". |
| Name | Type | Req | Description |
|---|---|---|---|
| issue_key | string | – | – |
| issue_url | string | – | – |
| latency_ms | number | – | – |
| model_used | string | – | – |
| summary | string | – | – |
| test_suite | object | – | – |
| tokens_used | number | – | – |
{"api_key":"\u003cyour-api-key\u003e","issue":{"description":"A registered user can request a reset link; the link expires after 30 minutes.","key":"PROJ-1","summary":"Password reset by email"},"model":"gpt-4o-mini"}json_diff ~187
Compute a deep structural diff between two JSON values. Returns added, removed, and changed keys with dot-notation paths. Like git diff but for JSON objects — perfect for API response regression testing. Arrays are not compared blindly by position: the same elements in a different order collapse to a single "reordered" change, and an array of records sharing a stable identity field (id, uuid, key, name…) is matched by that field, so paths read [id=42] and a moved record is not reported as N rewrites.
| Name | Type | Req | Description |
|---|---|---|---|
| after | string|object|array | yes | The modified JSON (after) — a JSON string, or the value itself. |
| before | string|object|array | yes | The original JSON (before) — a JSON string, or the value itself. |
| max_depth | number | – | Max nesting depth to recurse (default: 10) |
| Name | Type | Req | Description |
|---|---|---|---|
| added | boolean | – | – |
| changes | array | – | – |
| identical | boolean | – | – |
| modified | boolean | – | – |
| removed | boolean | – | – |
| total_changes | number | – | – |
{"after":{"a":1,"b":2},"before":{"a":1,"b":2}}json_schema_generate ~182
Infer a JSON Schema (draft-07) from a sample JSON value. Detects types, required fields, array item shapes, nested objects, and common string formats (email, uri, date, date-time, uuid). Returns a ready-to-use schema compatible with json_schema_validate. Use when you have a sample API response or LLM output and want to auto-generate a validation schema for CI/CD testing.
| Name | Type | Req | Description |
|---|---|---|---|
| input | string|object|array | yes | The sample JSON value to infer the schema from — a JSON string, or the value itself. |
| required_all | boolean | – | Mark fields present in every observed object as required (default: true) |
| samples | array|string | – | More samples of the same shape, merged with input: a field seen as null and as a date becomes ["string","null"]; one missing somewhere is not required. |
| Name | Type | Req | Description |
|---|---|---|---|
| schema | object | – | – |
| warnings | array | – | – |
{"input":{"a":[1],"b":true,"f":1.5,"n":1,"o":{"k":null},"s":"x"}}json_schema_validate ~192
Validate a JSON value against a JSON Schema (draft-07 subset). Supports type, required, properties, items, enum, const, pattern, format (email/uri/date/date-time/uuid), minimum/maximum, minLength/maxLength, minItems/maxItems, uniqueItems, additionalProperties, anyOf, allOf, oneOf, not, and local $ref (#/definitions/…, #/$defs/…). A schema using a keyword outside that list (patternProperties, if/then, dependencies, a remote $ref…) is never reported valid: the keyword is named in `unevaluated_keywords`. Returns all validation errors with dot-notation paths.
| Name | Type | Req | Description |
|---|---|---|---|
| schema | string|object|array | yes | The JSON Schema — a JSON string, or the schema object itself. |
| value | string|object|array | yes | The JSON value to validate — a JSON string, or the value itself. |
| Name | Type | Req | Description |
|---|---|---|---|
| error_count | number | – | – |
| errors | array | – | – |
| valid | boolean | – | – |
{"schema":{"$defs":{"P":{"properties":{"n":{"type":"string"}},"type":"object"}},"properties":{"p":{"$ref":"#/$defs/P"}},"type":"object"},"value":{"p":{"n":"five"}}}json_to_csv ~90
Convert a JSON array of objects to CSV format. Automatically detects columns from all object keys. Handles quoting and escaping per RFC 4180.
| Name | Type | Req | Description |
|---|---|---|---|
| delimiter | string | – | Column delimiter (default: ",") |
| headers | boolean | – | Include header row (default: true) |
| input | string|object|array | yes | The array of objects to convert — a JSON string, or the array itself. |
| Name | Type | Req | Description |
|---|---|---|---|
| column_names | – | – | – |
| columns | number | – | – |
| csv | string | – | – |
| rows | number | – | – |
{"input":[{"a":1},{"b":2}]}json_to_yaml ~80
Convert a JSON object to clean, human-readable YAML. Handles nested objects, arrays, multiline strings, and special characters. No external dependencies.
| Name | Type | Req | Description |
|---|---|---|---|
| indent | number | – | Indentation size in spaces (default: 2) |
| input | string|object|array | yes | The JSON to convert to YAML — a JSON string, or the value itself. |
| Name | Type | Req | Description |
|---|---|---|---|
| lines | number | – | – |
| yaml | – | – | – |
{"input":{"a":"x: y","c":null,"d":"yes","e":"0123"}}latency_benchmark ~110
Measure response time of one or more HTTP endpoints (GET/POST). Runs N iterations and returns min/max/avg/p95 latency. Useful for API and MCP server benchmarking.
| Name | Type | Req | Description |
|---|---|---|---|
| endpoints | string|array | yes | Endpoints to benchmark. Accepts a single URL string, an array of URL strings, or an array of {url, method?, body?, headers?, label?} objects. |
| iterations | number | – | Number of iterations per endpoint (default: 3, max: 10) |
| Name | Type | Req | Description |
|---|---|---|---|
| iterations | number | – | – |
| results | array | – | – |
{"endpoints":"https://www.ia-qa.com/mcp/ping","iterations":3}levenshtein_distance ~109
Compute the Levenshtein (edit) distance and normalized similarity ratio between two strings. Supports batch comparison. Useful for fuzzy string matching, deduplication, and test result comparison.
| Name | Type | Req | Description |
|---|---|---|---|
| a | string | – | First string (single-pair mode) |
| b | string | – | Second string (single-pair mode) |
| batch | array | – | Batch of {a,b} pairs (max 50) |
| case_insensitive | boolean | – | Ignore case differences (default: false) |
| Name | Type | Req | Description |
|---|---|---|---|
| a | – | – | – |
| b | – | – | – |
| count | number | – | – |
| distance | – | – | – |
| mode | string | – | – |
| operations_needed | – | – | – |
| results | – | – | – |
| similarity | string | – | – |
{"a":"kitten","b":"sitting"}lint_commit_message ~91
Validate a git commit message against the Conventional Commits spec (feat, fix, docs, style, refactor, test, chore, ci, perf, build). Returns compliance score, breaking change detection, and actionable suggestions.
| Name | Type | Req | Description |
|---|---|---|---|
| message | string | yes | Git commit message to validate |
| strict | boolean | – | Enforce strict rules: max 72-char subject, imperative mood check (default: false) |
| Name | Type | Req | Description |
|---|---|---|---|
| checks | – | – | – |
| has_body | boolean | – | – |
| is_breaking_change | boolean | – | – |
| scope | – | – | – |
| score | number | – | – |
| subject | – | – | – |
| type | – | – | – |
| valid | boolean | – | – |
{"message":"feat!: drop node 18"}list_llm_models ~137
List all LLM models available on ia-qa.com with their provider, API endpoint, and capabilities. Filter by provider name (e.g. "Groq", "HuggingFace", "OpenAI") or return the full catalog. Use this to discover which models are available before calling an LLM API, or to compare providers.
| Name | Type | Req | Description |
|---|---|---|---|
| provider | string | – | Filter by provider name (case-insensitive). E.g. "Groq", "HuggingFace", "OpenAI", "Anthropic", "Google", "DeepSeek", "xAI", "Ollama". Omit for full catalog. |
| Name | Type | Req | Description |
|---|---|---|---|
| filter | – | – | – |
| models | – | – | – |
| providers | – | – | – |
| total | number | – | – |
{"provider":"groq"}list_local_tests ~70
Discover .ia-eval.yaml LLM test suite files in the project directory. Scans CWD and standard sub-directories (evals/, tests/, contracts/). Returns file paths ready to pass to run_eval_contract.
| Name | Type | Req | Description |
|---|---|---|---|
| dir | string | – | Directory to scan (defaults to server CWD) |
| Name | Type | Req | Description |
|---|---|---|---|
| count | number | – | – |
| dir | string | – | – |
| files | array | – | – |
{}llm_fit_finder ~274
Shortlist LLMs on facts, no ranking: cloud models with the features you need within a monthly budget, cheapest first; or local models that fit your VRAM, largest first, with the ollama command. To pick one, run your cases on the shortlist (prompt_test_suite, run_eval_contract).
| Name | Type | Req | Description |
|---|---|---|---|
| features | array | – | Required features: vision, function_calling, json_mode, streaming, reasoning |
| max_budget | number | – | Maximum monthly budget in USD (based on tokens_per_day) |
| mode | string | – | cloud (API models) or local (Ollama/self-hosted). Default: cloud |
| quantization | string | – | Quantization (only for mode=local): Q4_K_M | Q8_0 | FP16. Default: Q4_K_M |
| tokens_per_day | number | – | Estimated daily token volume (default: 100000) |
| top_n | number | – | Number of models to return (default: 5) |
| use_case | string | – | Only "reasoning" filters (requires the feature); chatbot | code | rag | summarization | classification | agents | multilingual rank nothing |
| vram_gb | number | – | GPU VRAM in GB (only for mode=local). Default: 16 |
| Name | Type | Req | Description |
|---|---|---|---|
| mode | string | – | – |
| note | string | – | – |
| order | string | – | – |
| prices_as_of | – | – | – |
| quantization | – | – | – |
| results | array | – | – |
| tokens_per_day | – | – | – |
| total_matching | number | – | – |
| use_case | – | – | – |
| vram_gb | – | – | – |
{"features":["vision"],"top_n":5,"use_case":"chatbot"}llm_format_check ~92
Validate that an LLM output matches an expected format: JSON, Markdown, code block, bullet list, numbered list, table, YAML, XML, or custom regex. Essential for structured output testing.
| Name | Type | Req | Description |
|---|---|---|---|
| expected_format | string | yes | Expected format |
| output | string | yes | The LLM output to validate |
| regex_pattern | string | – | Custom regex pattern (only when expected_format is "regex") |
| Name | Type | Req | Description |
|---|---|---|---|
| checks | – | – | – |
| expected_format | – | – | – |
| failed | – | – | – |
| passed | – | – | – |
| total_checks | number | – | – |
| valid | boolean | – | – |
{"expected_format":"yaml","output":"a: 1\nb: [1, 2]"}llm_generate ~275
Generate text with an open-source model hosted on Groq, on the server's key — no API key needed. Models: openai/gpt-oss-20b, openai/gpt-oss-120b, qwen/qwen3.8-27b (default "openai/gpt-oss-20b"). HuggingFace models are not served on the server key. A reasoning model that spends max_tokens thinking is retried once at 4096. Rate-limited.
| Name | Type | Req | Description |
|---|---|---|---|
| max_tokens | number | – | Maximum tokens to generate (default: 2048, max: 4096) |
| model | string | – | Model ID (default: "openai/gpt-oss-20b"). Server-keyed whitelist only — Groq: openai/gpt-oss-20b, openai/gpt-oss-120b, qwen/qwen3.8-27b. Other ids from list_llm_models, HuggingFace ones included, are… |
| prompt | string | yes | The user prompt / instruction to send to the model |
| system | string | – | Optional system prompt to set context or persona |
| temperature | number | – | Sampling temperature 0.0–1.5 (default: 0.7) |
| Name | Type | Req | Description |
|---|---|---|---|
| content | – | – | – |
| latency_ms | number | – | – |
| model | – | – | – |
| provider | – | – | – |
| usage | – | – | – |
{"prompt":"Reply with the single word OK."}llm_json_schema_check ~85
Validate that an LLM JSON output matches a JSON Schema definition. Tests required fields, types, enums, nested objects, and arrays. Critical for function-calling and structured output testing.
| Name | Type | Req | Description |
|---|---|---|---|
| output | string | yes | The LLM JSON output (raw string, will be parsed) |
| schema | object | yes | JSON Schema (draft-07 subset) to validate against |
| Name | Type | Req | Description |
|---|---|---|---|
| error_count | number | – | – |
| errors | array | – | – |
| parse_error | – | – | – |
| parsed_type | – | – | – |
| valid | boolean | – | – |
{"output":"{\"name\":\"a\",\"age\":3}","schema":{"properties":{"age":{"minimum":0,"type":"integer"},"name":{"type":"string"},"role":{"enum":["admin","user"]}},"required":["name","age"],"type":"object"}}llm_output_validator ~240
Validate an LLM response against QA criteria: format checks (JSON, code, markdown), content rules (must-include, must-not-include), length constraints, language detection, and safety patterns. Essential for QA testing LLM-powered features.
| Name | Type | Req | Description |
|---|---|---|---|
| check_json_schema | string | – | If expected_format is JSON, provide required keys as comma-separated list to validate the structure |
| check_safety | boolean | – | Check for PII patterns (emails, phones, SSN), profanity signals, and prompt leakage |
| expected_format | string | – | Expected output format |
| expected_language | string | – | Expected language of the output (en, fr, es, de…). Checks for common words. |
| max_length | number | – | Maximum character length for the output |
| min_length | number | – | Minimum character length for the output |
| must_include | string | – | Comma-separated strings that MUST appear in the output |
| must_not_include | string | – | Comma-separated strings that must NOT appear (e.g. "TODO, FIXME, undefined, NaN") |
| output | string | yes | The LLM output text to validate |
| Name | Type | Req | Description |
|---|---|---|---|
| checks | – | – | – |
| failed | – | – | – |
| passed | – | – | – |
| total | number | – | – |
| verdict | – | – | – |
{"expected_format":"json","must_include":"a","must_not_include":"TODO","output":"{\"a\":1}"}lorem_ipsum ~120
Generate Lorem Ipsum placeholder text for UI mockups, design prototypes, or test data population. Configurable paragraphs (1–10), sentences per paragraph (1–20), and approximate words per sentence (3–30).
| Name | Type | Req | Description |
|---|---|---|---|
| paragraphs | number | – | Number of paragraphs to generate (1–10, default: 1) |
| sentences_per_paragraph | number | – | Sentences per paragraph (1–20, default: 5) |
| words_per_sentence | number | – | Approximate words per sentence (3–30, default: 10) |
| Name | Type | Req | Description |
|---|---|---|---|
| paragraph_count | number | – | – |
| paragraphs | – | – | – |
{"paragraphs":3,"sentences_per_paragraph":2}mcp_schema_lint ~92
Lint one MCP tool definition with the same rules as mcp_server_health_check: errors break the MCP spec (FAIL), documentation_issues are what its strict mode refuses — a parameter without description or type (FIX, never an A) — and suggestions are style only (snake_case) and change nothing.
| Name | Type | Req | Description |
|---|---|---|---|
| tool_definition | object | yes | MCP tool definition object with name, description, inputSchema |
| Name | Type | Req | Description |
|---|---|---|---|
| error_count | number | – | – |
| errors | – | – | – |
| grade | – | – | – |
| quality_score | – | – | – |
| warning_count | number | – | – |
| warnings | – | – | – |
{"tool_definition":{"description":"Return the current weather for a city. Use it when the user asks about the weather now.","inputSchema":{"properties":{"city":{"description":"City name, e.g. Paris","type":"string"}},"required":["city"],"type":"object"},"name":"get_weather"}}mcp_server_evaluate ~274
Check that a live MCP server over Streamable HTTP speaks the protocol, 2026-07-28 or earlier: handshake or server/discover, tools/list and its definitions, error codes, one tools/call, structuredContent against outputSchema, latency. Returns PASS/FIX/BLOCK with a 0-100 score, or INCONCLUSIVE (score null) when authorization is required. The verdict covers the PROTOCOL, never whether tools answer correctly: what_this_does_not_prove lists what was not measured. Called: only test_tool_name, or a tool annotated readOnlyHint: true. A refusal of invented arguments is not scored (tool_behaviour: refused_placeholder); pass test_tool_args to exercise the tool. Private addresses and stdio are out of reach: paste tools/list into mcp_server_health_check.
| Name | Type | Req | Description |
|---|---|---|---|
| test_tool_args | object | – | Real arguments, sent to the tested server. Without them, required ones are invented. |
| test_tool_name | string | – | Tool to call. Default: a read-only one, preferring no required arg. |
| url | string | yes | Public URL of the MCP endpoint (e.g. https://mcp.example.com/mcp). An origin alone (https://www.ia-qa.com) tries /mcp, then /. |
| Name | Type | Req | Description |
|---|---|---|---|
| checks | array | – | – |
| endpoint | string | – | – |
| latency | object|null | – | – |
| next_steps | array | – | – |
| scope | string | – | – |
| score | number|null | – | – |
| server | object | – | – |
| tool_behaviour | string | – | – |
| verdict | string | – | – |
| what_this_does_not_prove | array | – | – |
| what_this_proves | array | – | – |
{"url":"https://www.ia-qa.com/mcp"}How do I install the IA-QA — 152 QA & Dev Tools for AI Agents MCP server?
IA-QA — 152 QA & Dev Tools for AI Agents is a hosted endpoint at https://www.ia-qa.com/mcp, so there is nothing to install locally. Ready-made configuration for Claude, Cursor, VS Code, Codex and 5 more is on this page, copied from each client's own documentation.
remote · www.ia-qa.com
claude mcp add --transport http jcjamet-ia-qa-toolbox 'https://www.ia-qa.com/mcp'
What is the IA-QA — 152 QA & Dev Tools for AI Agents MCP server?
IA-QA — 152 QA & Dev Tools for AI Agents is an MCP server listed in the public MCP registry as io.github.JcJamet/ia-qa-toolbox. 152 free QA tools for agents: injection scan, evals, RAG, web audits. Smaller packs: /mcp/{pack}. This page covers its hosted endpoint (https://www.ia-qa.com/mcp).
Is the IA-QA — 152 QA & Dev Tools for AI Agents MCP server safe to use?
IA-QA — 152 QA & Dev Tools for AI Agents scores 89 out of 100 on VerifyMCP. That is a record of what we were able to check automatically, not an endorsement. The category breakdown on this page shows every signal behind the number, including the ones we could not confirm.
What tools does the IA-QA — 152 QA & Dev Tools for AI Agents MCP server expose?
IA-QA — 152 QA & Dev Tools for AI Agents exposes 152 tools: get_testing_guidelines, find_tool, generate_test_cases, jira_to_test_suite, analyze_diff_bugs, and 147 more. Their descriptions and schemas cost roughly 22,457 tokens of context every time the server is loaded.
Does the IA-QA — 152 QA & Dev Tools for AI Agents MCP server require authentication?
No. We connected to IA-QA — 152 QA & Dev Tools for AI Agents without credentials and it answered, so anything it exposes is reachable by anyone who knows the address.
Is the IA-QA — 152 QA & Dev Tools for AI Agents MCP server still maintained?
IA-QA — 152 QA & Dev Tools for AI Agents is still listed as active in the MCP registry. We last reached this channel on 5 November 2026. Those dates come from our own scans of the registry and the channel itself, not from anything the publisher announced.
What did VerifyMCP test on the IA-QA — 152 QA & Dev Tools for AI Agents MCP server?
VerifyMCP ran 13 checks on IA-QA — 152 QA & Dev Tools for AI Agents, covering TLS certificate, Authorization, HTTPS, HSTS header and 9 more. Supply Chain Security and Provenance & Transparency do not apply to a hosted endpoint, so they were not tested. The score does not measure everything, and the scoring documentation lists what it leaves out.