OpenMark
REMOTE · OPENMARK.AI
Benchmark your own prompt on 100+ LLMs with real API calls; pick the cheapest model that passes.
Available components
Last scanned (1 hour ago)
Scores higher than 46% of remote servers
Reachable. Last reached 23 Oct 2026.
Category AI and Machine Learning
How this component scores in each security and reliability category. Every signal is checked automatically against the live server, and we only credit what we can confirm. How we score → Why this is hard to score →
Endpoint Security51
- The endpoint's TLS certificate is valid, in date, and uses a strong key. View diagnostics → Pass
- Authorisation not fully verified: no authorisation is required to call this server, and 9 tool(s) never declared a destructiveHint. The MCP spec treats an absent hint as destructive by default, so we cannot call this surface safe. See how to fix → View diagnostics → Unverified
- HTTPS enforcement could not be verified: the plaintext port answered with HTTP 405, which proves neither a plaintext path nor enforcement. View diagnostics → Unverified
- The HSTS (Strict-Transport-Security) header is present. View diagnostics → Pass
- DNSSEC check failed: this domain isn't protected by DNSSEC. See how to fix → View diagnostics → Fail
Transport & Reachability100
- Verified streamable-http transport via a live MCP handshake. View diagnostics → Pass
Schema Quality & AI Usability64
- AI-judged instruction clarity (good).Pass
- Context-footprint check failed: tool/resource definitions use about 1773 tokens (~197/item across 9 items; 9 tools + 0 resources), over budget; trim descriptions and params. See how to fix → Fail
- Usage-examples check failed: none of the tools include examples. See how to fix → Fail
Stability & Change Management13
- Stability observed for 4 of 30 days with no destabilising changes; credit accrues until the full window elapses.Partial
Tool Coverage97
- 100% of tools have a non-trivial description (not blank, and not just the tool's name).Pass
- 91% of tool parameters carry a description.Partial
Tool Safety100
- No prompt-injection markers were found in the server instructions, tool names or descriptions we captured.Pass
- We read all 9 captured tool definition(s), and no name or description among them implies an irreversible operation.Pass
- An AI judge read all 10 captured unit(s) of tool text and found none that tries to manipulate the model reading it.Pass
Capabilities60
- Spec-recency check failed: implements MCP spec 2025-06-18; the latest is 2026-07-28. See how to fix → Fail
Every change we have recorded for this component, newest first. Security-relevant changes are always shown. ▲ marks a change for the better, ▼ a change for the worse; unmarked changes are neutral.
- 23 Oct 26 +1
No change was recorded against any check on this day. Stability & Change Management went from 10 to 13. That category is still filling its 30-day observation window: 3 days of observed history at the previous scan, 4 at this one. The score rises as the window fills, whether or not the server changes.
- 21 Oct 26 +1
- The server rewrote its instructions, which are the text every model session reads security
- Tool “get_task_format” rewrote its description, which is the text the model reads security
- Schema quality: 1610 → 1773 ▼ functional
- New tool “validate_task” functional
- “get_task_format” added an optional parameter “topic” cosmetic
- 20 Oct 26 −2
- The server rewrote its instructions, which are the text every model session reads security
- Tool “get_results” rewrote its description, which is the text the model reads security
- Tool “list_models” rewrote its description, which is the text the model reads security
- Tool “prepare_attachments” rewrote its description, which is the text the model reads security
- Tool “run_benchmark” rewrote its description, which is the text the model reads security
- Schema quality: pass → fail ▼ functional
- Tool coverage: 85% → 91% ▲ functional
- Stability: unverified → 0.03 ▲ functional
- New tool “estimate_cost” functional
- “get_results” added an optional parameter “show_misses” cosmetic
- “run_benchmark” added an optional parameter “custom_timeout” cosmetic
- “run_benchmark” added an optional parameter “fail_fast” cosmetic
- “run_benchmark” added an optional parameter “max_credits” cosmetic
- “run_benchmark” added an optional parameter “max_tokens” cosmetic
- “run_benchmark” added an optional parameter “preset” cosmetic
- “run_benchmark” added an optional parameter “temp_preference” cosmetic
- “run_benchmark” added an optional parameter “temperature_discovery” cosmetic
- “run_benchmark” added an optional parameter “timeout_profile” cosmetic
- 19 Oct 26 60
First indexed and scored.
Diagnostic detail from the automated scan of this channel: what the scanner observed at each step, so you can see exactly where a check passed or failed. It is informational only and never changes the trust score.
Captured 23 Oct 2026 · Probed https://openmark.ai/mcp
TLS valid
Negotiated TLS 1.3 with TLS_AES_128_GCM_SHA256 .
| Subject | Issuer | Valid from | Valid until | Key | Signature | Serial |
|---|---|---|---|---|---|---|
| CN=openmark.ai | CN=WE1,O=Google Trust Services,C=US | 20 Sept 2026 | 19 Dec 2026 | ECDSA 256 | ECDSA-SHA256 | 496d5af35f4a9b660e6c6b62e26fb09f |
| SANs: openmark.ai, *.openmark.ai | ||||||
| CN=WE1,O=Google Trust Services,C=US (CA) | CN=GTS Root R4,O=Google Trust Services LLC,C=US | 13 Dec 2023 | 20 Feb 2029 | ECDSA 256 | ECDSA-SHA384 | 7ff31977972c224a76155d13b6d685e3 |
| CN=GTS Root R4,O=Google Trust Services LLC,C=US (CA) | CN=GlobalSign Root CA,OU=Root CA,O=GlobalSign nv-sa,C=BE | 15 Nov 2023 | 28 Jan 2028 | ECDSA 384 | SHA256-RSA | 7fe530bf331343bedd821610493d8a1b |
Background: What to check on a remote MCP endpoint →
DNSSEC insecure
Validation of openmark.ai. — Not signed
| Zone | DS | Keys | Algorithms | Outcome |
|---|---|---|---|---|
| . | trust_anchor | 20326, 38696 | 8, 8 | Verified |
| ai. | present | 3799 | 8 | Verified |
| openmark.ai. | absent | Unsigned (proven) parent-signed NSEC/NSEC3 proves an unsigned delegation |
Authentication No authorisation required
The endpoint answered without asking for a token. Anyone who knows the URL can reach it.
| Result | No authorisation required |
|---|---|
| HTTP status | 200 |
| Header | Value |
|---|---|
| strict-transport-security | max-age=31536000; includeSubDomains |
| x-content-type-options | nosniff |
| x-frame-options | SAMEORIGIN |
| permissions-policy | camera=(), microphone=(), geolocation=() |
Background: How OAuth 2.1 works in the 2026 MCP spec →
Transports 2 probes
| Transport | URL | Outcome | Status | Location |
|---|---|---|---|---|
| streamable-http | https://openmark.ai/mcp | Verified | 200 | |
| http (plaintext) | http://openmark.ai/mcp | Inconclusive | 405 |
The tools this component advertises to a client, with an estimated token cost for each. Expand a tool to see its parameters and schema. The per-tool counts are indicative and are not scored directly; the schema's total context footprint is one signal in Schema Quality & AI Usability. A tool's description is untrusted text the model reads on every call, which is what makes this list a security surface and not just an inventory: how tool poisoning works →
estimate_cost Estimate a run's cost ~384
Credits a run_benchmark call would use, before spending anything: the expected total, the split per model, the balance needed to start, the worst case, and the user's balance. Same inputs and run settings as run_benchmark. Use it to drop an expensive model, or to compare settings (e.g. max_tokens), before running.
| Name | Type | Req | Description |
|---|---|---|---|
| custom_timeout | integer | – | Seconds per answer, with timeout_profile custom |
| fail_fast | boolean | – | Stop a model after repeated errors to save credits. Set false to let every model finish, e.g. to diagnose failures |
| max_tokens | integer | – | Output token limit per answer, 16 to 16000; 0 = no limit. Keep the default unless results show it was too low: a higher limit raises the credits needed to start. get_results flags models whose answer… |
| models | array | yes | – |
| preset | string | – | Like the app's presets: standard = 2 passes + snappy timeouts (the default), thorough = 3 passes + patient timeouts. Explicit stability_runs / timeout_profile override it |
| stability_runs | integer | – | Passes per model; 2+ shows which models are consistent |
| task_yaml | string | yes | The full task YAML (a JSON object with the same fields also works) |
| temp_preference | string | – | With temperature_discovery: which temperature wins a tie |
| temperature_discovery | boolean | – | Find each model's best temperature for this task. Costs about 6x as much |
| timeout_profile | string | – | How long to wait per answer: snappy 45s, patient 90s, custom = custom_timeout |
| upload_uid | string | – | From prepare_attachments, so image and document tokens are counted |
No output schema declared.
No examples provided.
get_account Account, credits and plans ~46
The user's plan, credit balance and limits, and what a credit pack, Pro or Expert would unlock. Use it before large runs, or when a run is refused for credits or limits.
Input schema present but exposes no named parameters.
No output schema declared.
No examples provided.
get_results Get benchmark results ~157
Status of a benchmark job, and once completed the ranked results (score, stability, cost per run, latency), the tests each model missed with its answer, a recommendation for the chosen objective and quick picks for the others.
| Name | Type | Req | Description |
|---|---|---|---|
| job_id | string | yes | – |
| optimize_for | string | – | accuracy (same order as the app's results table): best score, then cheapest, then fastest. cost / speed / stability: cheapest / fastest / most consistent model within `tolerance` points of the top sc… |
| show_misses | boolean | – | List the tests each model got wrong, with the expected answer and the model's reply |
| tolerance | number | – | Score points below the top score that still count as good enough |
No output schema declared.
No examples provided.
get_task_format Task YAML format ~112
How to write an OpenMark task: the YAML structure, all 18 scoring modes, and tips. Call this before writing a task. Pass a topic for the full detail on one subject (the app's Task Guide): fields, scoring, composite, chaining, documents, examples, best_practices, or a scoring mode name.
| Name | Type | Req | Description |
|---|---|---|---|
| topic | string | – | Optional: fields, scoring, composite, chaining, documents, examples, best_practices, or a scoring mode name such as regex_match |
No output schema declared.
No examples provided.
list_models List models ~121
Current OpenMark model ids with list prices ($ per 1M tokens), as text and as a structured list. Use these ids in run_benchmark. Append @low, @medium or @high to a reasoning model id to set its reasoning effort.
| Name | Type | Req | Description |
|---|---|---|---|
| images_only | boolean | – | Only models that accept image attachments |
| max_output_price | number | – | Only models at or below this $ per 1M output tokens |
| search | string | – | Filter by id, name or provider, e.g. 'claude', 'gemini flash' |
No output schema declared.
No examples provided.
prepare_attachments Prepare file attachments ~92
Get an upload_uid and a one-hour upload link (no API key needed) for images, PDFs or text documents, with the exact curl command. Upload the files, reference them by file name in each test's attachments list, then pass the upload_uid to estimate_cost / run_benchmark.
| Name | Type | Req | Description |
|---|---|---|---|
| file_names | array | – | Optional: the local file paths to upload, to get a ready-to-run command |
No output schema declared.
No examples provided.
run_benchmark Run a benchmark ~441
Save a task (YAML, see get_task_format) to the user's OpenMark account and run it on the given models with real API calls. Returns a job_id and the expected cost; poll get_results. Spends the user's credits; pass max_credits to only start within a budget. Optional run settings match the app's Configure & Run panel (preset, stability_runs, max_tokens, temperature_discovery, temp_preference, timeout_profile, custom_timeout, fail_fast).
| Name | Type | Req | Description |
|---|---|---|---|
| custom_timeout | integer | – | Seconds per answer, with timeout_profile custom |
| fail_fast | boolean | – | Stop a model after repeated errors to save credits. Set false to let every model finish, e.g. to diagnose failures |
| max_credits | integer | – | Budget: if the expected cost is higher, nothing runs and you get the quote instead |
| max_tokens | integer | – | Output token limit per answer, 16 to 16000; 0 = no limit. Keep the default unless results show it was too low: a higher limit raises the credits needed to start. get_results flags models whose answer… |
| models | array | yes | Model ids from list_models |
| preset | string | – | Like the app's presets: standard = 2 passes + snappy timeouts (the default), thorough = 3 passes + patient timeouts. Explicit stability_runs / timeout_profile override it |
| stability_runs | integer | – | Passes per model; 2+ shows which models are consistent |
| task_yaml | string | yes | The full task YAML (a JSON object with the same fields also works) |
| temp_preference | string | – | With temperature_discovery: which temperature wins a tie |
| temperature_discovery | boolean | – | Find each model's best temperature for this task. Costs about 6x as much |
| timeout_profile | string | – | How long to wait per answer: snappy 45s, patient 90s, custom = custom_timeout |
| upload_uid | string | – | From prepare_attachments, when tests use attachments |
No output schema declared.
No examples provided.
validate_task Check a task ~73
Check a task YAML the way OpenMark will before running it: structure, scoring settings, and the test limit of the user's plan. Free, saves nothing. Returns the problems to fix and warnings worth a look.
| Name | Type | Req | Description |
|---|---|---|---|
| task_yaml | string | yes | The full task YAML (a JSON object with the same fields also works) |
No output schema declared.
No examples provided.
watch_task Watch a task for better models ~95
Watch the task of a completed job: when new or cheaper models ship, OpenMark re-tests up to 4 of them next to the user's pick and emails the user only when one beats it. Each check uses the user's credits at the normal rate.
| Name | Type | Req | Description |
|---|---|---|---|
| job_id | string | yes | – |
| model | string | – | The model to keep as the pick (default: the best-scoring model). Use the model you recommended. |
No output schema declared.
No examples provided.
How do I install the OpenMark MCP server?
OpenMark is a hosted endpoint at https://openmark.ai/mcp, so there is nothing to install locally. Ready-made configuration for Claude, Cursor, VS Code, Codex and 5 more is on this page, copied from each client's own documentation.
remote · openmark.ai
claude mcp add --transport http ai-openmark-openmark 'https://openmark.ai/mcp'
What is the OpenMark MCP server?
OpenMark is an MCP server listed in the public MCP registry as ai.openmark/openmark. Benchmark your own prompt on 100+ LLMs with real API calls; pick the cheapest model that passes. This page covers its hosted endpoint (https://openmark.ai/mcp).
Is the OpenMark MCP server safe to use?
OpenMark scores 60 out of 100 on VerifyMCP. That is a record of what we were able to check automatically, not an endorsement. The category breakdown on this page shows every signal behind the number, including the ones we could not confirm.
What tools does the OpenMark MCP server expose?
OpenMark exposes 9 tools: get_task_format, validate_task, list_models, prepare_attachments, estimate_cost, and 4 more. Their descriptions and schemas cost roughly 1,521 tokens of context every time the server is loaded.
Does the OpenMark MCP server require authentication?
No. We connected to OpenMark without credentials and it answered, so anything it exposes is reachable by anyone who knows the address.
Is the OpenMark MCP server still maintained?
OpenMark is still listed as active in the MCP registry. We last reached this channel on 23 October 2026. Those dates come from our own scans of the registry and the channel itself, not from anything the publisher announced.