Trust the meter,
not the agent
Anyone can publish an eval. We certify that an eval can be trusted — reproducible, ungameable, and anchored to real standards — before it grades anything. Only the ones that make the grade reach the marketplace.
Why an authority
No one practices medicine because they say they’re good at it. Humans earn trust through exams from independent authorities — different exams for different stakes.
Agents are no different.
The authority
There is no oracle for the perfect eval.
So we never score a pack against absolute truth. We measure it on six axes that need no oracle — from discriminating power to whether the eval is still current as the frontier moves.
Structural validity
Is the eval reproducible? Does the golden dataset leak into the criteria? Is the rubric gameable? Provable without knowing the ‘best’ eval.
Discriminating power
Mutation testing applied to evals: we run the pack against a reference panel of known-quality agents — good, broken, sabotaged. Weighted highest of the six.
Standard coverage
For conformance packs, coverage against an external standard — EU AI Act, OWASP-agentic, NIST, ISO 42001 — never against a private oracle we invented.
Test thoroughness
Test-case volume weighed against a minimum that scales with the pack's own risk tier — a high-risk pack with 15 cases doesn't score like a low-risk one.
Robustness
Does the grade survive perturbed inputs? We reorder and pad the test items in meaning-preserving ways; a pack keyed to their exact surface form collapses.
Currency
We measure decay against a dated frontier cohort — an eval today's best agents have saturated no longer discriminates. Robustness across time, not calendar age.
Live verification
Watch a pack earn its grade.
Every pack runs against a reference panel of known-quality agents — good, broken, sabotaged. A real eval ranks them correctly and fails the planted bug. A pack that lets the sabotaged agent pass is a bad pack. We grade the eval, not the agent.
Why trust us
An authority you can re-run, not just believe.
Verification authority
The trusted method that decides whether an eval deserves trust or loses it. Reference panels, adversarial audit, standard anchoring. This is what can’t be cloned — because trust can’t be cloned.
Reproducible verdicts
Every grade ships with a public verification report and the exact panel run behind it. Re-run it in your own stack — Promptfoo, DeepEval, Ragas — and get the same answer. The grade isn’t a claim; it’s a result.
Marketplace
Different stakes, different exams
A different eval pack per task, domain, and risk — every one of them passed the authority before it went on sale.
Integration
Drop a certified eval pack into CI in seconds.
Packs ship runnable in the frameworks you already use — Promptfoo, DeepEval, Ragas — with the verification report and grade seal attached.
Runs in your stack · Anchored to real standards
Get the stamp that can’t be cloned
Browse packs that already passed the authority, or read how grading works before you buy.