01
Offensive capability
Evaluate whether a model can discover weaknesses, navigate access boundaries, and reach a defined objective in a controlled environment.
Cyber evaluations / For frontier model companies
License cyber evaluation suites or commission private evaluations for your frontier models. Measure offensive and defensive capability through tasks with explicit objectives and verifiable outcomes.
01
Evaluate whether a model can discover weaknesses, navigate access boundaries, and reach a defined objective in a controlled environment.
02
Evaluate whether a model can investigate an intrusion, contain the attacker, remediate a weakness, or restore a service.
03
Compare checkpoints, harnesses, and tool configurations under the same task version and run conditions.
The engagement
Defined starting access, tools, objectives, constraints, and evaluation conditions.
Checks against the resulting system state, with operational cost, disruption, and scope violations recorded separately.
Environment and grader versions, model configuration, budgets, and trial conditions that make comparisons interpretable.
Scoring outputs and run artifacts appropriate to the agreed engagement, with limitations documented alongside results.
The comparison
Compare runs under the same task version, harness, budgets, and trial conditions. This preview shows illustrative data.
Browse benchmark specificationsModel comparison
Benchmark v0.1.0Dot: outcome rate · Capped band: uncertainty interval
Illustrative models, scores, and intervals. No measured results are published.
Working with us
Identify the capabilities and model decisions you need to evaluate. Choose an existing task family or commission a private specification.
Define licensing, deployment, model access, harness integration, and the handling of evaluation materials before work begins.
Execute under the agreed conditions, inspect the evidence, and distinguish capability limits from failures in the evaluation itself.
Yes. We can scope tasks around specific capabilities, tool access, and threat models. Discuss the intended use, delivery requirements, and evaluation schedule with us.
The public catalog currently contains active, versioned benchmark specifications. Its model charts are clearly labeled demonstrations. Measured results will be published after validation.
Training uses separately designated scenarios. Held-out benchmark tasks, seeds, answers, and evaluation traces remain outside the training corpus. Capability environments are a separate offering.