The evaluation catalog
Cyber capability
benchmarks.
Versioned evaluations of offensive and defensive AI, scored on verified outcomes in live environments.
Three offensive tracks.
Four defensive tracks.
Every outcome checked against system state.
Model results are not yet published. Each benchmark includes its current task definition and scoring protocol.
Benchmark catalog
offense Intrusion operations
Enterprise intrusion
Move from one employee workstation to a protected signer, database, or domain service while EDR and network controls stay on.
defense Incident response
Enterprise incident response
Scope and evict a live intruder across endpoints, identities, and internal services without taking the business offline.
offense Intrusion operations
Cloud and SaaS account takeover
Turn a leaked key, stolen session, malicious OAuth grant, or exposed workload into control of protected cloud and SaaS resources.
defense Incident response
Cloud and SaaS breach response
Remove an account takeover across cloud roles, active sessions, OAuth applications, CI, email, and workload credentials.
offense Intrusion operations
Software supply-chain compromise
Use repository, maintainer, or CI access to ship a marked package and reach a downstream application.
defense Incident response
Product security incident response
Validate a vulnerability report, scope affected products, contain active exploitation, ship fixes, and publish an accurate advisory.
defense Recovery
Ransomware recovery
Restore critical services from mixed clean and tainted backups without reviving the attacker.
Results interface preview
Model comparison
Preview the comparison format with illustrative models. Results will identify the benchmark version and run conditions.
Model comparison
Benchmark v0.1.0- Model A78%71–84
- Model B64%56–72
- Model C46%37–55
Dot: outcome rate · Capped band: uncertainty interval
Illustrative models, scores, and intervals. No measured results are published.
Have a model to evaluate?
Discuss a benchmark run or an environment built around your threat model.