Stage 3: independently tested.
The rule that placed itSomeone other than the maker has tested it: an independent benchmark, a peer-reviewed paper, or a hands-on test by a trusted outlet.
The evidenceMETR found reported security flaws rose sharply in 2026: curl had 36 by late June against 9 in all of 2025, and OpenSSL 39 by early August against 6. METR, Have We Seen an Acceleration in Discoveries?, Aug 14, 2026.
A stage says how far the evidence has got. It is not a rating of the AI and not advice.
- 1ClaimedEvidence found
- 2LaunchedNo evidence yet
- 3Independently testedPlaced here
- 4Tested with real peopleNo evidence yet
- 5Regulator-clearedNo approval process exists
- 6Widely used with a verified recordNo evidence yet
Everything we found.
In ladder order, newest first within each kind. Makers' own claims are labeled as theirs.
What the makers claim
- Anthropic (company-reported)
“Mythos Preview has already found thousands of high-severity vulnerabilities”
Anthropic, Project Glasswing · Apr 7, 2026 · company-reported
- Anthropic (company-reported)
“So far, we've found and validated more than 500 high-severity vulnerabilities.”
Anthropic, zero-days research post · Feb 5, 2026 · company-reported
- OpenAI (company-reported)
“Aardvark identified 92% of known and synthetically-introduced vulnerabilities”
OpenAI, Introducing Aardvark · Oct 30, 2025 · company-reported
- Google Project Zero (company-reported)
“We believe this is the first public example of an AI agent finding a previously unknown exploitable memory-safety issue”
Google Project Zero, From Naptime to Big Sleep · Nov 1, 2024 · company-reported
What independent tests found
- METR
METR found reported security flaws rose sharply in 2026: curl had 36 by late June against 9 in all of 2025, and OpenSSL 39 by early August against 6.
“Discovery of cyber vulnerabilities has accelerated sharply.”
METR, Have We Seen an Acceleration in Discoveries? · Aug 14, 2026
- CyberGym (Wang, Shi, He, Cai, Zhang and Song)
On the CyberGym benchmark of real-world flaws, the best agent setups succeeded about 20% of the time, though the work also turned up 34 new zero-day vulnerabilities.
“Even the top-performing combinations only achieve a ~20% success rate, demonstrating the overall difficulty of CyberGym.”
CyberGym, arXiv 2506.02548 · Mar 24, 2026
- DARPA AI Cyber Challenge
In DARPA's AI Cyber Challenge final, AI systems found 54 of the planted flaws in 63 challenges, patched 43, and also discovered 18 real, previously unknown vulnerabilities.
“In the Final Competition, teams also discovered 18 real, non-synthetic vulnerabilities.”
DARPA · Aug 8, 2025
Not found in our sources yet: what has launched; tested with real people; how many people use it. No regulator approves AI for this task, so the approval step doesn't apply.
Track record and tools.
Failures and court cases are shown beside the stage, never folded into it. Tools are listed because the evidence names them, in no order of merit.
- Big Sleep
- Aardvark
- Claude Mythos Preview
Can AI do these yet?

Answer my health questions
Stage 4 of 6Tested with real people
Randomized trial: chatbot users made no better health decisions

Spot breast cancer on a mammogram
Stage 5 of 6Regulator-cleared
Transpara has been FDA-cleared since 2018, most recently in Aug 2026

Check a mole for skin cancer
Stage 5 of 6Regulator-cleared
EU Class III CE mark lets DERM clear benign cases alone (company)
