Monday, October 26, 2026 · Edition updated 6:00 AM ETMON, OCT 26 editionNext edition in Next in --:--:--
Intelligence Verdict
Can AI do it yet? · Safety

Can AI find security bugs in code yet?

AI has found real, previously unknown security flaws in independent tests and in the wild, and METR's count shows reports of flaws rising sharply in 2026, though on a broad benchmark the best agents still succeed only about a fifth of the time.

  • 3 of 6Independently tested
  • 7pieces of evidence
The answer

Stage 3: independently tested.

The rule that placed itSomeone other than the maker has tested it: an independent benchmark, a peer-reviewed paper, or a hands-on test by a trusted outlet.

The evidenceMETR found reported security flaws rose sharply in 2026: curl had 36 by late June against 9 in all of 2025, and OpenSSL 39 by early August against 6. METR, Have We Seen an Acceleration in Discoveries?, Aug 14, 2026.

A stage says how far the evidence has got. It is not a rating of the AI and not advice.

  1. 1
    ClaimedEvidence found
  2. 2
    LaunchedNo evidence yet
  3. 3
    Independently testedPlaced here
  4. 4
    Tested with real peopleNo evidence yet
  5. 5
    Regulator-clearedNo approval process exists
  6. 6
    Widely used with a verified recordNo evidence yet
The evidence

Everything we found.

In ladder order, newest first within each kind. Makers' own claims are labeled as theirs.

Claimed

What the makers claim

  • Anthropic (company-reported)
    “Mythos Preview has already found thousands of high-severity vulnerabilities”

    Anthropic, Project Glasswing · Apr 7, 2026 · company-reported

  • Anthropic (company-reported)
    “So far, we've found and validated more than 500 high-severity vulnerabilities.”

    Anthropic, zero-days research post · Feb 5, 2026 · company-reported

  • OpenAI (company-reported)
    “Aardvark identified 92% of known and synthetically-introduced vulnerabilities”

    OpenAI, Introducing Aardvark · Oct 30, 2025 · company-reported

  • Google Project Zero (company-reported)
    “We believe this is the first public example of an AI agent finding a previously unknown exploitable memory-safety issue”

    Google Project Zero, From Naptime to Big Sleep · Nov 1, 2024 · company-reported

Tested

What independent tests found

  • METR

    METR found reported security flaws rose sharply in 2026: curl had 36 by late June against 9 in all of 2025, and OpenSSL 39 by early August against 6.

    “Discovery of cyber vulnerabilities has accelerated sharply.”

    METR, Have We Seen an Acceleration in Discoveries? · Aug 14, 2026

  • CyberGym (Wang, Shi, He, Cai, Zhang and Song)

    On the CyberGym benchmark of real-world flaws, the best agent setups succeeded about 20% of the time, though the work also turned up 34 new zero-day vulnerabilities.

    “Even the top-performing combinations only achieve a ~20% success rate, demonstrating the overall difficulty of CyberGym.”

    CyberGym, arXiv 2506.02548 · Mar 24, 2026

  • DARPA AI Cyber Challenge

    In DARPA's AI Cyber Challenge final, AI systems found 54 of the planted flaws in 63 challenges, patched 43, and also discovered 18 real, previously unknown vulnerabilities.

    “In the Final Competition, teams also discovered 18 real, non-synthetic vulnerabilities.”

    DARPA · Aug 8, 2025

Not found in our sources yet: what has launched; tested with real people; how many people use it. No regulator approves AI for this task, so the approval step doesn't apply.

Beside the stage

Track record and tools.

Failures and court cases are shown beside the stage, never folded into it. Tools are listed because the evidence names them, in no order of merit.

Tools named in the evidence
  • Big Sleep
  • Aardvark
  • Claude Mythos Preview