AI Pentesting Agents: What Claude Mythos and AIxCC Proved

AI pentesting agents have real results now: DARPA's AIxCC final and Anthropic's Claude Mythos Preview. What they found, what it cost, and where humans still matter.

Part of theAI Security & Safety series
focus photography of printed board
Listen to this article
--:--

AI pentesting agents are real, but the startup story this post first told was not. No company called Mythos AI was a finalist in DARPA's AI Cyber Challenge. What actually happened is more interesting. In August 2025, the seven autonomous systems in DARPA's AIxCC final found 54 of 63 planted vulnerabilities and 18 real ones. In April 2026, Anthropic announced Claude Mythos Preview, an unreleased model that found and exploited zero-days on its own, including a 27-year-old bug in OpenBSD.

Correction: An earlier version of this post said a startup called Mythos AI was one of seven AIxCC finalists, quoted a CEO we could not verify, and quoted a DARPA program manager named Pertti Kohonen. None of that holds up. DARPA's results list the seven finalists and Mythos AI is not one of them, the AIxCC program manager quoted at the final was Andrew Carney, and the final had already run when the post called it scheduled. This version is rebuilt on DARPA's and Anthropic's own publications.

Abstract glowing purple waveform pattern on a black background

I've been tracking the AI-in-security space closely, and most of what I've seen is incremental. Better scanners, fancier dashboards, pattern matching dressed up with a "powered by AI" badge. The two real stories in this post are different, and each is worth taking on its own terms.

What Is an AI Pentesting Agent and Why Does It Matter?

An AI pentesting agent is an autonomous system that uses large language models to perform offensive security operations — discovering, analyzing, and exploiting software vulnerabilities — without constant human direction. The difference between this and a traditional vulnerability scanner is the difference between a spell-checker and a writer. Scanners check known patterns against known databases. An AI pentesting agent reasons about code, forms hypotheses about where bugs might live, and attempts to prove those hypotheses by actually exploiting them.

Long-exposure photo of white and orange light streaks blurring across a dark background

The cybersecurity industry's problem is skills, not just headcount. In ISC2's 2025 workforce study, 95% of respondents reported at least one skills need, 59% cited critical or significant skills gaps, and 29% said they cannot afford to hire staff with the skills they need to adequately secure their organizations (CSO Online). An agent that can bridge that gap isn't a nice-to-have. It's a necessity.

Having worked on systems where security was always the thing that got pushed to "next sprint," I can tell you firsthand: most teams don't skip security because they don't care. They skip it because thorough penetration testing is expensive, slow, and hard to staff. If an AI agent can handle even 60% of what a junior pentester does, that changes the economics of application security overnight.

How Claude Mythos Preview Finds and Exploits Bugs

Anthropic describes Claude Mythos Preview as "a general-purpose, unreleased frontier model" and announced it on April 7, 2026, alongside Project Glasswing. Its vulnerability-hunting scaffold is simple to describe. Each target project runs offline in an isolated container. The model ranks files from 1 to 5 by how likely they are to hold a bug, reads the code, forms a hypothesis, and runs the project to confirm or reject it. A second Claude instance then checks whether the bug is real and worth reporting.

The results are the reason this matters. Anthropic reports a 27-year-old bug in OpenBSD's TCP SACK handling that lets an attacker crash hosts, a 16-year-old bug in FFmpeg's H.264 code, chains of Linux kernel bugs that escalate privileges, and vulnerabilities in every major web browser. The OpenBSD hunt cost under $20,000 across 1,000 runs.

The technical approach chains together multiple capabilities: code analysis, hypothesis generation, exploit construction, and verification. The agent doesn't just flag a potential issue. It attempts to build a working exploit, which is how you separate a real vulnerability from a false positive. If you've ever dealt with the output of a static analysis tool that flags 500 "critical" issues, 480 of which are noise, you know exactly why this distinction matters.

Humans still sit at the end of the pipeline. Anthropic triages every bug it finds and sends the highest-severity ones to professional human triagers before disclosing them to maintainers. In 89% of 198 manually reviewed reports, expert contractors agreed with Claude's severity rating exactly, and Anthropic says more than 99% of what it found was still unpatched when it published. Its Linux kernel remote-code-execution attempts failed despite thousands of scans.

Architecturally it is an ordinary agentic loop; the key difference is that the action space is adversarial. For more on that, see how AI agents are being architected more broadly.

At launch, Anthropic said it did not plan to make Mythos Preview generally available. Instead it ran it through Project Glasswing with AWS, Apple, Broadcom, Cisco, CrowdStrike, Google, JPMorganChase, the Linux Foundation, Microsoft, NVIDIA and Palo Alto Networks, plus more than 40 other organizations that build or maintain critical software. It committed up to $100M in usage credits, $2.5M to Alpha-Omega and OpenSSF, and $1.5M to the Apache Software Foundation. Anthropic set post-credit pricing at $25/$125 per million input/output tokens. On the CyberGym benchmark it scored 83.1%, against 66.6% for Claude Opus 4.6.

The access picture has moved since then. On June 9, 2026, Anthropic released Claude Fable 5 for general use, alongside Claude Mythos 5, which is the same underlying model with the safeguards lifted in some areas, for a limited group of cyberdefenders through Glasswing. On October 6, 2026, it expanded its Cyber Verification Program into three access tiers that include Claude Mythos 5.1, and moved existing Glasswing members to the Specialized Access tier. It still says it does not plan to make Mythos Preview generally available.

DARPA's AI Cyber Challenge: What Actually Happened

DARPA's AI Cyber Challenge isn't a hackathon. It is a DARPA competition, run with ARPA-H, in which teams build autonomous cyber reasoning systems that find and patch software vulnerabilities. Seven finalists came out of the semifinal at DEF CON 32 in August 2024, each with $2 million (Infosecurity Magazine). DARPA announced the final results on August 8, 2025, at DEF CON 33.

Team Atlanta won $4 million, Trail of Bits $3 million and Theori $1.5 million. The other finalists were all_you_need_is_a_fuzzing_brain, Shellphish, 42-beyond-bug and Lacrosse. Across the final, the systems analyzed more than 54 million lines of code, found 54 of 63 synthetic vulnerabilities (86%) and patched 43. They also found 18 real, non-synthetic vulnerabilities and produced 11 patches for them. The average submission took 45 minutes and cost $152 per task.

When DARPA puts millions of dollars behind a technology category, it's not because they think it's cute. It's because they think it's critical to national security.

Andrew Carney, AIxCC's program manager, said of the real-world bugs: "We are now in the process of disclosing [these real-world zero-day vulnerabilities] to maintainers" (Infosecurity Magazine). DARPA says all seven finalist systems are being released as open source under OSI-approved licenses. That pairing, find AND fix and then give the tools away, is the part people should pay attention to. The defensive application is just as significant as the offensive one.

I've seen enough security vulnerabilities in production systems to know that the patch side is where the real value lives. Finding bugs is glamorous. Fixing them before attackers find them is what actually keeps systems safe.

Can AI Actually Replace Human Penetration Testers?

Short answer: no. Longer answer: it doesn't need to.

The "AI vs. human pentesters" framing is wrong. The right question is whether AI plus human pentesters beats the testing a team can actually afford today.

AI pentesting agents are already good at systematic analysis of large codebases. They're consistent — they don't get tired at 3 AM. They iterate through hypothesis-test cycles fast. And they can find novel instances of known vulnerability classes that signature tools would miss.

What they're still bad at: creative lateral thinking, understanding business logic flaws that require deep domain knowledge, social engineering, and making judgment calls about what actually matters versus what's technically exploitable but practically irrelevant.

From my experience building production systems, the vulnerabilities that actually get exploited are rarely the clever zero-days. They're the boring ones. Misconfigured permissions. Unvalidated inputs. Secrets committed to repos. An AI agent that systematically catches the boring stuff would prevent the vast majority of real-world breaches. That's not sexy, but it's correct.

The Ethics of AI Pentesting Agents: Are We Building Hacking Tools?

Let me just say it plainly: yes, an AI system that can find and exploit vulnerabilities is, definitionally, a hacking tool. The same technology that helps a company find its own vulnerabilities could help an attacker find them first. This tension isn't new. Metasploit, Burp Suite, Nmap — all of these can be used offensively or defensively. The cybersecurity industry has always lived with this.

What's different with AI agents is scale. A human attacker probes one target at a time. An AI agent can probe thousands simultaneously. That asymmetry is new and worth taking seriously.

The flip side of scaling offensive capability is scaling impersonation, and voice is quickly becoming the easiest attack surface to automate. If you’re evaluating countermeasures, Deepfake Voice Detection: 7-Step Detector Eval Guide [2026] walks through a practical way to benchmark detectors before you trust them in production.

Both real programs are framed as defense. AIxCC scored patching alongside discovery, and Anthropic keeps its least-restricted Mythos models for vetted security teams through Glasswing and the Cyber Verification Program, while the public Fable model ships with safeguards.

But let's be honest about the security implications of giving AI systems adversarial capabilities. The genie is out of the bottle. Seven AIxCC teams are open-sourcing their systems, and Anthropic offers Mythos models with lifted safeguards to vetted security teams through Glasswing and the Cyber Verification Program. The question isn't whether AI pentesting agents will exist. It's whether defenders will adopt them faster than attackers do.

Adversarial capability isn't the only way AI agents end up leaking sensitive data by accident — permission design matters just as much, which is exactly what went wrong when Claude Code's 'Don't Ask Again' Leaked 1,122 Secrets into public GitHub repos.

The legal side is straightforward. AI-driven security testing falls under the same frameworks as traditional pentesting: you need authorization to test a system. Using an AI agent to probe a system without permission is just as illegal as doing it manually. The tool doesn't change the law. But it does change the enforcement challenge considerably.

What This Means for the Future of Cybersecurity

AI pentesting agents are about to shift the economics of an entire industry. Not in the "press release says revolutionary" way. In the quiet way where pricing models change and hiring patterns shift and suddenly your board is asking why you're not using AI for security testing.

Here's my prediction: within two years, every major cloud provider will offer some form of AI-powered vulnerability discovery as a built-in service. The standalone pentesting engagement — hire a team for two weeks to poke at your application — won't disappear, but it becomes the premium tier of a market where AI handles the baseline. Open-sourced AIxCC systems and gated programs like Glasswing are early versions of that shift.

The AIxCC results are a real signal. Eighteen real-world vulnerabilities found by autonomous systems whose average competition task cost about $152 is not a toy result. Anthropic's caveats are the counterweight: memory-safety bugs are easy to verify, logic bugs are not, and its Linux kernel remote-code-execution attempts came up empty.

Either way, DARPA, competitive hacking veterans, and serious AI researchers are all converging on this problem. And in my experience, that kind of convergence usually means something real is happening. The boring answer is the right one: AI won't replace security professionals. But security professionals who use AI will replace those who don't. If you're building anything that touches production, now is the time to pay attention.

Photo by Jr Korpa on Unsplash.

Continue reading

a person is typing on a black keyboard

Is Claude Computer Use Safe? Security Risks and How to Sandbox It

Claude can now click, type, and navigate your desktop like a human. The security implications of handing an LLM full OS-level control are massive — and most people aren't thinking about them.

windows 11 laptop screen settings privacy — illustration for article on Windows 11 Coding Agent Telemetry:

Windows 11 Coding Agent Telemetry: Opt Out [2026]

A reverse engineer proved Windows 11 logs which AI coding agents you run — claude.exe, cursor.exe, copilot.exe — and uploads it to Microsoft by default. Here's how to turn it off.

an old book with writing on it

AI Cracked Gödel's Notebooks: Hype vs Proof [2026]

AI helped decode 1,009 pages of Kurt Gödel's shorthand notebooks, said to hold hundreds of proved theorems. Here's the real OCR+LLM pipeline behind it — and why Penrose isn't impressed.

a man sitting in front of a laptop computer

Spec-Driven Development vs Waterfall for AI Agents

Spec-driven development is being pitched as the structured antidote to vibe coding. Here's why it isn't Waterfall rebranded, and the specific places it breaks once a real agent is the one reading the spec.

Cite this article
Kunal Ganglani (2026, April 11). AI Pentesting Agents: What Claude Mythos and AIxCC Proved. Kunal Ganglani. Retrieved November 17, 2026, from https://www.kunalganglani.com/blog/ai-pentesting-agents-mythos-darpa

Frequently Asked Questions

Can AI replace human penetration testers?

Not entirely. AI pentesting agents excel at systematic, large-scale analysis and catching common vulnerability patterns consistently. But they still struggle with creative lateral thinking, business logic flaws, and contextual judgment. The real value is that AI handles baseline testing so human experts can focus on complex, high-value assessments.

What is DARPA's AI Cyber Challenge (AIxCC)?

AIxCC is a DARPA competition, run with ARPA-H, that tests whether autonomous AI systems can find and patch software vulnerabilities. Seven finalists came out of the DEF CON 32 semifinal in August 2024. At the DEF CON 33 final, Team Atlanta won $4 million, Trail of Bits $3 million and Theori $1.5 million, and the systems found 18 real-world vulnerabilities.

How do AI pentesting agents find vulnerabilities?

Unlike traditional scanners that match known patterns, AI pentesting agents use large language models to reason about code. They read source code, build a model of how the application works, hypothesize where flaws might exist, then attempt to construct working exploits to verify those hypotheses. This approach can catch novel vulnerabilities that signature-based tools would miss.

Are AI hacking tools legal?

AI pentesting tools fall under the same legal frameworks as traditional security testing tools like Metasploit or Burp Suite. Using them with proper authorization on systems you own or have permission to test is legal. Using them against systems without authorization is illegal, regardless of whether the testing is performed by a human or an AI agent. The tool doesn't change the law.

What is Claude Mythos Preview?

Claude Mythos Preview is an unreleased Anthropic frontier model, announced on April 7, 2026, that autonomously found and exploited zero-day vulnerabilities, including a 27-year-old OpenBSD bug and a 16-year-old FFmpeg bug. At launch Anthropic said it did not plan to make it generally available and offered it to partners through Project Glasswing. In June 2026 it released Claude Fable 5 for general use and Claude Mythos 5, the same underlying model with fewer safeguards, for a limited group of cyberdefenders.

Was Mythos AI a finalist in DARPA's AI Cyber Challenge?

No. DARPA's published results list the seven finalists: Team Atlanta, Trail of Bits, Theori, all_you_need_is_a_fuzzing_brain, Shellphish, 42-beyond-bug and Lacrosse. An earlier version of this post said otherwise, and it has been corrected.