Anthropic's Eugene Yan: AI Can Find 20× More Software Bugs. The Real Crisis Is What ...

In April 2026, Mozilla's Firefox team shipped 400 security bug fixes in a single month. That's 20 times what the browser averaged per month across 2025. Then in February and March, the team had already tripled its usual pace to 60 or 70 fixes — before the April figure blew both months away. According to Mozilla's own attribution, roughly 271 of those April fixes, two-thirds, traced back to vulnerabilities surfaced by Claude Preview, Anthropic's frontier model.

The same capability was on display inside Anthropic itself. Eugene Yan, who runs AI-driven security work for the company as part of a team he calls Entropic, described scanning 23,000 open-source repository candidates, flagging 6,200 rated high or critical, reporting 1,600 to maintainers — and watching only 100 patches get pushed upstream. Discovery throughput was no longer the bottleneck. Everything downstream was.

Speaking on the AI Engineer podcast, Yan put the problem bluntly: "Finding vulnerabilities now is quite straightforward. The bottleneck has now shifted to verification, triage, and patching." That single sentence redefines what security teams should be optimizing for in 2026.

The numbers tell a story of a supply chain that snapped under its own volume.

A 94% drop-off between reporting and patching is not a detection problem. It is a verification, prioritization, and resourcing problem. Yan's point is that engineering organizations have spent years optimizing the left side of this funnel — building better scanners, training better models, running larger-scale crawls. The UK AI Security Institute's cybersecurity benchmark, which measures how long a model can sustain a cybersecurity task, shows 2026 models making a visible step-jump above the prior trendline. The models got good. The processes around them did not.

The Firefox numbers reinforce the asymmetry. A 20× surge in discovered vulnerabilities didn't break Mozilla — the team shipped them. But Mozilla is a well-resourced organization with dedicated security engineers. The open-source maintainers on the receiving end of Anthropic's 1,600 reports had no comparable infrastructure.

Yan's framework begins with a document, not with code. He calls it the threat model, and the teams that write one thoroughly see true-positive rates hit 90% or higher. The ones that skip it drown in false positives.

He quoted a CISO who captured the fundamental limitation of every large language model applied to code: "The model has great context of the code but poor context of the system." A model can read every line of a repository and still miss the fact that a particular service runs behind a VPN and a web application firewall, that an API boundary is trusted by design, or that a vulnerability class was silently mitigated during an on-call rotation three years ago. That knowledge lives in senior engineers' heads and in incident postmortems — documents a model will never see unless someone writes them down explicitly.

The threat model is where that implicit knowledge gets externalized. Yan recommends bootstrapping it from existing documentation and historical CVE patches, then interviewing the system experts who know why things were architected the way they are. The document should explicitly list compensating controls: "This endpoint is internal-only," "This database is behind a WAF that blocks SQL injection," "This service requires VPN access." Without those written down, the model will flag every textbook vulnerability it finds, regardless of whether it is actually exploitable.

Yan organizes the workflow that effective teams converge on into two phases: setup and loop.

Before any scanning begins, teams need a sandbox — a container-based replica of the target environment with the application, database, cache, and agent all running in isolation. Yan stressed that the sandbox must have no egress and no production credentials. Its job is reproducibility: every agent starts from the same baseline, and every proof-of-concept gets detonated safely. One team he worked with said their single biggest leverage point was "having the model test beds essentially sandboxes with live systems where they can run and detonate the PoCs to confirm that they are true positives."

The core of Yan's architecture is that discovery and verification must be handled by different agents. If the same agent does both, it will self-censor during discovery — consciously or not — because it knows it will later have to prove its own findings. That kills recall.

Discovery is optimized for breadth. Give the model as much written context as possible, plus tools — API query tools, log readers, code readers — so it can interact with the system dynamically rather than reasoning from static files. Yan noted that he has been able to cut his prompt size by roughly 50% with each new Claude version, because the models internalize more context on their own. One pentesting team that adopted interactive tooling achieved what Yan described as "almost 100%" true-positive rates.

He demonstrated the point with a minimal example: a five-line Python function that builds a SQL query through string interpolation. Current models spot that instantly — the discovery step is not where the difficulty lies.

Verification is where the real work happens. The verification agent receives only the vulnerability report, sees none of the discovery agent's reasoning, and is explicitly adversarial — its job is to try to prove the finding false. It detonates a proof-of-concept in the sandbox. In the SQL injection example, the verification agent runs a curl command and watches customer PII get exfiltrated from the database. The exploit is confirmed. Without this step, a finding is just a suggestion.

Triage is where business context from the threat model becomes decisive. Yan described a case where an agent flagged a medium-severity issue as high because the database contained healthcare data. A human reviewer downgraded it to low: the service was internal-only, behind a VPN, and protected by an application firewall that blocked SQL injection. The threat model had documented those compensating controls; the agent, lacking that system context, had overestimated the risk.

Yan's warning: product engineers will stop reading vulnerability reports entirely if they receive hundreds of medium- and low-severity tickets. The trust evaporates. Triage must be calibrated with written rules that both security and product teams agree on — once, up front — and those rules must be fed to the ranking agent.

Patching closes the loop. Yan specifies three criteria for any AI-generated patch: the original exploit must stop working, the existing test suite must stay green, and a fresh discovery agent must re-attack the patched code to verify the fix is comprehensive. He called this the "generative verifier loop." In the SQL injection example, the fix was a single line of code — moving the parameter out of string interpolation. The patch also included a second diff that updated the threat model to document the previously unstated compensating controls, so the scan iteration would be smarter.

"When you close the loop, they now become capital expense. You get better with each iteration you run," Yan said. A security scan that produces a list of findings is an operating expense — you pay for it, you get a spreadsheet, you do it again quarter. A scan whose learnings feed back into the threat model and the verification rules becomes a capital asset that compounds.

Yan is candid that the technical problems are solvable. "Non-technical problems are an order of magnitude harder than technical problems," he said, citing a former director. He identified three organizational constraints that break as agentic pipelines scale:

Vulnerability routing. At dozens of findings per month, a human can manually assign Jira tickets. At hundreds, every team needs an automated owner-lookup system based on code ownership. This does not require an LLM — it is a basic engineering investment that many organizations have not made.

Severity calibration. Security engineers and product engineers systematically disagree on what "high severity" means. The only durable fix is for both sides to sit down, write down shared definitions, and feed those into the triage agent. Yan observed that product teams lose trust when inundated with over-ranked findings. The rubric is a social document, not a technical one.

Patching bandwidth. Yan noted that few professionals at Anthropic still write code entirely by hand, but many still manually construct patches from vulnerability reports. The goal should be AI-generated patches with human validation, using the generative verifier loop to catch regressions automatically. "Human attention doesn't scale," he said — and patching is where the attention bottleneck bites hardest.

Yan's advice for teams that want to begin is deliberately incremental. Start with open-source dependencies — a bounded, non-production domain where mistakes are safe. Use an interactive tool like Claude Code hands-on rather than aiming for full automation on day one. "Learn where you get cut. Learn what kind of context you're missing. Learn where precision is low," he said.

Above all, do not build a scanner and stop. Scanning is not the bottleneck. The process design — verification, triage, patching, and the organizational agreements around severity — is where the work actually happens. Anthropic has released open-source repositories that include interactive skill definitions and autonomous harness configurations; Yan pointed to a customizable harness in step five of the provided framework as a practical starting point.

The AI security pipeline is converging on the same architecture as every machine-learning pipeline: the returns on model improvement increasingly come not from raw detection capability but from systems engineering around recall, precision, ranking, and closed-loop validation. The threat model is the equivalent of a feature store — the tacit knowledge that makes a model's outputs trustworthy must be externalized, or the agent will miss context that any human junior engineer would have inferred. The most brittle constraint Yan identifies is not a model's reasoning ability. It is whether security and product teams can agree, in writing, on what counts as a real problem. Until they do, the agent will produce outputs neither group fully trusts. The frontier of AI security, Yan's work suggests, is less code and more the organizational discipline of writing down the unwritten rules.

Once added, BigGo Finance appears first in Google Top Stories, so you get the broadest, most up-to-the-minute, and most comprehensive global financial news first.