Why Legacy SAST Scanners Leave You Exposed to AI-Armed Attackers
See what AISLE can find and fix autonomously in your own code.
For two decades, finding vulnerabilities in source code was slow, expensive, and tedious. But today, vulnerabilities that were effectively invisible for years are being found quickly, cheaply, and autonomously. In fact, according to the Berkeley Vulnerability Initiative, which tracks CVEs found by agentic systems, AI-native tools are finding hundreds of vulnerabilities each quarter, and that only includes those found by defenders.
Discovering real vulnerabilities with an LLM is one thing, but running a reliable security workflow is another. Cyber leaders are right to ask whether these headline-grabbing AI discoveries are anything more than a scientific curiosity. Can AI-native code analyzers be trusted to deliver consistent results, or are they just a novel complement to SAST scanners?
Abundant, publicly available evidence indicates that AI cybersecurity systems aren’t just a supplement to SAST. Instead, they provide the step change in analysis and remediation that organizations need in order to find, prioritize, and resolve exploitable vulnerabilities fast enough to make a difference.
What Legacy Pattern Matching Tools Miss
Even as AI vulnerability detection grabs headlines, the primary tooling cybersecurity teams use for code analysis is still based on pattern matching. For instance, rule-based SAST finds what someone wrote a rule for. That means SAST has two basic problems: it misses bugs no one wrote a rule for, and it creates too much noise when rules are broad.
The data suggests that these problems have significant real-world implications. A scientific study found that state-of-the-art commercial SAST tools miss between 47% and 80% of real-world CVEs. Yet on synthetic benchmarks, the same tools scored near 95%. In other words, traditional SAST can look strong in a benchmark while missing the “CVE-worthy” vulnerabilities that survive in real software. And those are exactly the vulnerabilities that AI-native attackers are now getting better at finding.
Figure 1: Recall on real CVEs vs. synthetic benchmarks. Source: Lipp, Banescu & Pretschner, ISSTA 2022.
These findings were recently corroborated in a new benchmarking study. Researchers took 95 public CVEs discovered by AISLE to build a test that compares legacy SAST tools and AI scanners. To make the findings as robust as possible, they deliberately selected small, cheap LLMs for the benchmark to see if even relatively weak models outperform their legacy counterparts.
Each analyzer was given source code and target-file scope and asked to identify the code path, root cause, attack condition, and impact of each CVE. The AIs they tested found more than five times as many CVEs as legacy scanners. For instance, while the best SAST tool, Snyk Code, found 12.1% of the CVEs, the best LLM, GPT-Luna, found 68%.
Importantly, the AIs didn’t find every vulnerability in a single run. Instead, they accumulated findings over four distinct runs, reflecting the need to engineer a harness around the models for reliable performance. As the study explains, “vulnerability discovery is a systems problem, not a single model call. The scaffold steers the model toward vulnerability reasoning, and triage keeps the volume of reported findings low enough for human review.”
Figure 2: Expected coverage from repeated passes with AI analyzers. Source: AISLE HoF-Bench (2026), arXiv:2607.27030
For defenders, that last part is key. Findings are important, but they can only be acted upon when they’re not buried under false positives. Here, too, open-weights models outperformed their legacy counterparts, returning only 3–4 deduplicated findings per discovered CVE.
This research provides an explanation for something security leaders have long known: SAST has become a checkbox exercise. Few teams treat it as a tool where real security benefits can be achieved. Instead, it’s a way to satisfy an auditor, produce a report, and demonstrate control.
Open source tells a similar story from the other side. In projects without a compliance buyer forcing the checkbox, maintainers tended to allocate their scarce time to tools and processes that clearly help: linters, sanitizers, fuzzing harnesses, code review processes, and bug bounty reports. Commercial SAST is rarely used, because when no forces the checkbox, the signal is often not worth the noise.
Signal-to-Noise Rates and the Threshold for Trust
Missing CVEs is bad enough, but false positives are where traditional tooling fails users most.
Legacy SAST tools create a level of distrust that becomes self-reinforcing. Alert queues are noisy, engineers have learned to ignore them, and real findings are swallowed by everything else. That may be the largest hidden tax imposed by traditional AppSec tooling: it simply trains engineering teams to stop paying attention.
To be fair, SAST took shape in a different world. Most attackers did not have direct access to proprietary source code, and finding novel, application-specific vulnerabilities was still expensive. So while SAST was missing a large share of real bugs and sending teams down false positive rabbit holes, organizations were still protected by the cost and difficulty of discovery.
Today, those conditions are gone. Attackers no longer rely on slow, manual discovery. Instead, AI-native systems search across whatever surface is available: source code, binaries, dependencies, public commits, documentation, endpoints, and runtime behavior. And they can do it faster, cheaper, and at a much greater scale than the old tooling model assumed.
As a result, the bugs SAST misses are no longer safely buried in complexity. They are machine-searchable attack surfaces.
Figure 3: Summary of benchmarking study results. Source: XX YY
The Power of AI-Native Cybersecurity Analysis
There is still a lot of confusion about what AI systems can and cannot do, especially in relation to traditional AppSec techniques like SAST, DAST, and SCA. There is also confusion about what it takes to operationalize AI-native vulnerability discovery in a way that is reliable enough for real engineering teams.
AI-native vulnerability discovery is not simply better than SAST. It is an entirely different security process: one that reasons about code behavior, verifies exploitability, and produces actionable findings rather than matching source code against a database of known patterns.
For example, the breadth of AI-powered vulnerability detection is already much wider than many assume. Initially, some skeptics argued that AI systems would be better at systems-level memory safety bugs than at classic web vulnerabilities. Yet the UC Berkeley data does not support that view. By vulnerability class, the single most common category found by agentic systems is cross-site scripting, which is the most classic web bug there is, sitting alongside use-after-free and out-of-bounds writes.
Figure 4: Share of AI-discovered CVEs by weakness class. Source: Berkeley Vulnerability Initiative.
Because AI-native security analyzers reason through codebases, they infer trust boundaries, follow data flows, and determine whether a path is actually reachable and exploitable. That means they don’t just pick up on the easy cases. For instance, many of the vulnerabilities that AISLE has helped to uncover survived more than 25 years of human audits and fuzzing. That is exactly the kind of long-lived, real-world vulnerability traditional tooling was never built to reliably find.
The Evidence: AI Finds Vulnerabilities in Hardened Code
AISLE is an AI-native system purpose-built for complex cyber tasks, and the public track record shows what it does on real-world code. AISLE’s system has found the overwhelming majority of newly disclosed vulnerabilities across multiple hardened open-source codebases, including OpenSSL and curl, projects already shaped by years of human audit, fuzzing, and static analysis.
In OpenSSL, AISLE identified 20 of 23 zero-days across three consecutive security releases. In curl, AISLE analysis contributed to more than 200 bug fixes in a codebase already run continuously through tools like Coverity, CodeQL, Clang, and OSS-Fuzz.
Figure 5: Detection in hardened real-world codebases. Sources: OpenSSL Foundation advisories; Daniel Stenberg / curl; corroborated by Schneier on Security.
Why AI-Native Detection Should Be Your System of Record
The natural instinct may be to run AI-native tooling next to existing SAST as an additional layer of defense. That is understandable, but it misses the point. If your engineers already distrust their scanner stack, simply adding another tool to it won’t win that trust back. After all, keeping legacy SAST around may buy redundancy on easy cases, but it degrades the signal quality of the overall workflow to such a degree that engineers don’t trust it. Adding another, albeit AI-powered, tool to that flow won’t fix that underlying issue.
How do you win back their trust? Incremental improvements aren’t enough. To persuade engineers to start acting on findings again, instead of dismissing them by default, you need a quantum shift in efficacy. Behavior changes when engineers see that virtually every alert they get is grounded in a real, pressing issue. And when behavior does change, the cost dynamics of your security program change with it.
That’s ultimately why AI-native security solutions aren’t “just another scanner.” Rather, abundant evidence indicates that they should become the primary source of vulnerability signal: the system of record that determines what is real, reachable, and exploitable.
Delivering a Trustworthy VulnOps Workflow With AISLE
AISLE verifies every finding before it ever reaches you. Where other AI-native tools surface what looks exploitable, AISLE confirms it, and its unique distillation process brings the effective false positive rate below 5%. That is the threshold where engineers start trusting findings again.
Crucially, AISLE reasons about your code in the context of your systems, so its findings reflect actual reachability with your guardrails taken into account. That same reasoning enables AISLE to go beyond detection: prioritizing findings, proposing fixes, and verifying that new patches don't break anything downstream.
After all, mere detection doesn't itself improve security. Remediation does.
And yes, there are still limits. The same Berkeley map that shows the breadth of AI-native discovery also keeps a “missed by AI” table: classes of vulnerabilities that agentic discovery has not yet fully penetrated. That table is not empty, meaning that some classes may still be harder for AI systems to discover reliably. For example, CWE-36, Absolute Path Traversal, and CWE-552, Public File Access, remain relatively infrequent in AI-native findings.
But that does not rescue traditional SAST. Many of these same classes are almost nonexistent in traditional SAST detection as well. More importantly, the frontier capability of AI-based reasoning systems advances every time the model or harness improves. The ceiling is moving, so much so that fully on-prem open-weights models now rival the frontier, and without the geopolitical baggage.
See What AI Finds in Your Code with AISLE Snapshot
Just in the last year, the ceiling moved dramatically. Now, the ballgame is no longer about which scanner ships the most rules. Rather, it comes down to whose system can reason the most carefully about code.
To see what AI finds in your code, get an AISLE Snapshot. This one-time code audit delivers the industry’s leading detection engine into your deployment environment, from air-gapped networks to the cloud. Get your Snapshot.