Escape AI Pentesting Cascade v2: wider coverage, deeper exploits, less noise

Three months ago, we changed our approach to AI pentesting and introduced Cascade, our multi-agent pentesting product that keeps memory of engagements relevant to your business.

Since then, we've watched it run hundreds of times with our partners, and worked relentlessly to improve it in the areas that matter: wider coverage of your application, deeper understanding of its inner workings, and a lower rate of false alarms.

Today we're proud to be releasing a new version.
Here is what it brings to the table for your security program.

We stopped hoping the agents would look everywhere

As you might remember from our previous blogpost, one of the main reasons AI pentesting systems underperform is a rather trivial one: they fail to exhaustively explore your application. This stems from a genuine tension between the need to start attacking the surfaces you already found, and the need to keep looking for new attack vectors, the classic exploration-exploitation dilemma.

Solving it completely would require major scientific achievements. But redesigning our orchestration system around surfaces makes it possible to leverage knowledge of the application surface to its fullest extent.

In practice, this means that in white-box mode we ensure every file in your application has been examined, while in black-box mode we leverage sitemaps and API specs to find surfaces that even the best explorer agents would have missed.

This improved coverage system improves black-box API coverage from 60% to 90%, while white-box mode goes from 80% of files examined to a guaranteed 100%.

The harness is the product, not the model

Recently, it has become apparent that even non-flagship models can saturate existing benchmarks simply by being plugged into a high-quality, security-oriented harness. That is mostly due to the importance of agentic tool-use and security topics have gained in model post-training. It should also be kept in mind that as soon as a benchmark is made public, it becomes training data for the next wave of models and therefore much more easily saturated. That's part of why we rotate the underlying applications between benchmarking rounds: comparing V2 against a target the model may have already seen would flatter the new version instead of testing it.

To give state-of-the-art models room to show their strength, and to properly assess the impact of harness improvements, it became necessary to develop our own internal benchmarking systems. This is no easy feat.

As recent events like the OpenAI × Hugging Face incident highlighted, these models are getting so good at their job that they start hacking not only the target webapps they are evaluated on, but the evaluation system itself. The sandbox is the only thing ensuring your results are not contaminated, and that you're not putting others at risk; it should never be an afterthought. We designed them to be as varied as possible, so we're sure they represent the cases Escape AI pentesting meets in production:

• Different scales of codebase. Some targets are realistic, enterprise-scale applications with needle-in-a-haystack bugs; others are more reasonably sized and focus the evaluation lens on execution depth rather than coverage.
• Open-source and purpose-built. Forks of existing open-source applications (Web Apps and APIs) alongside applications we built ourselves, so no contamination from the LLM training set is possible.
• Injected vulnerabilities and real CVEs. Injections span the difficulty range, from simple reflected XSS we expect most agents to find to multi-step exploits requiring both code and business-logic understanding. Others were never injected at all: they are CVEs that surfaced between our forking and our benchmarking.
• Kept evolving. Benchmarks have to stay alive, moving with each generation of models and with what the security community learns. It's impossible to predict what exploit modes will appear, so we keep monitoring and rising to the occasion.

Under this design philosophy, we ended up with benchmarks that even the most competent models were not able to fully saturate, while still showing signal for lower competency models and allowing measurable improvements not only in what the agents find, but in how reliably they find it and how few false alarms they raise.

Here is what that gives, target by target, between V1 and V2. Critical and informational findings are what you want to go up; noise is what you want to go down.

Black-box Cascade delivers serious depth on its own, which is what most customer environments need it to do. That depth just grew: V2 now finds in black-box mode what V1 needed white-box access to find, closing a 30% gap. White-box still goes further when you can give it the access, but black-box is no longer the compromise it used to be.

Three layers between an agent's hunch and your inbox

As you might have noticed in your own interactions with them, LLMs tend to be overdramatic when security is involved. What that means for you is that any single imperfect configuration, even one that isn't really exploitable, gets treated as a high-priority signal that deserves your attention.

That is not sustainable. AI-alert fatigue will settle in, making you more likely to miss the actual exploits that need remediation. That's why we have put so much work into false-positive reduction since our initial release, which led us to our current three-layer architecture:

Each of these layers is pulling its weight:

• The reproducer confirms that what the agent observed was really a consequence of its actions as it understood them, not just a coincidence.
• The adversarial reviewer makes sure that what is reported is an actual vulnerability, not a false alarm or an already known finding.
• The business-logic reviewer leverages the business-knowledge database built during the engagement to retrospectively filter findings that, while being real security concerns, are necessary for the intended behaviour of the app.

Combined, they reduce false-positive findings by a factor of three. The share of high-quality findings among the issues Escape's AI Pentesting reports is now above 85%.

A pentest that keeps up with your releases

The new version of Escape's AI pentesting covers more of your application, goes deeper into it, and reports less of what you don't need to read. The three improvements reinforce each other: coverage gives the agents somewhere to go, the harness lets them prove what they find, and the rejection layers keep the proof burden on us rather than on you.

Next: sharpening that continuous loop and focusing agents on what changed since the last run, teaching them to chain exploits so we can unearth even more complex vulnerabilities.

Get started

None of this matters until it's pointed at your own application. Book a session with our team and we'll run Cascade against an environment that looks like yours, white-box or black-box, so you see real, exploitable findings instead of a demo script.

Learn more about Cascade: