Methodology: How we discovered over 2k high-impact vulnerabilities in apps built with vibe coding platforms

Hey there,

With Halloween around the corner, what’s scarier for organizations than vulnerabilities in their web applications?

And it's even scarier when the development of these applications is in the hands of users not familiar with security practices.

This year, the Escape research team has focused on a growing area of concern: vibe-coded applications. Purpose-built platforms like Lovable.dev, Base44.com, and Create.xyz have democratized application development, enabling non-developers to deploy full-stack applications without writing a single line of code. Just ask your colleagues on the marketing team what they've already deployed or experimented with under the hood! ;)

As vibe-coded applications become more accessible, inexperienced users are creating fresh risks.

Our research team analyzed over 5,600 publicly available applications and identified more than 2,000 vulnerabilities, 400+ exposed secrets, and 175 instances of PII (including medical records, IBANs, phone numbers, and emails).

Unlike other published articles on that topic, the goal of this research was to move beyond isolated case studies by identifying issues at scale that would otherwise require hours of manual work to uncover.

You can review the complete results in our comprehensive report. Meanwhile, in this article, we'll show you the methodology that guided us to these impressive findings.

Methodology

Data Gathering Strategy

To collect and analyze our dataset, multiple domain sources were leveraged, including official launch directories (e.g., launched.lovable.dev), Shodan indexing, Reddit communities, and manual crawling.

First, we retrieved data from launched.lovable.dev to compile a dataset of 4,000 web applications. We then expanded our target set using additional sources, including lovable.dev, base44.com, vibe-studio.ai, bolt.new, create.xyz. Afterward, we performed subdomain enumeration on these domains to identify additional targets.

Following a cross-source analysis of the aggregated dataset, we derived three independent fingerprinting methods for detecting Lovable-based web apps:

After developing a fingerprint for the Lovable web app, we used Shodan to locate live instances of web apps that appear to be implemented with Lovable. To further augment the dataset, we scraped curated posts and comment threads from the Reddit communities r/lovable and r/base44.

The resulting URLs were curated through a multi-stage process:

• Deduplication to remove redundant entries
• Automated reachability checks to exclude dead hosts: we filtered live assets by checking whether each main page returned an HTTP status code in the 200–399 range
• Filtering to distinguish between non-application landing pages and functioning deployments.

Initial platform coverage included Lovable (~4,000+ applications discovered), Base44 (~159), Create.xyz (~449), Vibe Studio, and Bolt.new (smaller samples).

The data collection was conducted as a one-time process. During collection, several limitations were deliberately imposed to ensure legal and ethical compliance. Domains that could be reasonably identified as educational or health-related were excluded, as typical users are not authorized to probe such systems. This exclusion reduced coverage but ensured adherence to established ethical norms of web crawling and security research.

We acknowledge several sources of potential bias in our methodology:

• Sampling bias. Reliance on launch directories, Shodan, and community postings may overrepresent applications that are actively promoted or easily discoverable, while underrepresenting private or restricted deployments.
• Temporal bias. As data collection was performed once, the dataset represents a snapshot in time. Applications and platforms evolve rapidly; vulnerabilities may have been patched or newly introduced since collection.
• Platform imbalance. The dataset is heavily skewed toward Lovable deployments, with substantially fewer applications discovered for Base44, Create.xyz, Vibe Studio, and Bolt.new. This imbalance may disproportionately influence prevalence measurements.

At the same time, focusing on domains openly accessible to anyone online gives us a useful window into how these applications are actually built and used in practice. This approach highlights the security habits (and mistakes) that most often appear in real deployments, especially when apps are created by people with little security background.

Looking at this public-facing slice of the ecosystem helps us see not just isolated flaws, but broader patterns across sectors.

Attack Surface Scanning

After the dataset was curated, the next stage of the methodology focused on systematically mapping the attack surface of each application, i.e., extracting all hosts, web apps, and APIs exposed by the domains we found (further defined as “assets” fed into Escape’s Attack Surface Management scanner). Our goal was to build a structured model of each application’s externally visible footprint and then subject high-value pieces of that footprint to dynamic testing.

As a platform, Escape is a collection of security scanners. A typical Escape’s ASM scanner is a tool that automates the identification of all exposed assets, correlates them, and helps prioritize which ones are most likely to be exploited. Its scanner structure can be seen as follows:

The scanner first ingests the “assets”. In our case, these assets include hosts, web apps, APIs, and schemas. The scanner then proceeds through a multi-step process of validation and reachability checks, followed by fingerprinting and metadata collection (such as WAF, cloud provider, framework, and GeoIP). This process ensures that only valid, accessible assets are mapped and ready for further testing.

Discovery techniques

Once the assets are identified, they are classified into the discovery phase. We relied on a layered discovery strategy to maximize coverage while minimizing intrusiveness:

• Domain and host discovery. Subdomain enumeration and passive indices were used to enumerate hosts associated with each base domain. As mentioned before, we validated reachability via benign HTTP(s) probes and excluded hosts that returned only generic landing pages.
• Web crawling and route enumeration. A headless-browser crawler rendered pages and followed links, collecting URL structures, JS bundles, and client-side routing artifacts.
• Static frontend analysis. JavaScript and HTML were parsed to extract embedded API endpoints, fetch/XHR/WebSocket URLs, inline tokens or keys (when present in the public bundle), and configuration objects that reveal backend schemas or third-party integrations.
• API discovery. Endpoints discovered in JS, observed in network traces during crawl sessions, or exposed via documented routes were collated into service models. When available, open schema fragments (e.g., JSON responses illustrating resource shapes) were used to infer parameterization and access control points. We also performed API discovery by brute-forcing API paths, i.e., testing for common paths at scale and identifying API-like responses.

By feeding the data collected during the discovery process into the scanner, we built a comprehensive, continuously updated map of each application's publicly visible footprint.

While analyzing the structure of specifically Lovable websites, we came across the integration of Lovable and Supabase. In this structure, we specifically identified and targeted APIs integral to the application’s functionality that could be discovered and analyzed at scale.

During our analysis, we also discovered that anonymous JWT tokens were exposed in the JavaScript bundles of the Lovable front end. These tokens were linked to PostgREST APIs as part of the Supabase backend integration. According to the documentation, Supabase “automatically generates a RESTful API from the database schema, allowing applications to interact with the PostgreSQL database through an interface, all from the browser.”

However, while Supabase's default security rules are permissive for development, they leave important security gaps if not properly configured before going live. Specifically, Row-Level Security (RLS) policies must be implemented to ensure that only authorized users can access or modify specific rows in the database, such as ensuring that users can access only their own data. The issue arises when RLS is misconfigured between the API layer and the database. This creates security risks, as unauthorized access could occur if JWT tokens (used for authentication) are exposed in the frontend code.

In our previous research, we explored how sensitive information, such as API tokens and secrets, was exposed in JavaScript bundles, an issue we've encountered again in this context.

Therefore, while Lovable can assist in generating RLS policies, it is vital for users to manually review these policies (which can be a challenge for less experienced “vibe coders”).

Introduction of Lightweight DAST Surface Scanning or "Visage Surface Scanner"

Given the structure of the integration between Lovable front-ends and Supabase backends via API, and the fact that certain high-value signals (exposed keys, for example, anonymous JWTs to APIs linking Supabase backends, client-side routes, embedded endpoints) only appear in frontend bundles or source output, we introduced a lightweight, read-only scan to harvest these artifacts and feed them back into the ASM inventory.

We called this scanner the Visage Surface Scanner. Unlike previous versions, this scanner is less in-depth: it doesn't execute any actions on the web application or trigger processes. Instead, it analyzes source code and frontend responses to identify secrets or routes that can be added to the asset inventory in our Attack Surface Management tool through a feedback loop.

The Visage Surface scanner was integrated into the Attack Surface Management (ASM) web app scanner, enabling us to scan Lovable web apps for vulnerabilities and identify exposed anonymous JWT tokens and Supabase API routes. These findings were then fed into the Escape API-focused Business Logic Security Testing Scanner, where they were analyzed for real-world security issues:

After adding the discovered URLs to the ASM and running the Visage Surface Scan, we now discovered in total 14,600 assets in the ASM, composed of:

• 5,600 web applications
• 1,280 API services
• 6,500 hosts
• 1,103 schemas (found and generated via frontend traffic)

Surface extraction & modeling

These assets were then filtered and organized into an Attack Surface Management (ASM) per application within Escape’s platform that included:

• Hosts and subdomains
• Web application entry points and client routes
• REST/GraphQL/WebSocket endpoints and their observed request/response shapes
• Authentication and session management endpoints
• Third-party integrations and service endpoints (e.g., Supabase, analytics, storage)

Security Testing and Dynamic Analysis

Once the attack surface was extracted and modeled, we applied targeted security testing using in-house dynamic application security testing (DAST) techniques (see more info on the web app scanner here and the API scanner here). The objective was not to exhaustively exploit weaknesses, but to identify recurring classes of misconfigurations and vulnerabilities in a safe, controlled manner.

We ran DAST scans in a “passive” mode, explicitly configured to avoid destructive operations, high-volume brute force attempts, or payloads that could disrupt target services. This design choice was made to respect ethical and legal boundaries and to minimize unintended impact on live deployments. While this conservative approach ensures safety, it also introduces an important bias: the results presented here are lower-bound estimates. Running Escape’s full scanning capabilities (e.g., injection payloads, deeper fuzzing, and aggressive brute-force) would almost certainly surface a larger volume of issues, including higher-severity vulnerabilities.

All REST API endpoints passed our REST DAST scan, using the schema produced by the Visage Surface scanner and the credentials stored in the web application. We attempted to implement an automated registration agent for the web application to provision an account, execute a comprehensive scan, and forward the resulting authentication token to the REST DAST tool.

Observations and Biases

Two important observations shaped the testing results:

• Most vulnerabilities were exposed without authentication

Across platforms, critical weaknesses (e.g., exposed Supabase tokens, misconfigured APIs, and missing row-level security) were accessible directly through public endpoints. Tokens such as Supabase service keys were often trivially retrievable from frontend bundles, underscoring that many security issues in vibe-coded apps exist “in the open,” without requiring any privileged access. If we decided to go more in-depth, we could develop an AI-driven auto-authentication system that leveraged headless browser automation and agent-based orchestration.

2. Results understate the true risk

Because scans were run in a passive mode, the findings reflect only a subset of exploitable issues. A more aggressive testing configuration would likely have uncovered additional vulnerabilities with greater impact. In this sense, our findings represent a conservative baseline rather than the full extent of the security risks present in these ecosystems.

Data Cleanup and Verification

After running the ASM-driven DAST scans, the raw output contained a large volume of findings with the Escape platform, ranging from high-confidence exposures to some noisy signals. To ensure that only meaningful vulnerabilities and secret disclosures were included in our analysis, we applied a multi-stage cleanup and verification process.

Deduplication and Normalization

Deduplication and normalization were performed automatically by the Escape platform, which consolidated findings across multiple discovery vectors.

Verification of Exposed Secrets

• Pattern-based filtering: Escape applied platform-specific rules to distinguish genuine credentials (e.g., API keys, Supabase tokens, environment variables) from placeholders or noise.
• False-positive reduction: Values resembling generic identifiers (UUIDs, hashes, opaque IDs) were automatically flagged and excluded if not usable as credentials.
• Safe live validation: Where legally and ethically permissible, exposed tokens were tested against non-destructive requests to public endpoints to verify validity. Tokens granting elevated privileges (particularly Supabase service role keys) were flagged as critical exposures due to the level of access they provided.

Vulnerability Validation

• Vulnerabilities suggesting missing or weak authentication were validated by replaying requests without tokens or with modified headers to confirm whether access restrictions were enforced.
• Manual spot-checks. A representative subset of findings was manually validated to assess the precision of automated classification and to calibrate severity scoring.

Conservative Scope

It is important to note that the cleanup and verification process was intentionally conservative. Only findings that could be confirmed with high confidence were retained. As a result, the vulnerabilities and exposed secrets presented in this study represent a verified baseline rather than the full universe of potential issues.

Conclusion

Mapping out the attack surface of these vibe-coded applications wasn’t easy, but it was necessary. By taking a comprehensive approach to identifying exposed assets and vulnerabilities, we uncovered risks that would otherwise go unnoticed.

Our study reveals that these vulnerabilities are spreading across diverse websites, industries, and asset types. As the vibe coding phenomenon continues to explode, especially within modern tech industries, the challenge grows. Even though users may be increasingly tech-savvy, many still lack critical security awareness. As a result, the risks are only amplifying.

Organizations must respond quickly to these threats by adopting best practices to protect their assets. The sooner they act, the better.

Not sure where to start? We've compiled the comprehensive list in our report.

Need guidance tailored to your specific needs? Feel free to reach out!