API key leaks are no longer uncommon, and so are subsequent breaches. So why are sensitive tokens still so easily exposed?
To find out, Intruder’s research team investigated what traditional vulnerability scanners actually cover and built a new secret detection method to address gaps in existing approaches.
Applying this at scale by scanning 5 million applications revealed over 42,000 exposed tokens across 334 secret types, exposing a major class of exposed secrets that are not well handled by existing tools, particularly single-page applications (SPAs).
This article details existing secret detection methods and reveals what we found when scanning millions of applications for secrets hidden in JavaScript bundles.
Established secret detection methods (and their limitations)
Traditional secret detection
A traditional, fully automated approach to discovering application secrets is to search a set of known paths and apply a regular expression that matches the known secret format.
Although this method is useful and can detect some leaks, it has obvious limitations and cannot detect all types of leaks, especially those that require application spidering or authentication through scanners.
A good example of this is Nuclei’s GitLab personal access token template. The scanner is supplied with a base URL (e.g. https://portal.intruder.io/) and the template looks like this:
Make an HTTP GET request to https://portal.intruder.io/. Inspect the direct response to that one request, ignoring other pages, JavaScript files, and other resources. Attempts to identify patterns in GitLab personal access tokens. If found, make a follow-up request to GitLab’s public API to check if the token is active. If active, raise the issue.
This is obviously a simple example, but this approach is effective. This is especially true if the template defines many paths through which secrets are publicly exposed.
This format is typical of infrastructure scanners, which typically do not run headless browsers. Once the scanner is given a base URL to scan (such as https://portal.intruder.io), subsequent requests made by the browser (such as JavaScript files needed to render the page, such as https://portal.intruder.io/assets/index-DzChsIZu.js) are not made with this old-fashioned approach.
Dynamic Application Security Testing (DAST)
Dynamic Application Security Testing (DAST) tools are generally a more robust method of scanning applications and tend to have more complex features, allowing full spidering of applications, support for authentication, and broader capabilities to detect application-layer weaknesses. In fact, DAST scanners may seem like a natural option for secret detection on application frontends. There’s nothing to prevent the DAST scanner from discovering available JavaScript files or scanning for secrets within them.
However, this type of scanning is more expensive, requires careful configuration, and in practice is usually reserved for a few high-value applications. For example, DAST scanners are unlikely to be configured for every application that exists across a wide range of digital assets. Additionally, many DAST tools do not implement a sufficient range of regular expressions compared to well-known command-line tools.
This creates an obvious gap that should be covered by traditional infrastructure scanners but is not. It’s also likely that even a DAST scanner won’t cover it due to deployment, budget, and maintenance limitations.
Static Application Security Testing (SAST)
Static Application Security Testing (SAST) tools are the primary method for analyzing source code to identify vulnerabilities and discover secrets before the code reaches production. These are effective at capturing hard-coded credentials and preventing leakage of some classes.
However, we found that the SAST method also did not cover the whole picture. Also, some secrets in the JavaScript bundle were slipping through the cracks in a way that static analysis missed.
Build secret detection checks for JavaScript bundles
At the time this study began, it was not clear how common this problem was. Are secrets actually bundled with JavaScript front ends, and are they widespread enough to justify an automated approach?
To find out, we built automated checks and scanned approximately 5 million applications. The result was a much larger number of exposures than expected. The output file alone was over 100 MB of plain text and contained over 42,000 tokens across 334 different secrets.
Although we did not completely triage all results, we identified a number of high-impact exposures among the samples we reviewed.
what we found
code repository token
The most impactful breaches we identified were tokens for code repository platforms such as GitHub and GitLab. A total of 688 tokens were found, many of which were still active and allowed full access to the repository.
In one case, shown below, a GitLab personal access token was embedded directly into a JavaScript file. The scope of the token was set to allow access to all private repositories within the organization, including CI/CD pipeline secrets for follow-on services such as AWS and SSH.
Project management API key
Another major exposure involved API keys for Linear, a project management application embedded directly in front-end code.
This token exposed your organization’s entire Linear instance, including internal tickets, projects, and links to downstream services and SaaS projects.
Learn more
We’ve identified leaked secrets across a wide range of other services, including:
CAD Software API – Access to user data, project metadata, and building designs including hospitals
Link shortener – ability to create and enumerate links
Email Platform – Access to mailing lists, campaigns, and subscriber data
Webhooks for chat and automation platforms – 213 Slack, 2 Microsoft Teams, 1 Discord, and 98 Zapier, all active
PDF Converter – Access to third-party document generation tools
Sales intelligence and analytics platform – access to scraped company and contact data
don’t reveal secrets
The Shift-Left control is important. SAST, repository scans, and IDE guardrails catch real problems and prevent any class of exposure. However, as this research shows, it does not cover all possible paths for secrets to be introduced into production.
Secrets introduced during build and deployment can bypass these safeguards and be incorporated into front-end code long after the shift left control is already running. And as automation and AI-generated code become more common, this problem will become even bigger.
Therefore, single-page application spidering is required to capture secrets before they reach production. We built automated SPA secret detection into Intruder so teams can actually discover this. learn more.
Source link
