What is a Shadow API? Understanding the risks and strategies to prevent their sprawl
API Sprawl is a serious challenge in 2024, and shadow APIs are part of the problem. But do you actually know what Shadow APIs are and why they represent danger for organizations?
With the ever-increasing number of exposed APIs, the pain of not being able to gain visibility over all of them is creeping up on security engineers. Unfortunately, it's all too common to have APIs in staging and production that no one on your development or security teams remembers or knows about. And if you don't know what's exposed and what those APIs do, you can't secure them.
Roughly 5 billion (31%) of the 16.7 billion malicious requests observed in the report by CQ Prime Threat Research team targeted unknown, unmanaged, and unprotected APIs, spanning a wide range of use cases.
Imagine one day a hacker gains access to the leaked API key or token (like we did at Escape as part of the API Secret Sprawl research) and then gets access to your APIs that you didn't even know existed and weren't properly secured.
And then you wake up like Facebook with the need to pay a $15,000 bounty. This vulnerability was discovered within a niche API, which reminds us that sometimes important vulnerabilities might exist not within primary APIs but in secondary ones, which may have fewer protective measures in place.
In this article, we will dive into details of what shadow APIs are, why they are dangerous, and strategies to detect and control their sprawl.
What is a Shadow API?
In simple terms, a shadow API refers to an API that exists outside the visibility and control of an organization's IT department. Unlike regular APIs that are registered, governed, and protected by security controls, shadow APIs operate under the radar. They are often created by developers for various reasons, such as quickly fulfilling project needs or bridging the gap between applications lacking official APIs. However, their unregulated nature makes them attractive targets for malicious actors.
Understanding the risks of Shadow APIs
Shadow APIs pose significant risks to organizations. First, they lack proper authentication and access controls, making them vulnerable to unauthorized access (hello, BOLA) and data breaches. Attackers can exploit these APIs to gather sensitive information and plan their attacks without detection. Moreover, as shadow APIs operate outside the purview of security monitoring solutions, any malicious activity can go unnoticed until significant damage has been done. This can result in costly data breaches, system outages, and non-compliance with privacy regulations, which are, in 2024, getting more and more stringent like PCI DSS 4.0.
In 2023 we used Escape’s in-house API scanner to crawl 5138 public APIs on the internet. An outstanding amount of APIs had minor security misconfiguration, but up to 12 percent of APIs had more serious vulnerabilities, and around half a percent had critical ones.
Considering that we ran only a passive scan, in the wild, and with no authentication provided, we consider that those results raise concerns about how many less-known public APIs are easily exploitable by malicious actors.
How to spot Shadow APIs
Detecting shadow APIs within an organization's software environment is crucial for effective risk mitigation. Several practices can help identify suspicious API activity:
Automate API discovery and inventory
The most effective method for gaining control over all APIs within your organization is by implementing continuous API discovery and inventory. For all your API types. The solution that automatically discovers and catalogs all APIs, along with identifying the potential risks they present to the organization, is your best bet. That's what we do at Escape as well. You can find the demo on YouTube.
Automate API documentation
You can't have the ghosts lurking in the shadows if everything you and your developers do is properly documented. That's the ideal world but easier said than done. If developers need to log all data manually, it's a waste of time and it'd be impossible to motivate everyone on the team.
Here's a quick example of how you can automate API documentation if you're building with Flask, using tools like Swagger (OpenAPI) and tools like Swagger UI or ReDoc:
from flask import Flask
from flasgger import Swagger
app = Flask(__name__)
swagger = Swagger(app)
@app.route('/hello/<string:name>/', methods=['GET'])
def hello_world(name):
"""
This is a sample endpoint which says hello
---
tags:
- Hello
parameters:
- name: name
in: path
type: string
required: true
description: The name to say hello to
responses:
200:
description: A simple hello message
schema:
type: string
"""
return 'Hello, {}!'.format(name)
if __name__ == '__main__':
app.run(debug=True)
In this example, we're using Flasgger, a Flask extension that integrates Swagger for automatic API documentation generation.
Once you run this Flask application, you can navigate to http://localhost:5000/apidocs to see the Swagger UI interface where your API documentation will be automatically generated based on the docstrings provided in your Flask routes.
You can customize and expand on this example to suit the needs of your API documentation.
Monitoring outbound proxies
Another way to spot shadow APIs is by monitoring outbound proxies. These proxies intercept all outgoing API calls and record detailed logs of requests and responses. By analyzing these logs, security teams can identify unrecognized API transactions, unusual response times, and unexpected resource allocations that may indicate the presence of shadow APIs.
Leveraging log analysis
Implementing capable logging tools allows organizations to closely monitor endpoints, APIs, and response data. Regularly reviewing and analyzing logs can help identify suspicious API activity. By evaluating end-to-end application performance and looking for patterns or anomalies in the logs, security teams can detect the usage of shadow APIs.
Implementing live monitoring
Live monitoring tools or tactics enable organizations to identify API calls as they occur. By implementing live monitoring at the API gateway level, security teams can automatically detect and flag any unauthorized or unusual API activity in real time. This proactive approach aids in the early detection and prevention of shadow API attacks.
Scanning source code
To address shadow APIs before they reach production, organizations can scan their source code for any traces of unauthorized APIs. By incorporating static source code analysis with SAST tools into their security testing processes, security teams can identify and eliminate shadow APIs during the development phase. Regular code scanning helps maintain API integrity and reduces the risk of unregulated APIs.
Preventing Shadow API attacks
In addition to detecting shadow APIs, organizations should implement preventive measures to mitigate the risks associated with these unregulated ghosts. Here are some strategies to consider:
Implementing API governance
Establishing a robust API governance framework ensures that all APIs, including shadow APIs, are properly managed and controlled. This involves creating policies and procedures for API development, deployment, and usage. By enforcing API governance, organizations can minimize the Shadow API sprawl and ensure adherence to security standards.
I think it's mostly just showing that we have a scaling problem. I don't think that the problem is so much that we have too many APIs. I think is that our governance and management practices haven't quite caught up, - Erik Wilde on the fireside chat on API Sprawl
Regular security audits
Conducting regular application security audits is essential to identify any vulnerabilities or weaknesses in an organization's API ecosystem. By performing comprehensive security assessments, including vulnerability scanning and penetration testing, IT teams can identify and address potential entry points for shadow API attacks. Regular audits help maintain the security posture of APIs and prevent unauthorized access.
Educating developers
Developers play a crucial role in preventing the spread of shadow APIs. By educating developers about the risks associated with unregulated APIs and providing them with approved alternatives and best practices, organizations can foster a culture of API security awareness. Regular security training sessions and knowledge sharing help developers make informed decisions and minimize the use of shadow APIs.
Conclusion
Let's wrap things up on a positive note! Shadow APIs are a challenging problem, but if you've got the right tools and processes, you'll be able to keep them in check.
By staying on top of things with cool monitoring practices like automated API discovery, documentation, and keeping an eye on outbound proxies and logs, we can spot those sneaky APIs before they cause any real harm.
You can stay on top of your security game, by implementing effective monitoring practices, such as automated API discovery and inventory, automated API documentation, monitoring outbound proxies, leveraging log analysis, implementing live monitoring, and scanning source code.
Additionally, preventive measures like API governance, regular security audits, and developer education will play a vital role in minimizing the usage of unregulated APIs in your organization.
By taking a proactive approach to API security, you can protect your organization's systems, data, and reputation from the dangers posed by shadow APIs!
💡 Want to learn more? Check out the following articles: