What auditing 1000 endpoints told us about GraphQL Security Best Practices

This post is a write-up of a talk I gave at the GraphQL San Francisco Conference. You can find the video on youtube. If you are in the bay area do not forget to also join the GraphQL SF group on meetup.

It's commonly said that GraphQL is less secure than REST by default.

As a company specialized in GraphQL Security, we have explored its security implications for more than a year on a wide range of production-grade endpoints.

In this talk, we use data from our free tool GraphQL.Security to explore what are the best practices for making GraphQL as secure, if not more, than REST.

About GraphQL.Security

GraphQL.Security is a free and simple online security checker for GraphQL. It quickly checks for 18 common security best practices on any GraphQL endpoint.

It has been used by thousands of developers to quickly check the security of their endpoints, giving our team at Escape - GraphQL Security unique insights about the state of GraphQL Security.

The State of GraphQL Security

The following study has been done on a subset of 1000 endpoints scanned by GraphQL.Security over the last months. GraphQL.Security can only test endpoints that do not block all unauthenticated requests.

NB: Escape's full Live GraphQL Security Platform has authenticated endpoints scanning capabilities.

GraphQL.Security leverages our open source package GraphDNA to fingerprint the used engines, so we also have a unique insight into the market shares of each engine.

Not so surprisingly, Apollo is by far the most used engine here. Interestingly, GraphQLPHP is also quite used!

In total, we found 5869 security alerts among the tested endpoints, an average of 6 vulnerabilities per endpoint!

What were those endpoints vulnerable to?

• About 95% of endpoints still had HTTP level misconfigurations and potential CSRFs
• 80% of endpoints were vulnerable to Denial of Service or Complexity based attacks
• About 20% of endpoints were leaking sensitive information

Let's take a little break here: it looks like a lack of security is still endemic in the GraphQL ecosystem.

What are the GraphQL Security Best Practices and how to implement them?

I - Start with the basics: HTTP Layer and CSRF

It might look obvious and boring, but almost every production endpoint still lack proper security at the HTTP Level, and some are vulnerable to Cross-Site Request Forgery which could be easily available. Let's see how 👉

a) Set the right security headers

• Activate the Strict-Transport-Security header to avoid request hijacking in non-secure environments. Even if redirection to HTTPS is activated.
• Enforce Content-Type: application/json. multipart/form-data or
  application/x-www-form-urlencoded should not be accepted.
• If your endpoint is not public, be sure that your Access-Control-Allow-Origin header is not set to “*”

b) Never allow mutations using GET requests

Contrarily to REST endpoints, GraphQL represents fetching and changing data using Queries and Mutations instead of GET's and POST's. Some endpoints forget to enforce HTTP status code to match Queries and mutations and allow Mutations through GET requests.

This is a bad practice, both from a development and a security point of view.

First, GET requests should be safe, idempotent, and cacheable per the HTTP specification. This is not compatible with mutations.

Secondly, when combined with Cookie authentication, it can lead to critical Cross-Site Request Forgery, when a malicious attacker can trick any of your users to execute a custom mutation just by clicking on a link.

II - Deeper into the Graph: Avoiding abusive queries

The anatomy of a GrapHQL Query

GraphQL is quite a full-featured language. Its queries have Operations, Fields, Directives, Parameters, Aliases, Batches, and Fragments. They have variable Width and Depth.

In fact, limits must be set for everything 🔒

Every single semantic token in a GraphQL Query can be abused to create a Denial of Service at the parser, or resolver level. Or even worse.

a) Limit Batching and Aliasing

GraphQL Batching is not part of the GraphQL specification itself. It's just a feature of some engines, including Apollo Server, that can process multiple GraphQL Operations sent in one single HTTP call.

GraphQL Aliasing on the other hand is quite equivalent but part of the GraphQL specification itself, thus supported by all engines. It allows to query the same field multiple times with different parameters using aliases.

The problem? Batching and Aliasing both can be used to bypass rate limiting at the HTTP level. In particular, if login is achieved through a mutation, this can open the door for bruteforce attacks.

The solution (Apollo Server):

Deactivate Query Batching

const server = new ApolloServer({
  ...
  allowBatchedHttpRequests: false,
)}

Limit Query Aliasing

It's better not to deactivate aliasing completely but rather to limit overall request complexity. graphql-validation-complexity is a nice package that can help you do that quickly.

b) Limit Query Depth

The problem? Deep queries can be used to create arbitrary resource intensive operations.

query IAmEvil {
  author(id: "abc") {
    posts {
      author {
        posts {
          author {
            posts {
              author {
                # that could go on as deep as the client wants!
              }
            }
          }
        }
      }
    }
  }
}

Solution (Apollo Server)

You can easily limit the depth of queries with the very light graphql-depth-limit library.

Check how deep you expect legitimate queries to be and set a maximum depth accordingly.

import depthLimit from 'graphql-depth-limit'

const server = new ApolloServer({
  ...
  validationRules: [depthLimit(5)]
  });

c) Limit Query Complexity

The problem? Unlimited query complexity can be used to quickly gain access to a lot of data, and create denial of service.

Solution (Apollo Server)
Limit the complexity of queries to mitigate the abuse of GraphQL resources.

You can use a module to compute the complexity of each query and set a threshold on this complexity so that too broad requests are canceled.

To do so, you can either go with the simple to use graphql-validation-complexity module, which doesn't need you to modify your schema.

import { createComplexityLimitRule } from 'graphql-validation-complexity';

const ComplexityLimitRule = createComplexityLimitRule(1000);

const apolloServer = new ApolloServer({
    ...
    validationRules: [ComplexityLimitRule],
});

To get more information about complexity estimation, read : Securing Your GraphQL API from Malicious Queries

III - Avoiding Information Disclosure

a) Deactivate Debug Mode in Production

The Problem? Stack Traces give too much information on what’s vulnerable in your app👀

Here, in this real life example, a stack trace in production give us information on the used version of Apollo-Gateway, which contains a publicly known vulnerability. Vulnerabilities databases like the CVE database make it very easy for malicious users to correlate stack traces with vulnerabilities.

How to avoid stack traces in production?

Always avoid database or code error stacktrace to be directly returned to the client.

When using Apollo, you can disable exception tracebacks in the response by either setting NODE_ENV to production or enforcing it:

const server = new ApolloServer({
  ...,
  debug: false
)}

For more details, see Apollo's documentation on omitting stacktraces (opens new window).

b) No Introspection? Disable Field Suggestion

The problem? Endpoints with disabled introspection generally still leak the full Schema through field suggestion.

Several engines, like Apollo Server, will send back an error message with suggestions when a field is entered with a typo:

Unfortunately, this can be leveraged to build back any introspection schema even when introspection is deactivated. An open source tool, Clairvoyance, even exists exactly for that purpose.

At Escape, we have even found companies that were relying on disabling introspection only for protecting their admin endpoints. Yes, you heard me. Those were NOT AUTHENTIFIED, the developers thought that nobody could find them because introspection was deactivated.

But using Clairvoyance, we were able to quickly build back the full introspection, noticed an `update_admin` mutation that was unprotected, and were able to actually change the admin password.

How to avoid leaking introspection through field suggestions?

As of June 2022, Apollo has no option for deactivating field suggestions. A possible solution is to rewrite errors using a custom formatError middleware when in production.

const apolloServer = new ApolloServer({
  typeDefs: globalTypeDefs,
  resolvers: globalResolvers,
  ...
  // Mask errors from clients to mitigate security attacks
  formatError: (err) => {
    if (process.env.APP_ENV === 'production') {
      captureException(err);
      return new Error('Internal Server Error');
    }
    return err;
  },
});

Note: Escape's team is currently working on a plugin to solve that problem in a more convenient way.

Let's recap our best practices so far

1. Start with the basics: HTTP layer and CSRF
   a. Set Strict-Transport-Security, Content-Type, and Access-Control-Allow-Origin headers
   properly
   b. Never allow mutations with GET requests
2. Avoid abusive queries
   a. Limit query batching and aliasing
   b. Limit query depth
   c. Limit query complexity
3. Avoid Information Disclosure
   a. Disable stack traces / debug mode in production
   b. If introspection is deactivated, also deactivate field suggestion

Going beyond best practices: Deep Graph Security Testing

At Escape, we understood that nice best practices weren't enough for making GraphQL as secure as REST.

Especially, good security testing capabilities are needed to be sure all resolvers, even deep in the graph, work properly from a security point of view.

But legacy API security testing is based on linear sequences of requests, which is not suitable for the non-linear nature of GraphQL.

We decided to create a whole new technical approach for testing the security of GraphQL Endpoints: Deep Graph Security Testing.

Instead of relying on a linear sequence of Requests, Deep Graph Security Testing dynamically explores the query graph and tests all the resolvers for:

• Data leaks
• Access control flaws
• Injections in resolvers, even deeply nested ones
• 40+ other kinds of vulnerabilities

Deep Graph Security Testing happens in two phases:
1) Our tool identifies the sensitive fields or objects inside the specification

2) Our Graph Exploration Algorithm explores all the paths leading to all fields, testing all resolvers, even deeply nested ones.

What have we found with that?

So far, hundreds of companies are already using Deep Graph Testing through Escape's platform.

Here is an example of what we have found:

Delivery, inc. (name has been changed), an uber eats competitor, was using a GraphQL Schema generator. It generated the GraphQL Schema using the database schema.

They were also logging users' login events in a specific table of their DB for analytics.

Sadly, this table also became accessible deeply in the GraphQL Schema under a userLoginHistory query, due to it being generated from the database. And even worse, this table actually contained the hashed password of the user that logged in.

This vulnerability must not be overlooked as it is striking how many companies actually had critical security flaws in their GraphQL endpoints!

In fact, here is a non-exhaustive list of critical stuff found using Deep Graph Testing:

• JWT Tokens
• GCP Tokens
• AWS keys
• Personal Identifiable Information (Private phone numbers, Social Security Numbers)
• SQL/NoSQL Injections
• Database enumeration (IDORS)

Yes. That is scary. And that's why good security testing is a must.

Conclusion

By default, GraphQL is less secure than most REST implementations. Denial of Service, Cross-Site Request Forgery, and Sensitive Data Leaks are the most common flaws in GraphQL Endpoints.

Implementing Security Best Practices is a good start, but they are not enough to ship production-ready applications. Since GraphQL is new and lacks just lacked the proper tooling, many development teams are just skipping security...

That's why at Escape, we created the first GraphQL Security scanner that allows developers to find and fix vulnerabilities in GraphQL applications during the development lifecycle before they even reach production!

It understands the business logic of GraphQL APIs and looks for more than 50 kinds of vulnerabilities so that you never have to worry again about:

• Resolver performance (N+1 issues, cyclic queries, query complexity DOS…)
• Tenant isolation (access control, data segregation between users…)
• Sensitive data leaks (personally identifiable information, tokens, stack traces, secrets…)
• Injections (SQL, NoSQL, XSS…) and requests forgery
• … and more than 50+ advanced security issues!

We constantly update our engine with state-of-the-art GraphQL Security research so that you never have to worry about the security of your GraphQL application again!

Start monitoring the security of your endpoints today with a 7-day free trial.
Get results in one minute – no credit card required.

Wanna know more about automated GraphQL security testing? Read our blog article "How to test your GraphQL API?".