Rate Limiting GraphQL APIs by Calculating Query Complexity

Rate limiting is a system that protects the stability of APIs. GraphQL opens new possibilities for rate limiting. I’ll show you Shopify’s rate limiting system for the GraphQL Admin API and how it addresses some limitations of common methods commonly used in REST APIs. I’ll show you how we calculate query costs that adapt to the data clients need while providing a more predictable load on servers.

What Is Rate Limiting and Why Do APIs Need It?

To ensure developers have a reliable and stable API, servers need to enforce reasonable API usage. The most common cases that can affect platform performance are

• Bad actors abusing the API by sending too many requests.
• Clients unintentionally sending requests in infinite loops or sending a high number of requests in bursts.

The traditional way of rate limiting APIs is request-based and widely used in REST APIs. Some of them have a fixed rate (that is clients are allowed to make a number of requests per second). The Shopify Admin REST API provides credits that clients spend every time they make a request, and those credits are refilled every second. This allows clients to keep a request pace that never limits the API usage (that is two requests per second) and makes occasional request bursts when needed (that is making 10 requests per second).

Despite widely used, the request-based model has two limitations:

• Clients use the same amount of credits regardless, even if they don’t need all the data in an API response.
• POST, PUT, PATCH and DELETE requests produce side effects that demand more load on servers than GET requests, which only reads existing data. Despite the difference in resource usage, all these requests consume the same amount of credits in the request-based model.

The good news is that we leveraged GraphQL to overcome these limitations and designed a rate limiting model that better reflects the load each request causes on a server.

The Calculated Query Cost Method for GraphQL Admin API Rate Limiting

In the calculated query cost method, clients receive 50 points per second up to a limit of 1,000 points. The main difference from the request-based model is that every GraphQL request has a different cost.

Let’s get started with our approach to challenges faced by the request-based model. 

Defining the Query Cost for Types Based on the Amount of Data it Requests

The server performs static analysis on the GraphQL query before executing it. By identifying each type used in a query, we can calculate its cost.

Objects: One Point

The object is our base unit and worth one point. Objects usually represent a single server-side operation such as a database query or a request to an internal service.

Scalars and Enums: Zero points

You might be wondering, why do scalars and enums have no cost? Scalars are types that return a final value. Some examples of scalar types are strings, integers, IDs, and booleans. Enums is a special kind of scalar that returns one of a predefined set of values. These types live within objects that already have their cost calculated. Querying additional scalars and enums within an object generally comes at a minimum cost.

In this example, shop is an object, costing 1. id, name, timezoneOffsetMinutes, and customerAccountsreturn are scalar types that cost 0. The total query cost is 1.

Connections: Two  Points Plus The Number of Returned Objects

Connections express one-to-many relationships in GraphQL. Shopify uses Relay-compliant connections, meaning they follow some conventions, such as compounding them by using edges, node, cursor, and pageInfo.

The edges object contains the fields describing the one-to-many relationship:

• node: the list of objects returned by the query.
• cursor: our current position on that list.

pageInfo holds the hasPreviousPage and hasNextPage boolean fields that help navigating through the list.

The cost for connections is two plus the number of objects the query expects to return. In this example, a connection that expects to return five objects has a cost of seven points:

cursor and pageInfo come free of charge as they’re the result of the heavy lifting already made by the connection object.

This query costs seven points just like the previous example:

Interfaces and Unions: One point

Interfaces and unions behave as objects that return different types, therefore they cost one point just like objects do.

Mutations: 10 points

Mutations are requests that produce side effects on databases and indexes, and can even trigger webhooks and email notifications. A higher cost is necessary to account for this increased server load so they’re 10 points. 

Getting Query Cost Information in GraphQL Responses

You don’t need to calculate query costs by yourself. The API responses include an extension object that includes the query cost. You can try running a query on Shopify Admin API GraphiQL explorer and see its calculated cost in action.

The request:

The response with the calculated cost displayed by the extension object:

Getting Detailed Query Cost Information in GraphQL Responses

You can get detailed per-field query costs in the extension object by adding the X-GraphQL-Cost-Include-Fields: true header to your request:

Understanding Requested Vs Actual Query Cost

Did you notice two different types of costs on the queries above?

• The requested query cost is calculated before executing the query using static analysis.
• The actual query cost is calculated while we execute the query.

Sometimes the actual cost is smaller than the requested cost. This usually happens when you query for a specific number of records in a connection, but fewer are returned. The good news is that any difference between the requested and actual cost is refunded to the API client.

In this example, we query the first five products with a low inventory. Only one product matches this query, so even though the requested cost is seven, you are only charged for the four points calculated by the actual cost:

Measuring the Effectiveness of the Calculated Query Cost Model

The calculated query complexity and execution time have a linear correlation

By using the query complexity calculation rules, we have a query cost that’s proportional to the server load measured by query execution time. This gives Shopify the predictability needed to scale our infrastructure, giving partners a stable platform for building apps. We can also detect outliers on this correlation and find opportunities for performance optimization.

Rate limiting GraphQL APIs by calculating the amount of data clients query or modify adapts more to the use case of each API client better than a request-based model commonly used by REST APIs.  Our calculated query cost method benefits clients with good API usage because it encourages them to request only the data they need, providing servers with a more predictable load.

Additional Information

Guilherme Vieira is a software developer on the API Patterns team. He loves building tools to help Partners and Shopifolk turn their ideas into products. He grew up a few blocks from a Formula 1 circuit and has been a fan of this sport ever since.

We're planning to DOUBLE our engineering team in 2021 by hiring 2,021 new technical roles (see what we did there?). Our platform handled record-breaking sales over BFCM and commerce isn't slowing down. Help us scale & make commerce better for everyone.