State of GraphQL Gateways in 2023

The GraphQL ecosystem is growing rapidly, and solutions to complex problems are more accessible than ever before.

One of the most common questions about GraphQL is how to extend or enrich it. This can be achieved by adding custom flows, securing your /graphql endpoint, improving performance through caching, orchestrating upstream calls for distributed GraphQL, or monitoring and tracing it.

Either a specific ad-hoc solution or a gateway designed specifically for GraphQL can solve all of the following problems.

In this blog post, we’ll cover The Guild’s (and other companies) six-month journey of researching, exploring, and creating a report on the state of GraphQL gateways in 2023. We compared, tested, and benchmark various open-source solutions, and we’d like to share our process thus far.

But First, What’s a GraphQL Gateway?

A GraphQL gateway follows a workflow that adds functionality and acts as a proxy between the consumer and the actual GraphQL server(s) running the GraphQL schema(s). An architecture diagram of this setup will look like the following:

In the architecture chart above, the GraphQL gateway responsible for exposing your GraphQL server externally and adding capabilities to it. The GraphQL server, on the other hand, implements the actual GraphQL schema and runs resolvers, dataloaders, and any other custom code required for fetching and connecting your data or entities.

Incorporating a GraphQL gateway into your architecture can help to offload some of the common features and capabilities from your GraphQL server. Here’s a (partial) list of things that a GraphQL gateway can do:

1. Caching in GraphQL can be implemented in various ways, but it doesn’t need to be tightly coupled to your schema implementation. A GraphQL gateway can handle caching for the GraphQL server(s) to reduce network traffic to the actual server. This approach also results in a more lightweight and easier-to-maintain GraphQL server implementation.
2. Authentication and authorization can be managed by the GraphQL gateway. If you are using JWT or any other standard, you can verify the token, permissions, and scopes at the gateway level. Then, you can pass the authentication metadata (such as user ID) to the GraphQL server. This way, your GraphQL server does not need to handle authentication and can focus solely on implementing your application’s business logic.
3. When it comes to GraphQL servers, security is also a concern. By placing a GraphQL gateway in front of your GraphQL server, you can easily protect your GraphQL API from malicious queries and guard against common attack vectors.
4. A GraphQL gateway can also help harden your API and prevent the leakage of sensitive information, error details, or even PII.
5. Policy validations are also a popular feature in GraphQL. Things like rate-limiting, depth-limit, or complexity limit can all be implemented in your gateway instead of the server.
6. A GraphQL gateway can also address the lack in modern features like real-time or subscriptions in your GraphQL server. The gateway can expose a real-time network transport, such as WebSocket or SSE, to the consumers while sending a regular HTTP request to the upstream GraphQL server.
7. Schema filtering can also be implemented at the gateway level, allowing for different sets of features, fields, and types to be exposed on each endpoint.
8. And it can also do easy things, like rendering the latest version of GraphiQL for you, with the all the enabled transports configured (WebSocket/SSE/HTTP).

Benchmarking and Comparing GraphQL Gateways

To understand and compare the GraphQL gateway landscape, we began by collecting information about existing solutions. This included:

1. Which distributed GraphQL specifications are supported.
2. How the product/library is used (as code or as a product).
3. What features are supported.

For our current benchmarks setup, we made the following decisions to ensure a fair comparison between GraphQL gateways:

1. We used a distributed GraphQL specification to check the performance of a complex setup that requires the gateway to take a significant part of the execution flow. We chose the Federation (v1) specification since it is widely used and supported by many gateways.
2. We disabled response caching to ensure a complete request flow is executed.
3. We compared responses to ensure that all gateways respond in the same way.
4. We tracked all available vitals and metrics, such as network traffic, CPU, and RAM.
5. We tested different use cases to simulate real-life scenarios with different flows, such as peak times or upstream server delays.
6. We are running all scenarios for every change, and have a complete overview of the results and the stats of every gateway.
7. We are running Rust-based servers for the subgraphs implementation, to ensure it’s never becoming a bottleneck.

For the benchmark, we picked the following gateways:

1. Apollo-Server: a JS/TS library for implementing GraphQL servers or gateways.
2. Apollo-Router: the Rust-based product for running Apollo Federation.
3. Wundergraph: a Go-based GraphQL gateway and platform.
4. GraphQL-Mesh: a JS/TS GraphQL gateway based on GraphQL-Yoga, that also supports anything-to-GraphQL, also supports any other API protocol (REST, OpenAPI, gRPC, SOAP, etc) as subgraphs

In addition to that, we used various runtimes for the JavaScript/TypeScript solutions: Node.js (18 and 20), and Bun.

Also, all scenarios are running within Docker container environment, on a stable runner (a dedicated GitHub Actions runner, that runs 1 concurrent job, to avoid race conditions or missing resources), and limitations for memory and CPU.

This decisions above are subject to change, mainly because we want to introduce more options and more scenarios.

The entire source code is open-source, and we encourage developers to help us with the following:

• Share more use-cases and realistic scenarios; either by changing the specification used, gateways tested, plugins, execution flow or parameters.
• Improve the code of the actual gateways and help to improve the GraphQL ecosystem.

The Numbers

constant-vus-over-time

You can find here the latest results, report and statistics

This is the naive, simplest setup with no additional adjustments; the gateway is running the Federation v1 specification, with a large query (2 top level fields, and 4~7 nested levels, different entities across multiple subgraphs, and some fragments spreads).

• VUs: 300
• Time: 10 minutes
• CPU limit: 2
• Memory limit: 4GB

Here are some insights worth mentioning:

✅ Almost gateways tested were able to process all requests without any failures (side note: when CPU limit is lower, some JS gateways are failing to respond to requests, and the requests are failing with a timeout or connecting drop, as you can see with apollo-server on Node.js 18).

🏆 Apollo-Router and Wundergraph are the fastest gateways, with an average response time of 900ms and a p95 of 2500ms, while under pressure of incoming requests.

🚅 The fastest JS-based gateway is GraphQL-Mesh (almost x2 than other JS-based), and when running on Bun runtime, it’s even faster (x1.5).

📈 The peak memory usage of Apollo-Router was around 400MB, while Wundergraph needed more resources (x3, 1.3GB) to handle the same amount of requests. Memory consumption of GraphQL-Mesh in the test above was 550MB on Bun, and 1.4GB on Node.js 18.

📊 mercurius (Fastify-based) failed to handle ~50 requests.

constant-vus-subgraphs-delay

You can find here the latest results, report and statistics

Same as constant-vus-over-time but with a random delay (20~150ms) for all upstream HTTP calls. This scenario forces the gateway to keep more in-flights requests in-memory, and creates a more realistic scenario.

• VUs: 300
• Time: 10 minutes
• CPU limit: 2
• Memory limit: 4GB

Here are some insights worth mentioning:

🏆 Similar to the previous test, Apollo-Router and Wundergraph are the fastest gateways, with an average response time of 1100ms and a p95 of 2300ms, while under pressure of incoming requests.

🚅 The fastest JS-based gateway is GraphQL-Mesh (almost x2 than other JS-based), and when running on Bun runtime, it’s even faster (x1.5).

📈 The increased time of in-flight requests and the combination of Go runtime, led Wundergraph use 2.6GB of RAM, while Apollo-Router was able to handle the same amount of requests with 600MB of RAM.

📈 GraphQL-Mesh running on Bun was also able to handle a significant amount of requests with just 600MB of RAM.

constant-vus-subgraphs-delay-resources

You can find here the latest results, report and statistics

Same as constant-vus-subgraphs-delay-resources but with additional resources (CPU and RAM) and more concurrent VUs.

• VUs: 500
• Time: 10 minutes
• CPU limit: 4
• Memory limit: 8GB

Here are some insights worth mentioning:

📊 More CPU and more RAM were able to push the gateways tested to better results. Apollo-Router didn’t seem to use most of the provided RAM (only needed 700MB) while Wundergraph has high memory consumption (almost 3GB).

📈 The ability of GraphQL-Mesh to use Node.js’ cluster feature pays off when more CPUs are available to use.

📊 Under pressure, apollo-server (JS) was not able to some percentage of the requests.

ramping-vus

You can find here the latest results, report and statistics

Same as the previous setup, but this scenario aims to push the gateway to the limit by running a gradual increase of VUs. Some gateways are lagging behind or breaks at such scale (with the limitation of resources), and it’s easier to spot the top capacity of requests it can handle while under pressure.

• VUs: 50 -> 2000 (ramping)
• Time: 10 minutes
• CPU limit: 4
• Memory limit: 8GB

The chart below measures the p95 of the request duration (lower is better).

Here are some insights worth mentioning:

🏆 Apollo-Router and Wundergraph are the fastest gateways and are able to process all requests, with an average duration of 6500ms.

❌ Apollo-Server and mercurius (JS-based) fails to handle the load, and both are failing to respond to requests (with a timeout or connection drop). From the JS-based solution, GraphQL-Mesh is the only one able to handle all requests.

The Fine Print

Here are some important points to consider regarding the gateways tested in this benchmark:

• Federation specification compatibility:
  • Among the list of gateways tested, only apollo-server, apollo-router, and graphql-mesh fully support the Federation v1 specification without any adjustments. Other servers, such as wundergraph, do not have full support (Federation v2 specification is fully supported at the moment only by apollo-router, apollo-server and graphql-mesh).
  • We used the Federation Supergraph specification during the server runs, which is an artifact produced from a successful composition of subgraphs. Some gateways, like wundergraph and mercurius, do not support this specification, and a list of services was provided with a real-time composition based on GraphQL introspection.
  • We chose to use Federation v1 and not Federation v2 for this test because it is widely adopted and more supported by available gateways.
• Node.js runtimes:
  • For Node.js, we used version 18 (LTS) of the engine.
  • In some of the scenarios mentioned above, we used Bun (recently released v1).
• Fair comparison:
  • All gateways were running the same GraphQL schema, and the same GraphQL query was executed.
  • All gateways were running as Docker containers, using the latest available version (and were kept updated using Renovate).
  • Resources were limited to 1 CPU and 1GB RAM for all gateways in all scenarios.
• Subgraphs metrics:
  • In all the scenarios mentioned, all subgraphs consumed approximately 3% of the CPU and 10MB of memory. We measured this to ensure that the subgraphs were not becoming the bottleneck and affecting the results.
• Not only Federation:
  • The expectation from a GraphQL gateway is to do more than query planner - while GraphQL-Mesh and Wundergraph has broader scope and supports more use-cases, the Apollo tools are tailored to the Apollo ecosystem, and built for a specific purpose.

What Did We Learn?

• Compatibility is a big issue in the GraphQL ecosystem. The lack of a standard specification for GraphQL gateways makes it hard to compare and benchmark different solutions. We hope that future specifications will be more widely adopted and supported by the community.
• Performance is a big concern for GraphQL gateways. The ability to handle a large number of requests and to scale easily is a must-have for any GraphQL gateway. Resource consumption is also a big concern, and it’s important to aim to keep the memory and CPU usage low, to avoid bottlenecks and to keep the cost low.
• Runtime environment matters: Node.js is great, but we are keeping an eye on other runtimes, such as Bun. We are also looking forward to seeing more solutions built with these runtimes. Rust keeps RAM lower when written correctly, and Go is also a great option for building GraphQL gateways, but memory consumption could be higher.

Keeping This Report Updated

We welcome contributions from the community. If you notice any incorrect configuration, setup, or any other issue that affects the results or the comparison, please reach out to us and report a GitHub issue. This will help us keep this benchmark up to date and ensure a fair comparison.

We also encourage companies and developers to improve their stack. We would love to update this blog post when significant changes in results occur (the results in the repository are always up to date!).