GraphQL Mesh - Query anything, run anywhere

We are excited to announce GraphQL Mesh — a powerful open source library enabling developers to leverage the power of GraphQL without requiring changes to the technologies your servers already use.

We’ve been using this tool to significantly improve our clients stacks — improving the areas that a classic GraphQL gateways would have left untouched

The Basics

• GraphQL Mesh takes any source schema (openapi/Swagger, json-schema, SOAP, gRPC, SQL, Mongoose, GraphQL, Federated GraphQL, queue systems, and others) and let you query it as GraphQL without changing the source service
• Choose the fields you want, in the shape you want them
• Get full type-safety for remote APIs
• Schema unification/Stitching for any source
• GraphQL Mesh runs anywhere. As a standalone gateway, as a datasource, or run as a local, distributed GraphQL SDK to simplify service-to-service communications
• Each step of the transformation is fully customizable
• Ability to provide your API consumers the GraphQL experience without the operational overhead

Intro

GraphQL Mesh allows you to use GraphQL query language to access data in remote APIs that don’t run GraphQL (and also ones that do run GraphQL).

It can be used as a gateway to other services, or run as a local GraphQL schema SDK (data source) that aggregates data from remote APIs.

The goal of GraphQL Mesh is to let developers easily access services that are written in other APIs specs (such as gRPC, OpenAPI, Swagger, oData, SOAP, GraphQL and even SQL) with GraphQL queries, mutations and subscriptions.

GraphQL Mesh gives the developer the ability to modify the output schemas, link types across schemas and merge schema types. You can even add custom GraphQL types, resolvers, links and more.

It allows developers to control the way they fetch data, and overcome issues related to backend implementation, legacy API services, chosen schema specification and non-typed APIs.

Here are some use cases and how GraphQL Mesh can be used to support them:

• Querying legacy/non-typed APIs with an easy query language that is fully type-safe
• Aggregate, transform and link data across multiple APIs
• Proxy and have a simple way to access data from various API specifications without the need to learn each one specifically
• Access and fetch data across microservices (a merged Data Graph) without adding a central failure point to your deployment architecture

The way GraphQL Mesh works is:

• Collect API schema specifications from services, registries or simple files
• Create a runtime instance of fully-typed SDK for each service
• Convert the API specs of each service to GraphQL schema
• Applies custom schema transformations and schema extensions
• Creates fully-typed, single schema, GraphQL SDK to fetch data from your services.

GraphQL Mesh is acting as a proxy to your data, and uses common libraries to wrap your existing API services. You can use this proxy as an SDK in your service by running the GraphQL schema locally (with GraphQL execute), or you can deploy this as a gateway layer to your internal service.

Note: Our goal with GraphQL Mesh is to create an easy-to-use GraphQL proxy to your own data and services. You may want to consider implementing another layer to expose your data publicly.

A Short Break — Who Are We

GraphQL Mesh was created by the open source group called The Guild.

We currently maintain the following libraries:

We love what we do and are fulfilled by that, but if you wish to support us, here are some possible ways:

• Work with us - The best way to support us is to let us work with you :) We work with clients from all over the world and in the process learn from them how to make our libraries better. We work with companies from fortune 50 to small startups with different types of engagements. We help enhance developer teams, making individuals more productive and help to develop themselves. Reach out directly for a free meeting, even just for a fun chat to share ideas and get feedback!
• Tell people about us — Many people who use our libraries, don’t know who we are and don’t know about all the other libraries we’ve created. Help us spread the word!
• Contribute — Better docs -> answer issues -> Reproduce bugs and create tests — we can’t stress how valuable this would be and we would love to support your journey into open source! Also, as The Guild is distributed, create your own new libraries under your own name and share it with us, we’ll help you promote and maintain it if you want!

We have a large community as well which you can join by following the repositories and our Discord channel. We are having our first new monthly Guild community meeting very soon!

Use Cases (3 Example Use Cases)

1 — the Simple Use Case (GraphQL Data Source per Endpoint)

So you got yourself a new GraphQL server!

It sits between your frontend and your former REST backends. And you got the benefits of having type safety over the network.

But what about the services and data sources you query from your resolvers?

In your resolvers you still call regular REST endpoints? They might not be typed and might not return the shape of response you want them to return.

Wouldn’t it be nice if you could use those data sources as GraphQL sources?

One option would be to migrate those services to GraphQL or build a GraphQL wrapper for each, which might not be so feasible, or would take too long for the following reasons:

• In most large companies, wrapping each service in GraphQL or adding a GraphQL implementation to each service is not a realistic option. The service maintainers need to learn, agree and add new tasks for their already busy backlog.
• The capabilities the GraphQL engine brings us (orchestration, restructuring) are very powerful but usually runs on the provider. That means that each service will now behave differently on different consumers. In many Enterprise architectures, that behaviour can be harder to cache and puts a burden on the service team that might not be the right responsibility for them.
• It might not always be a good idea to dictate a single protocol for all your services to use. It’s a bit like dictating a single programming language for all your services in the organization.

Can we use the benefits of GraphQL without rewriting our services and moving execution burden to them?

That way we could get type safety and querying abilities (you don’t really care about less data over the network between services).

That’s a first place where GraphQL Mesh can help you — simply input any schema file (For example Swagger definitions), from any source to the Mesh config, and we’ll generate a fully typed GraphQL SDK, take would make it look like those sources exposed GraphQL themselves.

That SDK can run locally on your GraphQL server without affecting the underlining services at all.

2 — the Merging Use Case

Great! So now you have a lot of “GraphQL endpoints” that simplify and automate a lot of the work you needed to do in your resolvers.

But maybe some of those services could be combined into a single entity Graph?

The second place GraphQL Mesh can come in handy is that it lets you stitch together those converted GraphQL endpoints into a single endpoints and a Graph.

schema stitching for any source!

So now you can delegate a lot of logic from your manual GraphQL server into GraphQL Mesh.

3 — the Service Mesh Use Case

So you’ve done all that work on your GraphQL gateway.

But there are other services in your network that might need to query other services that only expose an HTTP/SOAP/gRPC endpoints.

They still need to orchestrate those calls, validate them and shape them to the structure they need (a good example might be a ML service that crunches data from many other services).

One of the interesting things about Mesh is that it doesn’t impose any changes to the services and that it can run as an SDK anywhere.

So now each service can get an SDK of the full Graph, which it can run locally in a distributed way, instead of having all services calls going through a central gateway.

Usage

Check out all the examples on our repo

To get started with GraphQL Mesh, create a simple .yaml file that points to your existing API services spec — your Swagger file, .proto files or your GraphQL service, and choose the correct handler for it.

Here’s a simple example for fetching and aggregating data across multiple external services. It uses 2 independent OpenAPI data sources, and links them together.

Before:

async function getCityAndWeather(city: string): Promise<{ city: string; weather: any }> {
  const cities /* any */ = await fetchApi(`https://cities-api/cities?search=${city}`)
  const city /* any */ = cities.data[0]
  const weather /* any */ = await fetchApi(
    `https://weather-api/forecast?lat=${city.latitude}&lon=${city.longitude}`
  )
 
  return { city, weather }
}

Code isn’t typed, and it requires you to write code that fetches and merges it manually. It also means that your code deals with request-response and the next time you’ll need to fetch data from those sources, you’ll need to duplicate code and apply those merging logics again.

It also means that your code should be fully aware of what APIs it fetches from, and what data they return.

After:

First, you can configure GraphQL Mesh to load your remote services, as following:

sources:
  # This source will create:
  #   type City { id: ID!  name: String!  latitude: Float!  longitude: Float! }
  #   type CitiesResult { data: [City!]! }
  #   type Query { cities(search: String!): CitiesResult }
  - name: Geo
    handler:
      openapi:
        source: https://cities-api/cities/schema.json
        headers:
          Authorization: API-KEY
 
  # This source will create
  #   type Forecast { clouds: Int  minTemp: Int  maxTemp: Int  }
  #   type ForecastResult { data: Forecast! }
  #   type Query { forecastByLatLon(lat: Int!, lon: Int!): ForecastResult  }
  - name: Weather
    handler:
      openapi:
        source: https://weather-api/forecast/schema.json

The data aggregation logic can be moved into a custom GraphQL resolver:

import { Resolvers } from './mesh/types.generated'
 
export const resolvers: Resolvers = {
  dailyForecast: async (city, args, { Weather }) => {
    return Weather.api.forecastDailyByLatLon(city.latitude, city.longitude).then(r => r.data)
  }
}

We are using city parent value, which is being fetched by the Cities API, and fetch data from the weather API based on the city location.

And your data fetching function can use GraphQL to query for the data, and GraphQL Mesh will generate the corresponding method getCityAndWeather for you:

query getCityAndWeather($search: String!) {
  cities(search: $search) {
    data {
      name
      latitude
      longitude
      forecast {
        clouds
        minTemp
        maxTemp
      }
    }
  }
}

This way, the implementation of the getCityAndWeather is being generated as a typed function, and the GraphQL query is agnostic to the services it fetches data from.

The logic linking those API services is now part of your GraphQL engine — you only need to deal with writing a GraphQL query to fetch the data you need.

GraphQL Mesh as a Local Schema

The output of GraphQL Mesh is a generated code, so you can import it directly from the generated directory. You can load the schema and use GraphQL execute to run queries and mutations.

Using this method, the query and the mutation will be executed locally in your application, and the GraphQL engine will be in charge of running the actual data fetching, using the source APIs.

This method brings the GraphQL schema management closer to the developer, and allows you to modify and manage the GraphQL schema according to your application needs.

GraphQL Mesh as a Gateway

You can use the generated schema and run it on its own server, as a gateway to your data fetching. You can share a single gateway with multiple applications — it means that changing and manipulating the schema is being managed by the gateway itself, and not the consuming applications.

Using GraphQL Mesh output as a gateway means that the consuming applications will execute GraphQL over the network to the gateway, and from the gateway to the source APIs.

Querying Data from Mesh

Because the output of GraphQL Mesh is a GraphQL schema, you can choose how to execute and query it, according to your needs and preference:

• You can use simple GraphQL queries and mutations directly, and fetch data from the generated schema.
• You can use GraphQL queries and mutations with GraphQL Code Generator, in order to generate a ready-to-use SDK based on your GraphQL operations. It will also make sure that your result types are types according to your GraphQL selection set.
• You can use any other GraphQL client (graphql-request, apollo-client, gqless).

How Does It Work?

This is a short version, In the upcoming week we’ll write more in depth article about each layer of the architecture and our thoughts and choices while implementing the library — Spec conversions, code-generation vs. runtime, type safe SDKs, schema transformations, merging types, caching, data loaders and optimizations

Converting APIs Specification to GraphQL

GraphQL Mesh initially loads your existing services’ APIs specification — either from a remote source (where introspection/reflection is available) or from local files.

It then attempts to convert your specification into a GraphQL schema, in an optimal way.

Converting APIs specifications to GraphQL schema it not always simple — some of the traditional API specification uses a request-response approach (each endpoint or method has a type for it’s request and a type for it’s response, and it’s not necessarily shared), compared to GraphQL, which uses type-based schema that allow links and re-use of types (hence the graph).

Erik Wittern, Alan Cha, and Jim A. Laredo from IBM wrote a very interesting article about the difficulties of converting REST-like API to a GraphQL schema. Those concepts are implemented in openapi-to-graphql library, which is being used by GraphQL Mesh and we are contributing to. We apply similar approach to other API specifications.

Aside from wrapping the service API with a GraphQL layer, GraphQL Mesh also allow you to easily extend or modify the converted schema, all with a fully-typed SDK, tailored specifically to the API service you are using

It also means that creating new fields or links between the schemas is easier, and you can always be sure it’s using the correct object structure.

Schema Transformations (Extensions, Merging)

GraphQL Mesh allows you to do manipulations on schemas. You can either manipulate the schema before unifying it (for example, prefix all types, change a type name, delete a field) or after unifying all schemas (link field across schemas or merge types).

All phases of Mesh are customizable and pluggable — Which means you can adjust it with any strategy that works for you.

One benefit is that we can now choose between Schema Stitching and Federation!

In our Schema Stitching example, we use graphql-tools-fork and it’s powerful schema-stitching implementation to create new merge types on the output schema.

In our Federation example, we used Apollo Server’s Federation and Gateway libraries, together with the graphql-transform-federation library, which let us add the Federation metadata to existing GraphQL schemas. So we use Federation as a local merging strategy, together with its query planner executor.

We can now compare those two strategies next to each other and show the benefits and downsides of each approach for many different use cases.

SQL as a Source

Right before we decided to open source the library, we had a thought — why only focus on API protocols.

There are many inspiring projects out there that convert database schemas to SQL.

So why not integrate those schemas as well?

So we decided to do a POC using the amazing Postgraphile project!

Check out our example for merging Geo DB with GitHub! We would love to hear and help you integrate with more databases and sources!

Summary

GraphQL Mesh has been a huge upgrade for us and our clients. But we believe this is only the start of the conversation around GraphQL and other protocols, merging remote schemas and execution. Try it out today and contact us with all the questions, use cases and feedback you might have!

Our repositories, Discord channel and newsletter.