@aws/neptune-for-graphql

Amazon Neptune utility for GraphQL™ schemas and resolvers

The Amazon Neptune utility for GraphQL™ is a Node.js command-line utility to help with the creation and maintenance of a GraphQL API for the Amazon Neptune database. It is a no-code solution for a GraphQL resolver when GraphQL queries have a variable number of input parameters and return a variable number of nested fields.

If you start from a Neptune database with data, the utility discover the graph database schema including nodes, edges, properties and edges cardinality, generate the GraphQL schema with the directives required to map the GraphQL types to the graph databases nodes and edges, and auto-generate the resolver code. We optimized the resolver code to reduce the latency of querying Amazon Neptune by returning only the data requested by the GraphQL query. (Note: the utility works only for Property Graph databases, not RDF yet)

You can also start with a GraphQL schema with your types and an empty Neptune database. The utility will process your starting GraphQL schema and inference the directives required to map it to the Neptune database graph nodes and edges. You can also start with GraphQL schema with the directives, that you have modified or created.

The utility has the option to generate the AWS resources of the entire pipeline, including the AWS AppSync API, configuring the roles, data source, schema and resolver, and the AWS Lambda that queries Amazon Neptune.

If you have a few queries with a static number of input parameters and return fields, and you are willing to code your GraphQL resolver, look at these blogs:

• Building Serverless Calorie tracker application with AWS AppSync and Amazon Neptune
• Integrating alternative data sources with AWS AppSync: Amazon Neptune and Amazon ElastiCache

Index:

• Install and Setup
• Starting from a Neptune database with data
• Starting from a Neptune database with data: Air Routes Example
• Starting from a GraphQL schema and an empty Neptune database
• Starting from a GraphQL schema and an empty Neptune database: Todo Example
• Starting from GraphQL schema with directives
• Customize the GraphQL schema with directives
• AWS resources for the GraphQL API
• Commands reference: utility CLI options
• Known Limitations
• Roadmap
• License
• Contributing

Install and Setup

The utility requires Node.js to be executed. Some features require reaching to a Neptune database, and having access to the AWS CLI with permissions to create AWS resources. You can also run the utility just to process input files without the need to connect it directly to a Neptune database or to create AWS resources. In this case you will find the GraphQL schemas and the resolver code in the local output directory.

Node.js is required in any scenario

Node.js is required to run the utility, v18 or above.

• To install it on macOS or Windows go to the Node.js website to download the installer.
• To install on an EC2-Instance follow the AWS documentation.

Install the Amazon Neptune utility for GraphQL

The utility is available as NPM package. To install it:

npm install @aws/neptune-for-graphql -g

After the installation run the utility `--help`` command to check if runs:

neptune-for-graphql --help

Reach to a Neptune database endpoint

If you are starting from a Neptune database with data, you need to enable the utility to reach the database endpoint. By default Neptune databases are accessible only within a VPC.

The easiet way is to run the utility from an EC2 instance which network is configured within your Neptune database VPC. The minimum size instance to run the utility is t2.micro (free of charge). During the creation of the instance select the Neptune database VPC using the Common Security Groups pulldown menu.

Next Step

• Install Node.js
• Install AWS CLI if you need to create the AWS resources, or you need an IAM role to access the Neptune database.

Starting from a Neptune database with data

Independently if you are familiar with GraphQL or not, the command below is the fastest way to create a GraphQL API. Starting from an existing Neptune database endpoint, the utility scans the Neptune database discovering the schema of the existing nodes, edges and properties. Based on the graph database schema, it inferences a GraphQL schema, queries and mutations. Then, it creates an AppSync GraphQL API, and the required AWS resources, like a pair of IAM roles and a Lambda function with the GraphQL resolver code. As soon as the utility complete the execution, you will find in the AppSync console your new GraphQL API called your-new-GraphQL-API-nameAPI. To test it, use the AppSync “Queries” from the menu. (Note: follow the setup instructions below to enable your environment to reach the Neptune database and create AWS resources.)

neptune-for-graphql --input-graphdb-schema-neptune-endpoint <your-neptune-database-endpoint:port> --create-update-aws-pipeline --create-update-aws-pipeline-name <your-new-GraphQL-API-name> --output-resolver-query-https

If you run the command above a second time, it will look again at the Neptune database data and update the AppSync API and the Lambda code.

To rollback, removing all the AWS resources run:

neptune-for-graphql --remove-aws-pipeline-name <your-new-GraphQL-API-name>

References:

• here an example using the Air Routes data on Amazon Neptune, showing the outputs of the utility.
• If you are wondering which AWS resources the utility is creating, look at the section below.
• To customize the GraphQL schema, look at the section below.

Starting from a GraphQL schema and an empty Neptune database

You can start from an empty Neptune database and use a GraphQL API to create the data and query it. Running the command below, the utility will automate the creation of the AWS resources. Your your-graphql-schema-file must include the GraphQL schema types, like in the TODO example here. The utility will analyze your schema and create an extended version based on your types. It will add queries and mutations for the nodes stored in the graph database, and in case your schema have nested types, it will add relationships between the types stored as edges in the graph database, again see the TODO example here. The utility creates an AppSync GraphQL API, and the required AWS resources, like a pair of IAM roles and a Lambda function with the GraphQL resolver code. As soon as the utility complete the execution, you will find in the AppSync console your new GraphQL API called your-new-GraphQL-API-nameAPI. To test it, use the AppSync “Queries” from the menu. (Note: follow the setup instructions below to enable your environment to reach the Neptune database and create AWS resources.)

neptune-for-graphql --input-schema-file <your-graphql-schema-file> --create-update-aws-pipeline --create-update-aws-pipeline-name <your-new-GraphQL-API-name> --create-update-aws-pipeline-neptune-endpoint <your-neptune-database-endpoint:port> --output-resolver-query-https

References:

• here an example using a TODO GraphQL schema, showing the outputs of the utility.
• If you are wondering which AWS resources the utility is creating, look at the section below.
• To customize the GraphQL schema, look at the section below.

Starting from a GraphQL schema with directives

You can start from a GraphQL schema with directives for a graph database. For the list of supported directives see the section below Customize the GraphQL schema with directives.

neptune-for-graphql --input-schema-file <your-graphql-schema-file-with-directives> --create-update-aws-pipeline --create-update-aws-pipeline-name <your-new-GraphQL-API-name> --create-update-aws-pipeline-neptune-endpoint <your-neptune-database-endpoint:port> --output-resolver-query-https

Customize the GraphQL schema with directives

The utility generate a GraphQL schema with directives, queries and mutations. You might want to use it as is, or change it. Here below ways to change it.

Please no mutations in my schema

In case you don't want your Graph API having the option of updating your database data, add the the CLI option --output-schema-no-mutations.

@alias

This directive can be applied to GraphQL schema types or fields. It maps different names between the graph database and the GraphQL schema. The syntax is @alias(property: your-name). In the example below airport is the graph database node label mapped to the Airport GraphQL type, and desc is the property of the graph database node mapped to the field description. See the Air Routes Example. The GraphQL guidence is pascal case for types and camel case for fields.

type Airport @alias(property: "airport") {  
  city: String
  description: String @alias(property: "desc")  
}

@relationship

This directive maps nested GraphQL types to a graph databases edges. The syntax is @relationship(edgeType: graphdb-edge-name, direction: IN|OUT).
See the Todo Example and the Air Routes Example.

type Airport @alias(property: "airport") {
  ...
  continentContainsIn: Continent @relationship(edgeType: "contains", direction: IN)
  countryContainsIn: Country @relationship(edgeType: "contains", direction: IN)
  airportRoutesOut(filter: AirportInput, options: Options): [Airport] @relationship(edgeType: "route", direction: OUT)
  airportRoutesIn(filter: AirportInput, options: Options): [Airport] @relationship(edgeType: "route", direction: IN)
}

@graphQuery, @cypher

You can define your openCypher queries to resolve a field value, add queries or mutations.

Here new a field outboundRoutesCount is added to the type Airport to count the outboud routes:

type Airport @alias(property: "airport") {
  ...
  outboundRoutesCount: Int @graphQuery(statement: "MATCH (this)-[r:route]->(a) RETURN count(r)")
}

Here an example of new queries and mutations. Note that if you omit the RETURN, the resolver will assume the keyword this is the returning scope.

type Query {
  getAirportConnection(fromCode: String!, toCode: String!): Airport @cypher(statement: "MATCH (:airport{code: '$fromCode'})-[:route]->(this:airport)-[:route]->(:airport{code:'$toCode'})")   
}

type Mutation {
  createAirport(input: AirportInput!): Airport @graphQuery(statement: "CREATE (this:airport {$input}) RETURN this")
  addRoute(fromAirportCode:String, toAirportCode:String, dist:Int): Route @graphQuery(statement: "MATCH (from:airport{code:'$fromAirportCode'}), (to:airport{code:'$toAirportCode'}) CREATE (from)-[this:route{dist:$dist}]->(to) RETURN this")
}

You can add query or mututation using a Gremlin query. At this time Gremlin queries are limited to return scalar values, or elementMap() for a single node, or elementMap().fold() for a list of nodes.

type Query {
  getAirportWithGremlin(code:String): Airport @graphQuery(statement: "g.V().has('airport', 'code', '$code').elementMap()") # single node
  getAirportsWithGremlin: [Airport] @graphQuery(statement: "g.V().hasLabel('airport').elementMap().fold()") # list of nodes
  getCountriesCount: Int @graphQuery(statement: "g.V().hasLabel('country').count()")  # scalar example
}

@id

The directive @id identify the field mapped to the graph database entity id. Graph databases like Amazon Neptune always have a unique id for nodes and edges assigned during bulk imports or autogenerated. In the example below _id

type Airport {
  _id: ID! @id
  city: String
  code: String  
}

Reserved types, queries and mutations names

The utility autogenerates queries and mutations to enable you to have a working GraphQL API just after having ran the utility. The pattern of these names are recognized by the resolver and are reserved. Here an example for the type Airport and the connecting type Route:

The type Options is reserved.

input Options {
  limit: Int
}

The function parameters filter, and options are reserved.

type Query {  
    getNodeAirports(filter: AirportInput, options: Options): [Airport]
}

The beginning of query names getNode..., and the beginning of the mutations names like createNode..., updateNode..., deleteNode..., connectNode..., updateEdge..., and deleteEdge....

type Query {  
  getNodeAirport(id: ID, filter: AirportInput): Airport
  getNodeAirports(filter: AirportInput): [Airport]
}

type Mutation {  
  createNodeAirport(input: AirportInput!): Airport
  updateNodeAirport(id: ID!, input: AirportInput!): Airport
  deleteNodeAirport(id: ID!): Boolean  
  connectNodeAirportToNodeAirportEdgeRoute(from: ID!, to: ID!, edge: RouteInput!): Route
  updateEdgeRouteFromAirportToAirport(from: ID!, to: ID!, edge: RouteInput!): Route
  deleteEdgeRouteFromAirportToAirport(from: ID!, to: ID!): Boolean
}

Re-apply your changes with --input-schema-changes-file

You might want to modify the GraphQL source schema and run the utility again getting the latest schema from your Neptune database. Every time the utility discover the a new graphdb schema it generates a new GraphQL schema. To inject your changes, you can manually edit the GraphQL source schema, and run the utility again using it as input instead of the Neptune database endpoint, or write your changes the file format below. As you run the utility with the option --input-schema-changes-file <value>, your changes will be applied at once.

[
     { "type": "graphQLTypeName",
       "field": "graphQLFieldName",
       "action": "remove|add",
       "value": "value"
    }
]

For Example:

[
    { "type": "Airport", "field": "outboundRoutesCountAdd", "action": "add", "value":"outboundRoutesCountAdd: Int @graphQuery(statement: \"MATCH (this)-[r:route]->(a) RETURN count(r)\")"},    
    { "type": "Mutation", "field": "deleteNodeVersion", "action": "remove", "value": "" },
    { "type": "Mutation", "field": "createNodeVersion", "action": "remove", "value": "" }
]

AWS resources for the GraphQL API

You have three option to created the GraphQL API pipeline:

• Let the utility create the AWS resources
• Let the utility create a CDK file, then you run it
• You manually create the AWS resources

Independently of the method you or the utility will need to create the following resources:

• Create a IAM role for Lambda called LambdaExecutionRole
• Attach to the LambdaExecutionRole the IAM policy AWSLambdaBasicExecutionRole
• For VPC (default) attach to the LambdaExecutionRole the IAM policy AWSLambdaVPCAccessExecutionRole
• For IAM create and attach to the LambdaExecutionRole a new IAM policy that only allow to connect and query Neptune
• Create a Lambda function with the LambdaExecutionRole
• Create a IAM for AppSync API to call the Lambda called LambdaInvocationRole
• Attach to the LambdaInvocationRole the policy LambdaInvokePolicy
• Create the AppSync GraphQL API
• Add to the AppSync API a Function with the LambdaInvocationRole to call the Lambda

Let the utility create the resources

With the CLI option --create-update-aws-pipeline, and its accesory options (see the commands reference), the utility automatically creates the resources.
You need to run the utility from a shell in which you installed the AWS CLI, and it is configured for a user with the permission of creating AWS resources.
The utility creates a file with the list of resources named pipeline-name.resources.json. Then it uses the file to modify the existing resources when the user runs the command again, or when the user wants to delete the AWS resources with the command option --remove-aws-pipeline-name <value>. The code of the utiliy uses the JavaScript AWS sdk v3, if you'd like to review the code is only in this file.

I prefer a CDK file

To option to trigger the creation of the CDK file starts with --output-aws-pipeline-cdk, and its accessory options (see the commands reference).
After you ran it, you will find in the output folder the file CDK-pipeline-name-cdk.js and CDK-pipeline-name.zip. The ZIP file is the code for the Lambda function. See CDK end to end example here.

Let me setup the resources manually or with my favorite DevOps toolchain

Here the detailed list of resorces needed to configure the GraphQL API pipeline.

Known limitations

• @graphQuery using Gremlin works only if the query returns a scalar value, one elementMap(), or list as elementMap().fold(), this feature is under development.
• Neptune RDF database and SPARQL language is not supported.

Roadmap

• Gremlin resolver.
• SPARQL resolver for RDF database.
• Generate GraphQL resolver and configurations for Apollo Server.

License

Copyright 2023 Amazon.com, Inc. or its affiliates. All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"). You may not use this file except in compliance with the License. A copy of the License is located at http://www.apache.org/licenses/LICENSE-2.0 or in the "license" file accompanying this file. This file is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. Full licence page.

Contributing

Follow AWS open source practices.
Contributing page.
Code of conduct page.