Skull Island

Skull Island is a declarative configuration management tool for Kong API Gateway to backup and synchronize Kong API Gateway configuration. We recommend using Kong Dashboard to add routes through a UI and then using Skull Island to backup the configuration changes and later synchronize it. Skull Island is inspired by Kongfig and Biplane.

Installation

This application is meant to be used as a command line tool. You can install the latest version globally:

npm install -g skull-island

If you have an older version and need to upgrade to the latest version:

npm upgrade -g skull-island

Running the application

Backup

In order to backup a running Kong API gateway configuration to disk, use the backup command:

skull-island backup --url http://127.0.0.1:8001

Kong API Gateway is running on 127.0.0.1 and the administration port is running on port 8001. You can find additional parameters using skull-island backup -h. The default backup file is generated in the current directory called kong-backup.json.

Synchronization

In order to synchronize a configuration on disk to a running Kong API Gateway, use the synchronize command:

skull-island synchronize --url http://127.0.0.1:8001

Kong API Gateway is running on 127.0.0.1 and the administration port is running on port 8001. You can find additional parameters using skull-island synchronize -h. The default backup file that is used for the synchronization process must exist in the current directory and is expected to be called kong-backup.json.

Teardown

In order to wipe a Kong API Gateway clean of entities (APIs, Plugins, Consumers, and Consumer Credentials), use the teardown command:

skull-island teardown --url http://127.0.0.1:8001

Kong API Gateway is running on 127.0.0.1 and the administration port is running on port 8001.

Building out a Kong API gateway environment to do local testing

Ensure you have the latest version of Docker running:

Docker Compose

To bring up Cassandra and the Kong API Gateway, use

docker-compose up

You can add a -d flag if you want to run it in the background and not have the logs pollute your terminal. You can use Kong Dashboard via http://localhost:8080 and use http://kong-api-gateway:8001 as the Kong node URL since Kong-Dashboard is running inside a Docker container so we make use of Container Name DNS resolution.

Once you are finished, and want to clean up, use

docker-compose down

Just Docker

1. create a virtual Docker network that will host Kong and Cassandra so they may communicate with each other

   docker network create kong-network
   

2. create the Cassandra Docker container on the kong-network

   docker run -d --name kong-database --network kong-network cassandra:3
   

3. create the Kong API Gateway Docker container on the kong-network and expose ports over to the host network to access the admin and proxy APIs

   docker run -d --name kong --network kong-network -e "KONG_DATABASE=cassandra"
     -e "KONG_CASSANDRA_CONTACT_POINTS=kong-database"
     -p 8000:8000
     -p 8443:8443
     -p 8001:8001
     -p 7946:7946
     -p 7946:7946/udp
     kong:latest
   

Integration tests

In order to run tests, you need to start up Kong locally by following the Docker Compose instructions and run: docker-compose up

Once the services are up, you can run integration tests by executing: gulp test

Please note, you will need to install gulp globally to do this.

Caveats and Design Decisions

Kong's basic-auth consumer credentials currently cannot be backed up and re-applied because the backup obtains encrypted credentials and reapplying the encrypted credentials causes the credentials to be encrypted once more which is incorrect and will cause your credentials to stop working. As a result, we disable synchronization of basic authentication consumer credentials by default. There is a flag (-b) which allows you to synchronize basic authentication credentials provided that you store plaintext credentials in the backup file. The intended use case for Skull-Island was to be used in conjunction with Kong-Dashboard in order to have a better process workflow rather than coming up with your own JSON configuration.

For the synchronization process, extra entities for APIs, Plugins and Consumers that are present on the server and are not present on disk are deleted. For now, all Consumer Credentials (except basic-authentication unless specified) are removed from the server completely and then synchronized from disk resulting in slight downtime.

upstream_url for API Entities

As part of the workflow, you might introduce fill-in-the-blanks, to minimize duplication for Skull-Island backups especially when you need to deploy your changes to multiple environments. upstream_url field is a Kong URL required field. if the kong-backup.json file has some endpoints without proper upstream_url field then those specific endpoints will be skipped and the script won't fail during synchronization process. This decision was taken because some environments may have more APIs defined when compared to other environments.

Certificate backups

Certificate backups are supported but follow a different convention. By default, Kong places the public and private key data directly in the JSON response. For example:

{
    "created_at": 1485521710000,
    "cert": "public-key1",
    "key": "private-key1",
    "id": "0e06dee7-ddba-4398-87b9-134944b0aa91"
}

Our convention is to refer to file paths for certificates:

{
    "id": "0e06dee7-ddba-4398-87b9-134944b0aa91",
    "created_at": 1485521710000,
    "cert_path": "certificates/0e06dee7-ddba-4398-87b9-134944b0aa91.pub.pem",
    "key_path": "certificates/0e06dee7-ddba-4398-87b9-134944b0aa91.pem"
}

In the example above, we create pem files and refer to them via relative file paths (so they will materialize in your current directory in the certificates folder). cert_path corresponds to your public key and key_path corresponds to your private key. Both keys are PEM encoded.

Note: If your certificate paths do not contain the certificates, the synchronization of these certificates will be skipped and a warning will be printed.

Release process

The release process uses the release-me NPM module

Notes

To my understanding, the dependency graph can be visualized like this:

• Consumers -depend on-> Plugins -depend on-> APIs
• SNIs -depend-on-> Certificates

There are global Plugins which have no dependency on APIs like Syslog. So in terms of deleting or adding, you need to start with the Consumers, followed by Plugins and finally APIs.

In order to perform a synchronization, you need to pull down the current state of the API gateway and have the backup file on hand. You need to perform the following steps for Consumers, Plugins and APIs

• Check what is present in the backup file and compare the entries to the entries pulled from the API gateway,

  • if we have less entries in the file then we need to remove entries from the API gateway.
  • If we have more entries in the file then we need to add entries into the API gateway

• See if existing entries need to be updated (you could always blindly update the server with the data from the file to avoid complications)

Note: you cannot rely on blind updates to existing records (for Consumer Credentials), you must look at each one for differences and if you find differences, then delete the one on the server and upload the new one. You could actually do this process blindly (delete then add).