opinionated-ci-pipeline

Opinionated CDK CI Pipeline

CI/CD utilizing CDK Pipelines.

See the announcement blog post for examples and architecture diagrams.

Features:

• pipeline deploying application from the default branch to multiple environments on multiple accounts,
• feature branch deployments to ephemeral environments,
• development environments deployments from the local CLI,
• build status notifications to repository commits,
• build failures notifications to SNS.

Currently supported source repositories are GitHub and Bitbucket.

Table of contents

• Table of contents
• Usage
  • 1. Install
  • 2. Set context parameters
  • 3. Create CDKApplication
  • 4. Create repository access token
    • GitHub
    • Bitbucket
  • 5. Bootstrap the CDK
  • 6. Deploy the CI Stack
  • Deploy development environment
• Parameters
• Notifications and alarms
• How to
  • Run unit tests during build
  • Enable Docker
• Library development

Usage

To set up, you need to complete the following steps:

1. Install the library in your project.
2. Specify context parameters.
3. Create CDKApplication with build process configuration.
4. Create repository access token.
5. Bootstrap the CDK on the AWS account(s).
6. Deploy the CI.

At the end, you will have CI pipeline in place, and be able to deploy your own custom environment from the CLI as well.

1. Install

For Node.js:

npm install -D opinionated-ci-pipeline

For Python:

pip install opinionated-ci-pipeline

2. Set context parameters

Add project name and environments config in the cdk.json as context parameters. Each environment must have account and region provided.

{
  "app": "...",
  "context": {
    "projectName": "myproject",
    "environments": {
      "default": {
        "account": "111111111111",
        "region": "us-east-1"
      },
      "prod": {
        "account": "222222222222",
        "region": "us-east-1"
      }
    }
  }
}

The project name will be used as a prefix for the deployed CI Stack name.

Environment names should match environments provided later in the CDKApplication configuration.

The optional default environment configuration is used as a fallback.

The CI pipeline itself is deployed to the ci environment, with a fallback to the default environment as well.

3. Create CDKApplication

In the CDK entrypoint script referenced by the cdk.json app field, replace the content with an instance of CDKApplication:

#!/usr/bin/env node
import 'source-map-support/register';
import {ExampleStack} from '../lib/exampleStack';
import {CDKApplication} from 'opinionated-ci-pipeline';

new CDKApplication({
    stacks: {
        create: (scope, projectName, envName) => {
            new ExampleStack(scope, 'ExampleStack', {stackName: `${projectName}-${envName}-ExampleStack`});
        },
    },
    repository: {
        host: 'github',
        name: 'organization/repository',
    },
    packageManager: 'npm',
    pipeline: [
        {
            environment: 'test',
            post: [
                'echo "do integration tests here"',
            ],
        },
        {
            environment: 'prod',
        },
    ],
});

This configures the application with one Stack and a pipeline deploying to an environment test, running integration tests, and deploying to environment prod.

The test and prod environments will be deployed from the branch main (by default). All other branches will be deployed to separate environments. Those feature-branch environments will be destroyed after the branch is removed.

To allow deployment of multiple environments, the Stack(s) name must include the environment name.

4. Create repository access token

An access to the source repository is required to fetch code and send build status notifications.

Once access token is created, save it in SSM Parameter Store as a SecureString under the path /{projectName}/ci/repositoryAccessToken.

See instructions below on how to create the token for each supported repository host.

GitHub

Create a fine-grained personal access token with read-only access for Contents read and write access for Commit statuses and Webhooks.

Bitbucket

In Bitbucket, go to your repository. Open Settings → Access tokens. There, create a new Repository Access Token with repository:write and webhook scopes.

5. Bootstrap the CDK

Bootstrap the CDK on the account holding the CI pipeline and all other accounts the pipeline will be deploying to.

When bootstrapping other accounts, add the --trust parameter with the account ID of the account holding the pipeline.

6. Deploy the CI Stack

Run:

cdk deploy -c ci=true

Deploy development environment

Run:

cdk deploy -c env=MYENV --all

to deploy arbitrary environments.

Parameters

Name	Type	Description
stacks	object	An object with a create() method to create Stacks for the application. The same Stacks will be deployed with main pipeline, feature-branch builds, and local deployments.
packageManager	npm | pnpm	Package manager used in the repository. If provided, the install command will be set to install dependencies using given package manager.
commands	object	Commands executed to build and deploy the application. The following commands are set by default: install synthPipeline deployEnvironment destroyEnvironment If you override the install command, either install the aws-cdk@2 globally or modify the other 3 commands to use the local cdk binary. Commands executed on particular builds: main pipeline: preInstall install buildAndTest synthPipeline feature branch environment deployment: preInstall install buildAndTest deployEnvironment feature branch environment destruction: preInstall install destroyEnvironment
cdkOutputDirectory	string	The location where CDK outputs synthetized files. Corresponds to the CDK Pipelines ShellStepProps#primaryOutputDirectory.
pipeline	object[]	CodePipeline deployment pipeline for the main repository branch. Can contain environments to deploy and waves that deploy multiple environments in parallel. Each environment and wave can have pre and post commands that will be executed before and after the environment or wave deployment.
codeBuild	object	Override CodeBuild properties, used for the main pipeline as well as feature branch ephemeral environments deploys and destroys.
codePipeline	object	Override CodePipeline properties.
slackNotifications	object	Configuration for Slack notifications. Requires configuring AWS Chatbot client manually first.

Notifications and alarms

Stack creates SNS Topics with notifications for main pipeline failures and feature branch build failures. Their ARNs are saved in SSM Parameters and outputed by the stack:

• main pipeline failures:
  • SSM: /{projectName}/ci/pipelineFailuresTopicArn
  • Stack exported output: {projectName}-ci-pipelineFailuresTopicArn
• feature branch build failures:
  • SSM: /{projectName}/ci/featureBranchBuildFailuresTopicArn
  • Stack exported output: {projectName}-ci-featureBranchBuildFailuresTopicArn

If you setup Slack notifications, you can configure those failure notifications to be sent to Slack.

Moreover, if you setup Slack notifications, an additional SNS Topic will be created to which you can send CloudWatch Alarms. It's ARN is provided:

• SSM: /{projectName}/ci/slackAlarmsTopicArn
• Stack exported output: {projectName}-ci-slackAlarmsTopicArn

How to

Run unit tests during build

Set commands in the commands.buildAndTest:

{
    commands: {
        buildAndTest: [
            'npm run lint',
            'npm run test',
        ]
    }
}

Enable Docker

Set codeBuild.buildEnvironment.privileged to true:

{
    codeBuild: {
        buildEnvironment: {
            privileged: true
        }
    }
}

Library development

Project uses jsii to generate packages for different languages.

Install dependencies:

npm install

Build:

npm run build

Install and deploy example application:

cd example
pnpm install
pnpm cdk deploy -c ci=true

Mentions

Git repository push to CodeCommit inside Lambda function based on https://msimpson.co.nz/Github-CodeCommit/