turbo-remote-cache-construct

Turborepo Remote Cache API Construct

An AWS CDK construct for easily deploying a Turborepo Remote Cache API infrastructure. This project implements a Remote Cache API for Turborepo using AWS services. It provides a scalable and efficient solution for storing and retrieving build artifacts in a distributed development environment.

Why use Remote Caching?

The Turborepo Remote Cache API allows teams to share and reuse build artifacts across different machines and CI/CD pipelines, significantly reducing build times and improving development efficiency. Read more about Remote Caching

Features

• ⚡ High-performance S3 integration for artifact storage
• 🔒 Flexible authentication with custom authorizer support
• 🔑 Pre-signed URL support for direct S3 access
• 🌐 Optional custom domain support
• ⚙️ Easy integration with existing CDK stacks
• 💰 Cost-effective usage-based pricing model
• ♻️ Automatic artifact cleanup to manage storage costs
• 🕐 Significant build time reduction for large projects

Getting Started

Read the guide to deploy the Remote Cache API in your own AWS account and use it with your Turborepo project.

Installation

To use this construct in your CDK project, install it using npm, pnpm, or yarn:

npm install turbo-remote-cache-construct

pnpm add turbo-remote-cache-construct

Usage

Import and use the construct in your CDK stack:

import * as cdk from 'aws-cdk-lib';
import { Construct } from 'constructs';
import { TurboRemoteCache } from 'turbo-remote-cache-construct';
import * as acm from 'aws-cdk-lib/aws-certificatemanager';
import * as lambda from 'aws-cdk-lib/aws-lambda';
import * as apigateway from 'aws-cdk-lib/aws-apigateway';

export class TurboRemoteCacheStack extends cdk.Stack {
  constructor(scope: Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    // load TURBO_TOKEN from .env file
    dotenv.config({ path: path.join(__dirname, '..', '.env') });

    if (!process.env.TURBO_TOKEN) {
      throw new Error('TURBO_TOKEN is not set');
    }

    new TurboRemoteCache(this, 'TurboRemoteCache', {
      turboToken: process.env.TURBO_TOKEN,
      apiProps: {
        // optional domain name options for using a custom domain with API Gateway
        domainName: {
          domainName: 'your-custom-domain.com',
          // create a certificate in us-east-1 for custom domain
          certificate: acm.Certificate.fromCertificateArn(this, 'Certificate', 'arn:aws:acm:us-east-1:123456789012:certificate/12345678-1234-1234-1234-123456789012'),
          endpointType: apigateway.EndpointType.EDGE,
          securityPolicy: apigateway.SecurityPolicy.TLS_1_2,
        },
      },
      // optional S3 bucket props for to configure the artifacts bucket
      artifactsBucketProps: {
        bucketName: 'turbo-remote-cache-artifacts',
        removalPolicy: cdk.RemovalPolicy.RETAIN,
      },
      // optional DynamoDB table props for to configure the events table
      eventsTableProps: {
        tableName: 'turbo-remote-cache-events',
        removalPolicy: cdk.RemovalPolicy.RETAIN,
      },
    });
  }
}

The S3 bucket, DynamoDB table, and API Gateway can be configured using the optional props. The defaults are are provided in the jsdoc comments and can be viewed in the source code. The example above uses apiProps to configure a custom domain name for the API Gateway. It also uses artifactsBucketProps to customize the bucketName and removalPolicy to retain the bucket when the stack is deleted. Similar options are available for the DynamoDB table.

Breaking Changes in 2.0

Authentication

• Deprecated simple token-based authentication (turboToken)
• Added support for custom authorizers via authorizerFunction
• Authorizer must provide teamId in the context

Team Identification

• Removed support for teamSlug in favor of teamId
• All artifact paths now use teamId instead of slug
• API endpoints expect teamId query parameter

New Features

• Added preflight/presigned URL support for S3 operations
• Added CORS support for S3 bucket
• Added custom user info endpoint support
• Support for direct S3 access using presigned URLs

Architecture

The construct uses API Gateway, Lambda, S3 integration, and DynamoDB to create a serverless Turborepo Remote Cache REST API. The construct can be self-hosted in your AWS account and be used with the Turborepo Remote Cache.

Components

• API Gateway: Used to create a REST API that will be used to interact with the Remote Cache.
• Lambda: Used to handle the API requests and responses for non-artifact related endpoints.
• S3 Integration: Allows API Gateway to integrate with S3 without a Lambda function which allows for a larger payload size and lower latency. Supports presigned URLs for direct S3 access using preflight requests.
• S3: Used to store the Remote Cache artifacts.
• DynamoDB: Used to store the Remote Cache events.

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

License

This construct is licensed under the MIT License. See the LICENSE file for details.