aws-sdk-tracer

This is a library intended to be used to instrument the AWS NodeJS SDK by wrapping the library and tracing calls made to the AWS API.

Context

Traces can be collected from a wrapped AWS SDK library (required by dependent) by either injecting a custom AWS.Request class which will parse successful transactions or by providing a custom logger to AWS.config.logger.

Traces are collected in an in-memory catalog which digests transactions into ervice utilization information which can be manually printed and/or retrieved through the wrapper. Additionally, there are built-in exporters which provide means for exporting the collected trace data.

The primary goal of this tracing is to facilitate developer knowledge of AWS service actions and resources used during development in order to generate tightly scoped policies for components deployed into an AWS account without the need to parse AWS CloudTrail logs for an account potentially used by several engineers. This can be done by specifying the filesystem exporter when wrapping the AWS SDK. This exporter will maintain a temporary file with data regarding the utilization of the SDK during runtime and between executions. At any point this file can be provided to a command line script gen-policy [<path>] (where <path> is the optional path to the exported file which, by default, is located at /tmp/aws-sdk-tracer.json). The script will attempt to parse the contents and generate an AWS Policy Document with statements pertaining to how each service is used.

Example:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "ECSUtilization",
      "Action": [
        "ecs:listClusters",
        "ecs:listTasks"
      ],
      "Effect": "Allow",
      "Resource": [
        "arn:aws:ecs:us-east-1:931703384005:cluster/dlp",
        "arn:aws:ecs:us-east-1:931703384005:cluster/devcoreinfra"
      ]
    }
  ]
}

Note: Policies generated by this library are very simplistic and not necessarily intended to be used as-is. Rather, the expectation is that these policy documents can be used as a jump-off point for determining how to scope the policies for your component once deployed. All statements generated will in effect "Allow" the actions which were traced.

Usage

Install the package using NPM as you would:

npm install --save aws-sdk-tracer

The quickest way to get started is to simply load the library and aws-sdk together:

const {AWS, wrapper} = require('aws-sdk-tracer').wrapAWSSDK(require('aws-sdk')); 
 
// ... after creating and using AWS services ...
 
wrapper.printUtilization();

This will utilize the default tracer type which will replace the AWS.Request class.

Since no exporters were specified, all trace data is in memory so wrapper.printUtilization() is used to print the utilization data.

The wrapper also supports options for console logging trace/AWSLogger events for each request sent as well as specifying various exporters.

const AWSSDKTracer = require('aws-sdk-tracer');
 
const wrapperOpts = {
    logger: console,
    exporters: [
        'filesystem',
    ]
};
 
const {AWS, wrapper} = AWSSDKTracer.wrapAWSSDK(require('aws-sdk'), wrapperOpts); 

Optionally, you can set the tracer option to "Logger" which, instead of injecting the traced AWS.Request class will provide a custom logger to the AWS SDK config.

const AWSSDKTracer = require('../');
 
const wrapperOpts = {
    logger: console,
    tracer: 'Logger',
    exporters: [
        'filesystem',
        'http'
    ]
};
 
const {AWS, wrapper} = AWSSDKTracer.wrapAWSSDK(require('aws-sdk'), wrapperOpts); 

Note: In either case, you do not need to use the AWS SDK as returned by the wrapper as the library itself is effectively wrapped.

const AWSSDKTracer = require('aws-sdk-tracer');
const AWS = require('aws-sdk');
 
const wrapperOpts = {
    logger: console,
    exporters: [
        'filesystem',
    ]
};
 
AWSSDKTracer.wrapAWSSDK(AWS, wrapperOpts); 
 
const ECS = new AWS.ECS();
 
// ... etc.
 

Warning: The wrapper will only wrap the AWS-SDK once if it is not already wrapped with either the Logger or Request options. If for some reason you want to change the type, the library must first be unwrapped then wrapped again.

Example:

const AWSSDKTracer = require('../');
 
const wrapperOpts = {
    logger: console,
    tracer: 'Logger',
    exporters: [
        'filesystem',
        'http'
    ]
};
 
const {AWS, wrapper} = AWSSDKTracer.wrapAWSSDK(require('aws-sdk'), wrapperOpts); 
 
try{
  wrapper.wrap(AWS); // throws error
} catch() {
  wrapper.unwrap(AWS);
  wrapper.setTracer('Request');
  wrapper.wrap(AWS); // good
}

Todo

There is a lot more that can be done; this is only touching the surface.

• finish implementation of HTTP exporter
  • the goal of this exporter would be for private utilization telemetry data
• better/more tests
• better options and configurability of wrapper and other components

TypeDocs

TypeDocs can be found onthe associated GitHub IO: https://ccampanale.github.io/aws-sdk-tracer/