node package manager
Stop writing boring code. Discover, share, and reuse within your team. Create a free org »

s3watcher

S3 Watcher

Watches an S3 bucket for CloudFront logs and an emits keys as logs are delivered.

It tracks its own state, which consists of a marker and a list of all keys that have been seen already. CloudFront logs are not delivered in order, so the watcher will scan 24 hours back from the current marker.

Install

It's a node module so install with npm:

npm install s3watcher

Usage

var s3watcher = require('s3watcher')
 
var watcher = s3watcher({
    awsKey: 'xxxx',
    awsSecret: 'xxxx',
    bucket: 'bucketname',
    prefix: 'foobar/baz',
    namespace: 'foobar',
});
 
watcher.pipe(process.stdout);
  • awsKey and awsSecret (both required) are obviously your AWS credentials
  • bucket (required) the name of the S3 bucket to watch
  • prefix (defaults to '') is the prefix to watch
  • namespace (defaults to 'default') is a unique string that allows you to run multiple watcher instances against the same bucket and prefix pair.

Tests

Copy .s3watcherrc.example to .s3watcherrc and provide actual values. Then run:

npm test

For debug output set the DEBUG environment variable:

DEBUG=s3watcher npm test