s3watcher

Watch an s3 bucket for new keys

S3 Watcher

Watches an S3 bucket for CloudFront logs and an emits keys as logs are delivered.

It tracks its own state, which consists of a marker and a list of all keys that have been seen already. CloudFront logs are not delivered in order, so the watcher will scan 24 hours back from the current marker.

It's a node module so install with npm:

npm install s3watcher
var s3watcher = require('s3watcher')
 
var watcher = s3watcher({
    awsKey: 'xxxx',
    awsSecret: 'xxxx',
    bucket: 'bucketname',
    prefix: 'foobar/baz',
    namespace: 'foobar',
});
 
watcher.pipe(process.stdout);
  • awsKey and awsSecret (both required) are obviously your AWS credentials
  • bucket (required) the name of the S3 bucket to watch
  • prefix (defaults to '') is the prefix to watch
  • namespace (defaults to 'default') is a unique string that allows you to run multiple watcher instances against the same bucket and prefix pair.

Copy .s3watcherrc.example to .s3watcherrc and provide actual values. Then run:

npm test

For debug output set the DEBUG environment variable:

DEBUG=s3watcher npm test