node package manager
Share your code. npm Orgs help your team discover, share, and reuse code. Create a free org »


S3 Watcher

Watches an S3 bucket for CloudFront logs and an emits keys as logs are delivered.

It tracks its own state, which consists of a marker and a list of all keys that have been seen already. CloudFront logs are not delivered in order, so the watcher will scan 24 hours back from the current marker.


It's a node module so install with npm:

npm install s3watcher


var s3watcher = require('s3watcher')
var watcher = s3watcher({
    awsKey: 'xxxx',
    awsSecret: 'xxxx',
    bucket: 'bucketname',
    prefix: 'foobar/baz',
    namespace: 'foobar',
  • awsKey and awsSecret (both required) are obviously your AWS credentials
  • bucket (required) the name of the S3 bucket to watch
  • prefix (defaults to '') is the prefix to watch
  • namespace (defaults to 'default') is a unique string that allows you to run multiple watcher instances against the same bucket and prefix pair.


Copy .s3watcherrc.example to .s3watcherrc and provide actual values. Then run:

npm test

For debug output set the DEBUG environment variable:

DEBUG=s3watcher npm test