tempo

Rotating Counters for Nodejs

Tempo

Scalable time-based counters that meant logging, graphing and providing a realtime statistics. All features in tempo are meant to have a constant size memory footprint defined by configuration.

For a quick example, please look at examples/simple-use-case.js

Features

  1. simple counters that sync over redis using a hash structure
  2. counters that keep track of historical data in buckets
  3. syncing with redis for aggregation on multiple servers
  4. throttling

Use Case

Lets say you are running a website and want to know in realtime where people are visiting.

 
  var redis = require('redis').createClient();
  var tempo = require('tempo');
 
  // create middleware to track counts 
  // create time counter, increment and sync 
  var min = tempo.min();
 
  app.use(function (reqresnext) {
    // the '1' is unnecessary because increment defaults to '1' 
    min.inc('requests', 1); 
    if (min.getCount('requests') > 1000) return next('throttled');
    next();
  })
 
  function showTotalRequestsInPastMin() {
    min.getKeys().forEach(function (k) { console.log(k, min.getCount(k)) });
  }
 
  function showRequestsOverTime() {
    min.eachCount('requests', function (counttime) {
      console.log("Requsts at " + (new Date(time)).toString() + '' + count); 
    });
 
    console.log(tempo.getCount('requests') + ' request(s) made in the last minute'); 
  }
 

Counter

The tempo TimedCounter class allows you to create a datastore object and keep data in memory.

  1. options hash
  2. per: milliseconds per bucket
  3. buckets: number of buckets
  4. timeout (optional): Time to live. Mainly for redis syncing. Defaults to (perbucket)+per2

Example for keeping data up to an hour of history:

var tempo = require('tempo');
var ds = new tempo.TimedCounter({ per: 60000, buckets: 60 });
  1. key: entity name
  2. n (optional, defaults to 1): a number to increment by.

Keeping track of how many times a user has logged in in the past hour:

  var ds = require('tempo').hour();
  ds.inc(userId);
  1. key: entity name

Grabbing logged in counts:

  var history = ds.getHistory(userId);

Returns an array of counts (per bucket)

  1. redis client
  2. prefix/namespace for the key to store in redis
* tempo's keys will look something like "<namespace>:<timestamp>"
  1. O(nt) where n is the number of keys and b is the number of buckets
  counter.sync(redis, 'web-stats', callback);

returns and array of all the keys in the counter O(nt) where n is the number of keys and b is the number of buckets

Runs a callback against every bucket in the counter with arguments (see examples below):

counter.eachCount('key1', 'key2', function (keyCount1, keyCount2, time) {
  console.log('key1', keyCount1, ' at ' + (new Date(time)).toString());
  console.log('key2', keyCount2, ' at ' + (new Date(time)).toString());
});
 
 
# Syncer
 
```javascript
var tempo = require('tempo');
var syncer = new tempo.Syncer(redisClient);

Adds a counter to the list of counters to sync at once

Shortcut to instantiate counter and add it

Syncs all counters to redis (push and pull)

Pushes data to redis

Pulls data from redis

Starts running a type of sync at given intervals between syncs

syncer.start('push', 3000) // push to redis every 3 seconds