node package manager
Share your code. npm Orgs help your team discover, share, and reuse code. Create a free org »



Node native addon module (C/C++) for Rabin fingerprinting data streams.

Build Status Build status

Uses the implementation of Rabin fingerprinting from LBFS.

Rabin fingerprinting is useful for finding the chunks of a file that differ from a previous version. It's one implementation of a technique called "Content-defined chunking", meaning the chunk boundaries are determinstic to the content (as opposed to "fixed-sized chunking").

Theres a JavaScript API and an accompanying command-line tool.

JavaScript API

var createRabin = require('rabin')

createRabin can be used to create multiple fingerprinting streams

var rabin = createRabin()

rabin is a duplex stream. You write raw data in, and buffers chunked by rabin fingerprints will be written out.

JavaScript Example

// require and create an instance 
var rabin = require('rabin')()
// pipe some data in 
var rs = fs.createReadStream('somefile.dat')
// handle output chunks 
rabin.on('data', function (chunk) {
  // chunks are created by taking your input data 
  // and splitting on each rabin fingerprint found 


$ npm install rabin -g
$ rabin myfile.txt --bits=14 --min=8192 --max=32768 # defaults
average 12500