unik
unik is a unique identifier generator for creating shardable and sortable IDs (non-RFC compliant).
warning: unik is experimental software
Install
npm install unik --save
Make sure you specify the unik version in your package.json.
Usage
var Unik = require('unik')
, db = require('mydb')
var unik = new Unik()
db.put(unik.flake(), data, function (err, res) { ... })
The Unik constructor also has a .create shortcut for convenience:
var unik = require('unik').create()
var id = unik.flake()
The constructor takes an options argument:
new Unik({
// base: use `10` to get the ID as an integer
base: 36
// unique: unique identifier < 1024 for `unik.flake()`. Use `.workerID` in a cluster, for example.
unique: process.pid % 1024
// seed: random seed OR unique identifier for `unik.bigflake()`
seed: random 0-1048576
// separator. Set to '' to get a string w/o breaks (makes timestamp not extractable)
sep: '-'
// epoch: custom start epoch. If you change this the resulting dates must fit in 41 bits
// Defaults to Sun, 01 Jan 2012 00:00:00 GMT
epoch: 1325376000000
// If set, will be appended to the end of the ID using `.separator`; useful if you need to store
// some data in the key itself. Resulting key length will be dependent on this value.
append: ''
})
unik.flake()
Snowflake-inspired 64-bit IDs. Uses a custom epoch start, the .unique option along with an internal counter for generating up to 8191 ids/ms.
| Input | Size |
|---|---|
| Time (milliseconds) | 41 bits |
| Counter sequence | 13 bits (*8192) |
| Process ID | 10 bits (*1024) |
Due to javascript's 53-bit integer limit, the ID is split in two parts, time-counter|pid.
.flake can generate a total of 18,446,744,073,709,551,616 values.
unik.bigflake()
96 bit IDs. Uses full unix epoch, node process.hrtime nanosecond resolution timer and a random seed, doesn't require a unique identifier.
| Input | Size |
|---|---|
| Time (milliseconds) | 53 bits |
| Time (nanoseconds) | 23 bits (*8388608) |
| Random seed | 20 bits (*1048576) |
.bigflake can generate a total of 79,228,162,514,264,337,593,543,950,336 values.
Performance and collisions
unik.flake() can generate 700k ids/second using two workers on a C2D 2.4ghz. It is collision-free as long as the .unique value is different for each process. Here is it after generating over a billion IDs:
unik.bigflake() is about 3x slower. It should be collision-free up to a few million ids/second, if you somehow manage that. There is an infinitesimal possibility of generating a duplicate, if two IDs are generated by separate processes at the exact same nanosecond and they contain the same random seed. Even though it uses the native Math.random it should be pretty safe. To guarantee no collisions, give it a known unique value using the seed option.
It has built-in protection for backwards-flowing time by holding back ID generation until the next clock tick.
Test
npm test or node test/sharding.js [workers] [burst_size] is designed to test for collisions using multiple processes. The main process will break when a collision is found.