DirDB - lightning fast database
DirDB key-value directory database
$ npm install dirdb
Run tests
Browse module (e.g. node_modules/dirdb
) install directory, and run tests:
$ npm test# or $ node test.js
Compare test results with travis run tests.
Include in your script
const dirdb = ;
Define the database root directory
First time, make sure the directory exists, is empty, and have the right user permissions mode.
const db = '/dir/path/name';
Chain calls
All async method functions returns the object they belong (db
core, or db.client()
stream).
obj // < can be: db or db.client() stream // and so on...;// ASYNC call exampledb // < can be: db or db.client() stream, see below // and so on...;
isdir(dirname)
If dirname
exist, return/callback object dirconfig
, or undefined
if not.
dirname
- String directory table name (folder), without slashes, e.g.name
// SYNCconst dirconfig = db;// ASYNCdb;
mkdir(dirname[, options])
Make a directory by name, not path, e.g. name
. If dirname
exist, throw/callback error
. Return/callback name
on success. For more options
, see below.
// SYNCconst name = db;// ASYNCdb;
rmdir(dirname)
Remove dir name and its contents, throw/callback error
if dirname
not exists.
// SYNCdb;// ASYNCdb;
list()
Return/callback dbconfig
object { dirname: dirconfig, ... }
.
// SYNCconst dbconfig = db;// ASYNCdb;
put(dirname, key, value[, callback])
Throw/callback error
if key exists. Return/callback { uid, hash, path }
if success.
dirname
- String directory table name, without slasheskey
- String|Buffervalue
- String|Buffer
Return { uid, hash, path }
uid
- String unique id ( birthDateInt36.Index )hash
- String key hash ( using:dirconfig
algorithm + digest )path
- String path key ( filepath.k
) and value ( filepath.v
)
// SYNCconst uid hash path = db;// ASYNCdb;
set(dirname, key, value[, callback])
Overwrite value if key exists, or create, if not. Return/callback { uid, hash, path }
if success.
// SYNCconst uid hash path = db;// ASYNCdb;
add(dirname, key, value[, callback])
Append value
if key exists, or create, if not. Return/callback { uid, hash, path }
if success.
// SYNCconst uid hash path = db;// ASYNCdb;
get(dirname, key[, callback])
Read key value
. Throw/callback error
if key not exists. Return/callback { value, uid, hash, path }
if success.
// SYNCconst value uid hash path = db; // value is Buffer// ASYNCdb;
del(dirname, key[, callback])
Delete key
. Throw/callback error
if key not exists. Return/callback uid
if success.
// SYNCconst uid = db;// ASYNCdb;
keys(dirname[, range[, callback]])
Return/callback object keylist
if success.
range
- Object{ start: Number, end: Number }
keylist
- Object{ uid: keyhash, ... }
// SYNCconst keylistA = db; // without range select, return allconst keylistB = db; // without end point, return all except first key ( index: 1, 2, ... )const keylistC = db; // return first two keys ( index: 0 and 1 )// ASYNCdb;// without start point, return first two keys ( index: 0 and 1 )db;
val(dirname, uid, keyhash[, callback])
Throw/callback error
if key not exists. Return/callback { key, value, path }
if success. See the above keylist
object for uid
and keyhash
.
// SYNCconst key value path = db; // key and value is Buffer// ASYNCdb;
stats()
Throw/callback error
if key not exists. Return/callback { uid, hash, path, stats }
if success.
stats
- Object, key value filelstat
fs.Stats
// SYNCconst uid hash path stats = db;// ASYNCdb;
setgc(dirname, option)
Set dirname
GC boolean option. When delete a key using del()
function, if GC is enabled (true), the directory where the key-value was saved, is deleted if is empty. Throw/callback error
if dirname not exists. Return/callback dirconfig
if success.
// SYNCconst dirconfig = db;// ASYNCdb;
server()
Server stream object. See the stream / socket examples below, of how to pipe server stream into client stream.
client([sync])
Client stream object. Server call method functions is sync false
(by default), for sync set true
. The sync/async server method can be set individually on any client function, with the last argument.
// call SYNC method functions on serverconst clientS = db;// call ASYNC method functions on serverconst clientA = db; // or false const client = db;client; // true: call SYNC method function on serverclient; // false: call ASYNC method function on server
Stream example
const client = db;client;client;
Socket stream example
const net = ;net;
mkdir(dirname[, options])
dirname
- String directory table name, without slashesoptions
- Object, see below
Directory table options
level
- Number, key hash directory divisor, default3
, minim0
and max limited byalgorithm
anddigest
value, see belowdmode
- Number, directory mode, default0o700
fmode
- Number, file mode, default0o600
algorithm
- String, key hash algorithm, defaultmd5
, possible:md5
|sha1
|sha256
|sha512
digest
- String, key hash digest, defaultbase64
, possible:base64
|hex
compress
- String, zlib compress type, defaultnone
, possible:none
|deflate
|gzip
gc
- Boolean, run garbage collector after key delete, defaulttrue
You can overwrite the default directory options:
const db = '/dir/pathname' // overwrite default options level: 3 digest: 'hex' gc: false;
mkdir()
options example
High level, means high directory divisor. To increase I/O speed on high number of keys entries, make sure a high level value is defined on db.mkdir
options. If there is only few key entries on directory, high level value will decrease the I/O speed.
// dir name 'logs'db;// sha1 = 35 unique characters , level = 4console; // 1500625// key entries are stored on 1500625 max sub-directories
Few examples of how to calculate the directory divisor, at maximum level
md5-base64
Math.pow(64, 22) - 64 unique characters, 22 long max levelmd5-hex
Math.pow(35, 32) - 35 unique characters, 32 long max levelsha1-base64
Math.pow(64, 27) - 64 unique characters, 27 long max levelsha1-hex
Math.pow(35, 40) - 35 unique characters, 40 long max level
High Availability, Backup and Restore Operations
DirDB is very light and simple database, because of that, HA / Back-Up / Restore can be done with tools like rsync or unison.
# Backup example $ rsync -abqz dirA dirB# High Availability example $ unison -auto dirA/ dirB/
dirA
local directory ( primary )dirB
remote / network directory ( secondary )
// High Availability, using unisonconst dbA = dirA; // primary DB// Host A ( local )net;// dirB > ssh://dev@192.168.1.10/home/alice/dirB// OR// dirB > socket://remote_host:port_num/path/to/dirBconst dbB = dirB; // secondary DB// Host B ( remote ) - another node.js servernet;
DB core only (not client), async stream write/read big values
const fs = ;// db core, write big value, source file '/path/big/file.ext'db;// db core, append big value, source file '/path/big/file.ext'db;// db core, read big valuedb;
For more info, consult or run the test.js file.
DirDB is licensed under the MIT license. See the included LICENSE
file for more details.