jdb

Overview

This project is inspired by nedb. It aims to create a flexible database without any weird syntax. Just few APIs will make everything work smoothly. JDB is an file-append-only, in-memory, non-block IO database.

It is one of the core module of nobone.

For further infomation goto How it works?

Features

• Super fast (see the benchmark).

• Light weight. Core code is only about 200 lines.

• Promise support.

• Use full functioned javascript to operate your data, no steep learning curve.

• Support both standalone mode (server) and in-application mode (lib).

• Build-in CLI tool to interact with database directly.

Quick Start

Installation

Install jdb first.

npm install jdb

As a Lib

 
jdb = require('jdb')()
 
# The data to play with. 
some_data = {
    "name": {
        "first": "Yad"
        "last": "Smood"
    }
    "fav_color": "blue"
    "languages": [
        {
            "name": "Chinese"
            "level": 10
        }
        {
            "name": "English"
            "level": 8
            "preferred": true
        }
        {
            "name": "Japenese"
            "level": 6
        }
    ]
    "height": 180
    "weight": 68
}
 
# Init the db file before you start. 
jdb.init().then ->
 
    # Set data. 
    jdb.exec
        data: some_data
        command: (jdb, data) ->
            jdb.doc.ys = data
            jdb.save 'saved'
        callback: (err, data) ->
            console.log data # output >> saved 
 
    # Or simple way to save data. 
    jdb.exec some_data, (jdb, data) ->
        jdb.doc.arr = data.languages.map (el) -> el.name
        jdb.save()
    .then ->
        console.log 'saved'
 
    # Don't do something like this! 
    wrong = ->
        jdb.exec command: (jdb) ->
            # Error: the scope here should not access the variable `some_data`. 
            jdb.doc.ys = some_data
            jdb.save()
 
    # Get the value. Much simpler. 
    console.log jdb.doc.ys.name # output >> [ "Yad", "Smood" ] 
 

CLI tool

To allow JDB to serve multiple clients, you can start it as a http server (standalone mode).

Install jdb globally.

npm install -g jdb

See help info.

jdb -h

Interactive mode, you have two global api. You can manipulate the data via doc, and run save() to permanent your change.

jdb -i

Start server at port 8081.

jdb -p 8081

JDB action exec only accepts raw json http request (do not url-encode the body!). For example:

POST /exec HTTP/1.1
Host: 127.0.0.1:8081
Content-Length: 88

{ "data": 10, "command": "function(jdb, data) { jdb.doc.a = 1; jdb.save(jdb.doc); }" }

It will return json:

{"a":1}

JDB action compactDBFile example:

GET /compactDBFile HTTP/1.1
Host: 127.0.0.1:8081

It will return:

OK

How it Works?

It simply executes all your js code to manipulate a doc object, and append each js code to a file. Each time when you start up the JDB, it executes all the code in the file, and the last time's doc object will come back again in the memory.

It uses json to decrease database file size, which means after all javascript commands are executed, they will become a single json object.

---

API

The main api of class Jdb.

init ([options])

• options

  • dbPath {Boolean}

    Where to save the database file. Default value is jdb.db.

  • compactDBFile {Boolean}

    Whether to compact db file before start up or not. Default true.

  • promise {Boolean}

    Whether to enable promise or not. Default true.

  • error {Function}

    The error handler when initializing database.

doc

The main storage object. Readonly. Do not write its property directly.

exec ([data], command, [callback])

exec (options)

A api and the only api to interact with the data in database.

• data {Object}

  data should be serializable object. It will be send with command, see the command (jdb) part.

• command (jdb) {Function}

  A function or corresponding source code. The code in this function is in another scope (database file scope). Do not share outer variable within it, see the wrong example in quick start part.

  • jdb {Object}

    An object from which you access the functions of the database. Here's the list of its members.

    • jdb.data {Object}

      The data object that is sent from the exec (options).

    • jdb.doc {Object}

      The main storage object.

    • jdb.save ([data]) {Function}

      When your data manipulation is done, call this method to permanent your change. It will automatically call the send for you.

      • data {Object}

        The same as the data of jdb.send.

    • jdb.send ([data]) {Function}

      Send data to the callback.

      • data {Object}

        Type is Object. It should be serializable.

    • jdb.rollback() {Function}

      Call it when you want to rollback the change that you made.

• callback (err, data) {Function}

  This function will be invoked after the save or send is called.

  • err {Object}

    It can only catch sync errors, you should handle async errors by yourself.

  • data {Function}

    The data you send from jdb.send(data) or jdb.save(data).

compactDBFile ()

Returns a promise. Reduce the size of the database file. It will calculate all the commands and save the final doc object to the file and delete all the other commands.

compactDBFileSync ()

The sync version of compactDBFile (callback).

---

Unit Test

npm run no -- test

Benchmarks

To run the benchmark:

npm run no -- benchmark

Though for MongoDB and Redis, most of their CPU time is ate by their DB adapters, but I think for some small projects, such as personal blog, or a non-cluster application, the adapter issue should also be taken into consideration.

JDB on Intel Core i7 2.3GHz SSD

• insert x 17,500 ops/sec ±3.08% (75 runs sampled)
• query x 803,273 ops/sec ±1.06% (94 runs sampled)

NeDB on Intel Core i7 2.3GHz SSD

• insert x 5,195 ops/sec ±2.94% (73 runs sampled)
• query x 336 ops/sec ±1.87% (83 runs sampled)

MongoDB on Intel Core i7 2.3GHz SSD

• insert x 3,744 ops/sec ±2.63% (76 runs sampled)
• query x 2,416 ops/sec ±3.89% (70 runs sampled)

Redis on Intel Core i7 2.3GHz SSD

• insert x 10,619 ops/sec ±2.33% (77 runs sampled)
• query x 10,722 ops/sec ±2.27% (80 runs sampled)

JDB http server on Intel Core i7 2.3GHz SSD

• exec x 1,111 ops/sec ±5.90% (60 runs sampled)

Road Map

• More fault tolerance support. Such as file system error handling.

• Maybe simple cluster support.

License

BSD

May 2014, Yad Smood