docstore

Simple and extendable JSON document storage and retrieval

Installation

$ npm install docstore

Usage

var ds = require('docstore');
 
ds.open('./docs/users', function(err, store) {
  assert(err == null);
 
  store.save({ name: 'Joe', age: 35 }, function(err, doc) {
    assert(err == null);
    console.log(doc._id); // ids are autogenerated and assigned
  });
 
  // See below for other methods available on store
});

Opening

The first step to using docstore is to open a store. All other operations are available from the store returned to you via the passed callback.

open(directory, options, callback)

Arguments

• directory - the directory to read and write documents to/from
• options - an object containing store options (see below)
• callback(err, store) - callback that will be invoked after opening the store

Events

• open(err, store) - event emitted when a store is successfully opened
• open(err, docdir) - event emitted when opening a store fails

Example

var ds = require('docstore');
 
ds.open('./docs/users', function(err, store) {
  assert(err == null);
  assert(store != null);
});

options

The following options are available when opening a store

• format - the format class to use for serializing and deserializing objects. It defaults to the JSON formatter in lib/format/json.js
• idgen - custom function for generting object ids. It defaults to the generator in lib/idgen.js

Saving

When saving, the store uses the configured format implementation to first serialize the object. If the object has a _id property it's value will be used as the filename. Otherwise, a random filename will be generated.

store.save(obj, callback)

Arguments

• obj - the object to save
• callback(err, obj) - callback that will be invoked after opening the store. It is passed an error if one occurs. If no error occurs it will pass the object as saved (with it's generated id).

Events

• save(err, object) - event emitted when an object is saved, error or not

Example

store.save({ name: 'Joe', age: 35 }, function(err, doc) {
  assert(err == null);
  console.log(doc._id); // ids are autogenerated and assigned
});

Retrieving

Any document can be retrieved given its id.

store.get(id, callback)

Arguments

• id - the id of the object to get
• callback(err, obj) - callback that will be invoked after retrieving the document. It is passed an error if one occurs. If no error occurs it will pass the document.

Events

• get(err, object) - event emitted when an object is successfully retrieved
• get(err, id) - event emitted when an object retrieval fails

Example

store.get('id12302202', function(err, doc) {
  assert(err == null);
  assert(doc._id === 'id12302202');
});

Removing

There are two methods for removing documents. You can remove them one at a time or clear all documents from a store at once.

store.remove(id, callback)

Arguments

• id - the id of the object to remove
• callback(err) - callback that will be invoked after removing the document. It is passed an error if one occurs.

Events

• remove(err, id) - event emitted when an object is removed, regardless of failure or success

Example

store.remove('id12302202', function(err) {
  assert(err == null);
  console.log('Deleted');
});

store.clear(callback)

Arguments

• callback(err) - callback that will be invoked after removing all documents. It is passed an error if one occurs.

Example

store.clear(function(err) {
  assert(err == null);
  console.log('All deleted');
});

Scanning

Scanning is the method used to find particular documents among a large collection. While there's only one scan method, it can be used in two ways. You can either get a single callback with all the results as an array or get them as a stream of document events.

store.clear(callback)

Arguments

• filter - a function to be used to determine if a document should be returned from the scan
• callback(err) - optional callback that will be invoked for all documents that have passed the filter function. If no callback is passed to the scan method, an EventEmitter is returned instead. It emits document events for all documents passing the filter function, an error event if any errors occur, and an end event when scanning is finished.

Examples

var filter = function(doc) {
  return doc._id === 'id12302202';
};
 
// get all results at once
store.scan(filter, function(err, docs) {
  assert(err == null);
  assert(docs.length === 1);
  assert(docs[0]._id === 'id12302202');
});
 
// get results as a stream
var stream = store.scan(filter);
stream.on('document', function(doc) {
  assert(doc._id === 'id12302202');
});
stream.on('end', console.log.bind(null, 'Done'));
stream.on('error', console.error.bind(null));

Custom Formats

As mentioned above, the docstore.open function accepts an options object, one option of which is format. A format object is used by the store to serialize an object before writing it to disk and deserializing it after reading from disk. It's up to you to decide how it should be represented on disk. A default JSON formatter is used if you don't specify one. It stores documentas as plain text JSON.

Writing your own custom format is easy, requiring only one property to be present and two methods to be implemented.

extension

extension is a property used to indicate the file extension to use for your format. It should include a leading ., such as .json for JSON formatted files.

serialize(obj)

The serialize method is called to format a JavaScript object as it is to be represented on disk. It must return the serialized form of the object.

deserialize(buffer)

The deserialize method is called to format a buffer read from disk back into its JavaScript object form. It must return a JavaScript object.

Example Format

exports.extension = '.json';
 
exports.serialize = function (obj) {
  return JSON.stringify(obj);
};
 
exports.deserialize = function (buffer) {
  return JSON.parse(buffer);
};

Encryption

docstore includes a format object that can be used to encrypt the files stored on disk. It simply wraps another format object in an encrypted form. While it's intended to be able to transparently encrypt any underlying format, it's only been tested with the default JSON format.

constructor(algorithm, password, underlyingFormat)

Arguments

• algorithm - the encryption algorithm to use. The available algorithms is dependent on your version of OpenSSL. On recent releases, openssl list-cipher-algorithms will display the available cipher algorithms.
• password - used to derive id and initialization vector (IV)
• underlyingFormat - the underling format for serializing and deserializing JavaScript objects. If no underlying format is given the default JSON format implementation will be used.

Example

var ds = require('docstore');
var Encrypted = require('docstore/lib/format/encrypted');
var encrypted = new Encrypted('aes128', 'password');
 
ds.open('./docs/users', { format: encrypted }, function(err, store) {
  console.log('You now have an encrypted store');
});

License

(The MIT License)

Copyright (c) 2012 Near Infinity Corporation

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.