abstract-store

a module to abstract key/value storage away from the underlying technology

Abstract-Store

Right now this is in development & only published in NPM to reserve the name. It will be fully functional and tested in a couple of days

The problem I faced, that it was highly unclear what backend storage technology would suit our needs best. Also there was no clear means to migrate from one to another.

So the idea of this module was to provide a very few methods that any storage mechanism would need to fulfill and then implement that interface with a multitude of backend means.

In addition there is a multi backend that allows for migration by reading from the specified stores in order and doing writes by writing to the stores either in order or only to the first. Ths way one could migrate the storage from one method to the next by keeping the old mechanism around as a read-only until all the data has been migrated.

npm install abstract-store
var Store = require('store');
Store('<url>'[, <options>], function(err, storeInstance) {

});

Supported URL schemas are:

  • mongodb - a valid mongodb URL that can also be provided to MongoClient
  • riak - to connect to a riak cluster. (Example: riak://default.host.name[:port], the options can contain information on other cluster nodes)
  • s3 - describes what Amazon-S3 Bucket to use (Example: s3://bucketname, the options contain access credentials and access methodology. See knox for details)
  • levelup - is almost like a file:// but it points to a base directory to use for storage. (Example: *levelup:///path/to/storage/location)
  • memcache - permits caching via MemCached. Of course this type will not support the key operation, but it will return an empty array of keys when asked to do it none the less. (Example: memcache://your-host:port)
  • multi - chain multiple storage mechanisms (Example: multi://name-to-describe-it the important stuff is in the options)

In order to add namespaces this module contains a bucket dispatch mechanism.

var Pea = require('pea');
var Store = require('store');
var bucket1 = Pea(Store, 'url1', {});
var bucket2 = Pea(Store, 'url2', {});
Pea.all(bucket1, bucket2).done(function(bucket1, bucket2) {
  Store.bucket({ namespace1:bucket1, namespace2:bucket2 }, function(err, storage) {
    storage.put('namespace1', 'key', 'value', function() { /* callback */ });
    storage.put('namespace2', 'key', 'value', function() { /* callback */ });
  });
});

The pseudo-code above would write the data to different backend stores depending on the name passed in. This way you can set up a global storage object that your entire app can use to interact with storage mechanisms. As long as you keep the same namespaces, you can easily switch out the backend mechanisms.

(The MIT License)

© 2013 by Philipp Dunkel p.dunkel@me.com. Licensed under MIT License.

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the 'Software'), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.