hyperbase
A general purpose storage interface
Why
I like the unix filesystem API a lot but it could be better:
- watch / unwatch should be as easy to use as read()
- it should be possible to atomically write data at multiple paths
- clients should be able to resolve arbitrary symlinks
- directories are great but there should also be an index or list type collection
- these should be pageable in either direction and cheaply reorderable
- these should provide a mechanism for determining total item count without loading all data
How
The concept and library itself aims to be abstract, but each storage engine requires a concrete driver implementation. Currently there is a driver available for Cloud Firestore (Note that to work properly, the Firestore implementation relies on two Cloud Functions). It should be possible to implement additional drivers even over simple key/value stores like lmdb or leveldb for example, although these would be considerably more complex (no built in transactions, events, security, etc).
Examples
Setup:
var Hyperbase =var HyperbaseStorageFirestore =// var firebase = <get firebase handle somehow>var db =storage: firebasefirestore
Working with Maps:
// storage layout:// {// things: {// 'a-thing': {// name: 'A thing'// }// }// }var thing = dbthing
Working with Lists:
// storage layout:// {// lists: {// 'all-the-things': {// size: 2,// items: {// 'a-thing': { i: 0 },// 'other-thing': { i: 1 }// }// }// },// things: {// 'a-thing': {// name: 'A thing'// },// 'other-thing': {// name: 'Other thing'// }// }// }var allTheThings = dballTheThings
Reordering list items:
var allTheThings = dballTheThings
Creating data:
var randomKey = dbvar aNewThing =name: 'A new thing'db
Working with links:
// storage layout:// {// things: {// 'some-thing': {// name: 'Some thing',// i18n: {// es: 'x',// fr: 'y'// }// }// },// i18n: {// x: {// name: 'Alguna cosa'// },// y: {// name: 'Quelque chose'// }// }// }var thing = dbthing
Nested links (and embedded Lists):
// storage layout:// {// people: {// 'a-person': {// name: 'A person',// 'best-friend': 'b-person',// friends: {// 'b-person': 0,// 'c-person': 1// }// },// 'b-person': {// name: 'B person',// 'best-friend': 'c-person',// friends: {// 'a-person': 0,// 'c-person': 1// }// },// 'c-person': {// name: 'C person',// 'best-friend': 'a-person',// friends: {// 'a-person': 0,// 'b-person': 1// }// }// }// }var person = dbperson
Deleting data:
var thing = dbthing
Test
You'll need a Firebase and service account credentials in a file called google.json
first, then do:
$ npm run test/firestore
Caveats
- A full local / remote round trip is required to resolve each link
- Cloud Firestore driver's list size feature needs help from Cloud Functions
Changelog
- 4.0
- Abstract again, Firestore driver provided
- 3.0
- Added tests, cosmetic API changes
- 2.0
- Tightly coupled with Firebase
- 1.0
- First pass at abstract
License
MIT