EruDB (e-roo-deebee) is a persistent, distributed, and fault-tolerant key-value data store written in ES6, powered by NodeJS, and backed by LevelDB. It features durable replication, service discovery, real-time events and superior at-rest and at-transport encryption features.
EruDB is designed and maintained by Seandon 'Eru' Mooy.
This is a truly open project! Pull requests will be merged and we will happily give out contributor access to those who contribute!
Firstly, because there is no simple cross platform datastore that can be easily installed as an NPM dependency. Secondly, solutions like RethinkDB and Redis are excellent, but have large dependency chains, do not work on all platforms, are not npm-tracked dependencies, and are easily misconfigured.
- Have a valid build chain for node-gyp
npm install --save erudb
- CLI Usage:
erudb -X GET localhost:9040/database/key
Within your app:
const db =host: '127.0.0.1'port: 9040db
const db =port: 9040peers: 'addr1' 'addr2...'
Note that all
Services are also
const db =port: 9040
EruDB also understands service discovery -
Service can simply accept a list of peers - but you may also use load balancing (or DNS round-robbin) and an expected number of peers and EruDB will sort that out for you as well!
const db =port: 9040expectedPeerCount: 3peers:loadBalancer: 'load_balancer:port'
Alternatively, a Redis cache can also be used for service discovery (works great with elasticache!):
const db =port: 9040peers:redis:host: 'some_redis'port: 9000db: 0key: 'erudb_peers'
Concepts & Terms
Safe, Strong, Weak
A Safe operation waits for quorum and accepted replication of data. Safe operations are the slowest & most reliable of all operations in EruDB.
A Strong operation waits for the master shard to store and accept the data. It does not wait for quorum replication, but returns once the master shard has OKed the data. Strong mode is the default mode for reads and writes in EruDB.
A Weak operation does not wait for data safety. It returns immediately upon handing the data to the EruDB Service. This mode is only appropriate for cache data and should be considered volatile.
A disk is a single independent storage unit. A disk can be a physical storage medium, a logical volume, or a cloud storage service like AWS S3. Multiple instances of EruDB can write to the same disk, but each instance will be aware of the actual replication that is represented. For example, EruDB will not attempt to replicate data to another instance of EruDB which would write to the same disk as the original instance. A warning will be issued if EruDB cannot reach it's desired replication factor relative to the number of disks in the cluster. This default is 3.
A host is a single independent compute unit. In the case of Docker, a host is not the container, but the physical host which the container resides on. A warning will be issued if EruDB cannot reach it's desired replication factor relative to the number of hosts in the cluster. This default is 3.
A group is an association of hosts which share some single point of failure. By default, we assume that the 3rd octet of the instances IP address represents a group (10.0.0.0 and 10.0.1.0 are separate "groups"). This can be overridden by configuration or service discovery plugins. A warning will be issued if EruDB cannot reach it's desired replication factor relative to the number of groups in the cluster. This default is 3.
A region an association of hosts which share a geographical location. Regions, by default, use public IP detection to determine geographically distinct hosts, and can be overridden by configuration or discovery plugins. A warning will be issued if EruDB cannot reach it's desired replication factor relative to the number of regions in the cluster. This default is 3.
An Index is not a natively supported feature of EruDB. EruDB is a key-value store. However, EruDB will feature ElasticSearch and Apache Spark integration for it's query language, allowing full text indexing of your data store.
Limited by design
"Limited by design" is an important philosophy of EruDB. Here are some things you cannot do with EruDB:
- You cannot use plaintext communication, ever. EruDB instances are secure.
- You cannot avoid authorization, ever. EruDB instances are auditable.
- You cannot insert invalid data, ever. EruDB instances are sane.
- You cannot disable warnings for clusters which are not highly available by design. EruDB clusters are resiliant.
Some settings can be changed in realtime via the management interface, but this is highly discouraged. Instead, treat EruDB like an immutable web service and launch new instances with new configuration and destroy old instances when ready.