RedKV is a key-value store interface for node.js which:
- Provides a uniform key-value interface for a variety of SQL or NO-SQL databases
- Connects to multiple databases possibly with different types
If multiple databases are configured, the reading and writing behave as if it is a caching system. When writing to RedKV, the key-value pair gets to write to all databases. When getting a key, RedKV sequentially tries a database from the front of the list, moves on to the next database if the key was not found, and stops until the key is found in a database. After that, the key-value pair is filled to all the databases in front of the list, up to the database where the key is actually found, just like filling a cache.
RedKV currently supports the following databases as its backend:
RedKV supports node.js version 6 and above.
Installation
Install with:
npm i redkv
Example
Let's create a RedKV with a Redis store at the front and a DynamoDB store at the back end:
let kvStore = new RedKV();
kvStore.addStore('redis', {
host: "127.0.0.1",
port: 6379,
db: 0
});
kvStore.addStore('dynamodb', {
region: "us-east-1",
endpoint: "https://DynamoDB.us-east-1.amazonaws.com",
httpOptions: {
timeout:3000
},
tableName: 'dev.calculator'
});
We need to wait until all the store finishes initialization, which is when the Promise
returned from .ready()
resolves:
kvStore.ready().then(()=>{
// Start using kvStore here
})
Now we can add key-value pairs to it:
kvStore.set('mykey', 'my string value');
// Set function returns a promise which resolves when all the stores finish setting the key, which in this case are redis and DynamoDB.
We can also read from redkv using .get(key)
kvStore.get('mykey')
.then(value=>{
// value should equal 'my string value'
})
The API .get(key)
will first try to read from the redis store. If succeeds, it resolves to the value found. If fails or the key does not exist in redis, it loads it from DynamoDB, writes it back to redis, and returns a promise that resolves to the value found.
APIs
RedKV supports the following APIs:
constructor
let kvStore = new RedKV();
.addStore(storeName, option) => store
The addStore API adds a new database (store) to redkv.
let kvStore = new RedKV();
//...
kvStore.addStore(storeName, options)
where storeName
is a string name of the store, which can be one of the supported databases:redis
, DynamoDB
, mongodb
, mysq
, and postgres
. Names of the databases are case-insensitive. The parameter options
are the options passed to the individual stores. Please refer to the section of the individual databases (stores) for details.
The .addStore
API returns a store object that has the same API as the RedKV object itself, but points to only this store instead of a list of stores.
.get(key) => Promise
The .get
API reads the value from the stores based on the key. If the key exists, it returns a promise which will resolve to a string value. Even if you set a number using the .set
API, the value get the promise returned from the .get
API will still be a string. See the .set
API for more details.
let kvStore = new RedKV();
//...
kvStore.get(key)
The key should be a string. Numbers will be automatically converted to strings, other data types will cause this API to return a Promise
that rejects.
When there are multiple stores in RedKV, the get
API tries to get the key from the stores sequentially, until it finds the key in a store S1
and gets it value. After that, it fills the key-value pair to the stores before store S1
. If none of the stores contains the key, the get
API returns a Promise
which will eventually resolve to null
.
The stores are managed as a sequential list in the same order it is added into RedKV.
For example, if a RedKV instance contains two stores with the .addStore
API: a instance of Redos and a DynamoDB. When the .get
API is called with a key, RedKV tries to find the key from Redis. If Redis contains the key, the .get
API will return a promise that will eventually resolve to the value in Redis. If Redis does not have the key, RedKV will try to get the value from DynamoDB, fill the value into Redis, and return a Promise that resolves to the value. If neith Redis nor DynamoDB has the key, the .get
API will return a Promise that resolves to null
.
.set(key, value) => Promise
The set
API writes the key-value pair to all stores in the RedKV instance. Both the key and the value are stored as strings. If the key or the value is a number, it will be converted to a string. Other non-string types, such as objects, undefined
, or null
, will cause this API to return a promise that rejects.
let kvStore = new RedKV();
//...
kvStore.set(key, value)
The set
API returns a Promise
which will resolve when all the stores set the key-value pair successfully or reject when one of the stores fails.
.delete(key) => Promise
The delete API deletes a key from all the stores. The delete
API returns a Promise
which resolves when all the stores finish deleting the key or rejects when any store fails to delete the key.
let kvStore = new RedKV();
//...
kvStore.delete(key)
.has(key) => Promise
The has API returns a Promise
that will resolve to a boolean value indicating if the key exists in one of the stores.
let kvStore = new RedKV();
//...
kvStore.delete(key)
.then(exists=>{
//exists is true when key exists in at least one store,
// and is false if it does not exist in any store
})
Stores
Redis
The Redis store can be added to a RedKV instance using 'redis'
as the parameter passed to the addStore
API:
let kvStore = new RedKV();
kvStore.addStore('redis', options);
//options : {
// host: "127.0.0.1",
// port: 6379,
// db: 0
// }
The options
object will be passed to node_redis module. Please refer to node_redis document for details.
Amazon DynamoDB
The DynamoDB store and be added to a RedKV using 'DynamoDB'
as the parameter passed to the addStore
API:
let kvStore = new RedKV();
kvStore.addStore('dynamodb', options);
The options
object will be passed to DynamoDB document client. Please refer to DynamoDB document client for details.
In addition to the fields passed to DynamoDB document client, the options
object may contain the following additional fields:
tableName
The tableName
is your DynamoDB table name. This field is required.
attributeName
Optional. If attributeName is set, RedKV stores the value in an attribute with the name indicated by the attributeName
field. If it is not set, it defaults to 'v'
.
The DynamoDB table should contain only one hash key and no partitional keys.
MongoDB
The MongoDB store can be added to a RedKV using 'mongodb'
as the parameter passed to the addStore
API:
let kvStore = new RedKV();
let options={url:'mongodb://localhost:27017'};
kvStore.addStore('mongodb', options);
options
is an object. All fields are optional. Supported fields include url
, collection
, keyField
, valueField
.
url
The URL to the MongoDB instance. If not provided, a default value of 'mongodb://localhost:27017/redKV'
is assumed.
collection
The collection of MongoDB where the data will be stored.
keyField
The field in the collection which will be used to store keys. This field will be used to create a unique index. If omitted, it will default to 'redK'
.
valueField
The field in the collection which will be used to store values. If omitted, it will default to 'redV'
.
dbName
The Database Name used in this store. If it is already part of the url. this parameter is optional.
MySQL
The MySQL store uses a table in a MySQL database for storing key-value pairs.
Before using, you are responsible for creating the table in your database. An example is:
CREATE TABLE redkv (redk VARCHAR(256), redv TEXT, UNIQUE(redk));
In this example, the table name is redkv
. A column named redk
is created with unique restrictions for storing the keys. A column named redv
is created to store the values. You are of course free to adjust these names. However, the column needs to store the key needs to have UNIQUE restrictions.
The configuration options need to be passed to the .addStore
method of RedKV
. The options including the name of the table, the column which will be used to store the key, and the column which will be used to store the values, and other parameters that are used to connect to the MySQL server, Example:
let kvStore = new RedKV();
let options = {
//Parameters used by mysqljs
connectionLimit : 10,
host : '127.0.0.1',
user : 'redtester',
password: 'redtesterpwd',
database: 'redkv_test',
//parameters used by RedKV to create the store
tableName: 'redkv',
keyColumn: 'redk',
valueColumn: 'redv'
};
kvStore.addStore('mysql', options);
The MySQL store of RedKV uses three additional parameters on top of the other parameters supported by mysqljs: They are:
tableName
Name of the table in the database to store the key-value pairs.
keyColumn
Name of the column which will be used to store keys.
valueColumn
Name of the column which will be used to store values.
PostgreSQL
The PostgreSQL store uses a table in a PostgreSQL database for storing key-value pairs.
Before using, you are responsible of creating the table in your database. An example is:
CREATE TABLE redkv (redk VARCHAR(256), redv TEXT, UNIQUE(redk));
In this example, the table name is redkv
. A column named redk
is created with unique restrictions for storing the keys. A column named redv
is created to store the values. You are of course free to adjust these names. However, the column needs to store the key needs to have UNIQUE restrictions.
The configuration options need to be passed to the .addStore method of RedKV. The options including the name of the table, the column which will be used to store the key, and the column which will be used to store the values, and other parameters that are used to connect to the PostgreSQL server, Example:
let kvStore = new RedKV();
let options = {
//Parameters used by node-postgres
host : '127.0.0.1',
user : 'redkvtester',
password: 'redtesterpwd',
database: 'redkv_test',
//parameters used by RedKV to create the store
tableName: 'redkv',
keyColumn: 'redk',
valueColumn: 'redv'
};
kvStore.addStore('postgres', options);
The PostgreSQL store of RedKV uses three additional parameters on top of the other parameters supported by node-postgres: They are:
tableName
Name of the table in the database to store the key-value pairs.
keyColumn
Name of the column which will be used to store keys.
valueColumn
Name of the column which will be used to store values.
Memory
The memory database uses a simple Map as the backend, therefore everything is stored in memory.
let kvStore = new RedKV();
kvStore.addStore('memory', options);
License
RedKV is released under MIT license.