A PostgreSQL table snapshot tool for node.js.
Install it using npm:
$ npm install taara
Or get it directly from: http://github.com/macobo/taara
To use the library, you need to configure to use storage engine. At this time, the library supports on-disk backups as well as S3 Backups.
Using S3 storage
var taara = ;var storageEngine = 'my-bucket' s3AuthParams;taara;
Using on-disk storage
var storageEngine = '/path/to/backup/location';taara;
Storing a snapshot of a table.
var pgAuth =host: "localhost"port: 5432user: "postgres"database: "postgres"password: "my-password";// Stored along with the snapshot, can be later used during the restore processvar userMetadata = my: "value" ;taara;
The snapshot will be stored at
on date) and snapshot at
Listing snapshots and fetching stored metadata.
taara// Print out list of identifiers// And fetch metadata for one of the backups.;
Restoring a snapshot
At Heap Analytics we use sharding, PostgreSQL and sharding extensively to store with our user data. As each shard is replicated, even if we lose a single database instance, we won't lose any data. However, if more databases than the replication factor should die, we'd need to deal with full-blown database restores simply to get at a few shards out of tens of thousands - something which is costly in terms of storage as well as time.
This library is an attempt to deal better with that problem - instead of making full-blown database backups, we store snapshots of tables on S3 with some Kafka metadata. Should shards "disappear" we can use the snapshot to restore the shards to a good state and replay all new events that that happened since using Kafka.
To setup, you need to install a few dependencies:
npm install -g typescript gulp
To compile and to do a test run:
docker-compose up -d- starts a database and fake_s3 instance. 
: More on Docker Compose here.