Pile On
Useful when you want multiple parallel processes to be able to "pile on" data to a shared structure, such as an object or an array. But you'd like to be able to access the object/array as a single object for reading.
An alternative use case is sharding data across several files but reading it back as one object.
Initialize
Pileon takes a a pre-configured S3 object from Amazon's aws-sdk module.
var Pileon = require('pile-on')(S3);
Object Pile On
Create an object:
// Creates bucket/key
Pileon.create(bucket, key, {init: true});
Append to that object log:
// Creates bucket/key-{timestamp}
Pileon.append(bucket, key, {stageOneComplete: true});
Read the log structure:
// Returns a promise for [{init: ture, stageOneComplete: true}]
Pileon.readObject(bucket, key);
Array Pile On
Create an array:
// Creates bucket/key
Pileon.create(bucket, key, []);
Append to that array log:
// Creates bucket/key-{timestamp}
Pileon.append(bucket, key, [{id: 1}]);
Append to that array log again:
// Creates bucket/key-{timestamp}
Pileon.append(bucket, key, [{id: 2}]);
Read the log structure:
// Returns a promise for [{id: 1}, {id: 2}]
Pileon.readArray(bucket, key);