A standardised batching mechanism with size and timeout controls
import { Batch } from '@numbereight/batch';
const batch = new Batch<JSONMember>({
command: async function handle(data: Array<JSONMember>) {
console.log(`got ${JSON.stringify(data)}`)
},
limit: 6,
getSize(v) => {
return JSON.stringify(v).length;
},
maxItems: 5,
timeout: 100,
});
// addData has signature `(item: D) => Promise<void>`
await batch.addData([10, 20, 30, 40, 50]);
// > got [10, 20, 30]
// is printed almost immediately as it exceeds the batch limit (6 characters)
await sleep(100);
// > got [40, 50]
// is printed after ~100ms as the timeout is exceeded
await batch.addData([1, 2, 3, 4, 5, 6]);
// > got [1, 2, 3, 4, 5]
// is printed almost immediately as it exceeds the max item (5 items)
await sleep(100);
// > got [6]
// is printed after ~100ms as the timeout is exceeded
// Get observability data
batch.observe();
Errors in the command
function are exposed as rejections to the commands that inserted data to that batch. This can result in an error being reported multiple times (as the batch is composed of multiple inserts).
Eoin maintains this package and should be able to field all questions and extra requirements.