@numbereight/batch

A standardised batching mechanism with size and timeout controls

Examples

import { Batch } from '@numbereight/batch';

const batch = new Batch<JSONMember>({
  command: async function handle(data: Array<JSONMember>) {
    console.log(`got ${JSON.stringify(data)}`)
  },
  limit: 6,
  getSize(v) => {
    return JSON.stringify(v).length;
  },
  maxItems: 5,
  timeout: 100,
});

// addData has signature `(item: D) => Promise<void>`
await batch.addData([10, 20, 30, 40, 50]);
// > got [10, 20, 30]
// is printed almost immediately as it exceeds the batch limit (6 characters)
await sleep(100);
// > got [40, 50]
// is printed after ~100ms as the timeout is exceeded

await batch.addData([1, 2, 3, 4, 5, 6]);
// > got [1, 2, 3, 4, 5]
// is printed almost immediately as it exceeds the max item (5 items)
await sleep(100);
// > got [6]
// is printed after ~100ms as the timeout is exceeded

// Get observability data
batch.observe();

Errors

Errors in the command function are exposed as rejections to the commands that inserted data to that batch. This can result in an error being reported multiple times (as the batch is composed of multiple inserts).

Maintainance and Questions

Eoin maintains this package and should be able to field all questions and extra requirements.