aws-streams

    1.5.0 • Public • Published

    aws-streams

    This library offers stream-like interfaces to various AWS services.

    Most streams will take care of using a batch API when given a batch of records, but it is up to the user to ensure the batch size is appropriate. The same is true for any given limit on the size of an individual record or set of records.

    All streams include, where appropriate, backoff and retry with jitter.

    The goal of this library is to be minimal and essentially do the least amount of work required to handle the data so that the user can decide how best to process it (e.g. fan out to Lambda workers)

    Usage

    const { service } = require('aws-streams');
     
    new service.Method('...', options);

    The first argument depends on the service - it might be the SQS Queue URL, or the Firehose Delivery Stream Name, etc. and the second argument is options, which is shown here with defaults:

    options: {
      client: undefined // custom client to use, or else new one is created from aws-sdk,
      retryLimit: 3,
      retryDelay: 200
    }

    Example

    This uses a custom-configured SQS client and longer retry delay, to simply empty an SQS queue:

    const aws = require('aws-sdk');
    const { sqs } = require('aws-streams');
    const stream = require('stream');
     
    const client = new aws.SQS({ ... });
    const retryDelay = 500;
     
    const receiveMessage = new sqs.ReceiveMessage('queue-url', { client, retryDelay })
    const processMessage = new stream.Transform({
      transform: (message, encoding, cb) => {
        cb(null, message.ReceiptHandle);
      }
    });
    const deleteMessage = new sqs.DeleteMessage('queue-url', { client, retryDelay });
     
    receiveMessage.pipe(processMessage).pipe(deleteMessage);

    Notes/Details

    DynamoDB

    DynamoDB uses and expects the DocumentClient as the 'client', not the native / raw DynamoDB service.

    Keywords

    Install

    npm i aws-streams

    DownloadsWeekly Downloads

    0

    Version

    1.5.0

    License

    MIT

    Unpacked Size

    18.1 kB

    Total Files

    22

    Last publish

    Collaborators

    • dougmoscrop