sqs-bulk-loader

AWS SQS Bulk Loader

A set of functions to help sending bulk messages in sequence or parallel to AWS SQS.

How to install

npm install sqs-bulk-loader

aws-sdk is peer dependency for this package. Make sure it is installed.

Running the tests

npm test

Getting started

Basic syntax is:

const {sendBatchedMessages, sendBatchedMessagesInParallel} = require("sqs-bulk-loader")();

const messages = [
  {
    "Id": "1",
    "MessageBody": '{"key1": "value1"}'
  },
  {
    "Id": "2",
    "MessageBody": '{"key2": "value2"}'
  }
];

// this needs to be in async function
const response = await sendBatchedMessages("someQueueUrl", messages);
console.log(response)

// OR you can use normal promise style as well.
sendBatchedMessages("someQueueUrl", messages)
  .then(response => console.log(response));

In case you need to inject the aws-sdk with custom settings. It can be done like this

const aws = require("aws-sdk");
const sqsClient = new aws.SQS();
const {sendBatchedMessages, sendBatchedMessagesInParallel} = require("sqs-bulk-loader")(sqsClient);

API

sendBatchedMessages

sendBatchedMessages(queueUrl, messages) ⇒ Promise

Function - Sends the messages passed in batch of 10 sequentially

Returns: Promise - - promise which resolves on success and rejects on error

Param	Type	Default	Description
queueUrl	String		SQS queue url
messages	Array		Array of messages as per sendMessageBatch's params

More details - https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/SQS.html#sendMessageBatch-property

sendBatchedMessagesInParallel

sendBatchedMessagesInParallel(queueUrl, messages) ⇒ Promise

Function - Sends the messages passed in batch of 10 parallely

Returns: Promise - - promise which resolves on success and rejects on error

Param	Type	Default	Description
queueUrl	String		SQS queue url
messages	Array		Array of messages as per sendMessageBatch's params
[options]	Object	{"batchSize": 10}	Optional object containing extra properties which will be passed to function. Currently it supports batchSize integer to control how many parallel requests to spawn. Default is 10

More details - https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/SQS.html#sendMessageBatch-property