North Pittsburgh Meatpackers

    @sagacify/sqs-move
    TypeScript icon, indicating that this package has built-in type declarations

    3.0.2 • Public • Published

    SQS-Move

    npm version ci semantic-release

    Moving SQS messages with ease. Messages MessageAttributes, MessageDeduplicationId & MessageGroupId will be preserved. Other Attributes will be lost.

    Motivation

    Most of the time when you use an SQS queue you also define a deadletter queue. This is a great idea, so when your service has a bug you can correct it and then repush all the messages in your deadletter queue to the orignal queue. Unfortunatly the AWS interface doeesn't provide an action to move your message. You have to do it programmatically, this is where sqs-move will help you.

    Of course, you can use it to move messages from any queue to any other queue.

    Installation

    $ npm install @sagacify/sqs-move

    Usage

    sqs-move is a simple function.

    Signature

    async function(sqsInstance, fromQueueUrl, toQueueUrl, {
      batchSize = 1,
      includes = null,
      excludes = null,
      transformBody = null,
      transformMessageAttributes = null,
      json = true
    } = {})

    Notes:

    • sqsInstance is expected to be an AWS.SQS instance
    • batchSize maximum is 10
    • includes & excludes are expected to be criteria a string or a flat key/value object (see: advanced exemple)
    • transformBody a function that takes the message original Body & *MessageAttributes as parameter and return a transformed body
    • transformMessageAttributes a function that takes the message original Body & *MessageAttributes as parameter and return a MessageAttributes body
    • json indicates if it is need to json parse message body on for includes and/or exclude

    * MessageAttributes is a simple map, it is automatically parsed and composed for you

    More on includes excludes

    If it is a string then message is only going to be moved if the message contains (includes) or not contains (excludes) the string provided.

    If it is a key/value object then message is only going to be moved if the message contains (includes) or not contains (excludes) the key/value provided. You can includes/excludes on deep property using the flat keys. This { 'user.address.country': 'BE' } will check in body if body.user.address.country === 'BE'.

    Simple example

    import AWS from 'aws-sdk';
    import sqsMove from '@sagacify/sqs-move';
    // OR
    const AWS = require('aws-sdk');
    const sqsMove = require('@sagacify/sqs-move');
    
    const sqsInstance = new AWS.SQS({
      accessKeyId: 'some-aws-id',
      secretAccessKey: 'some-aws-secret',
      region: 'eu-west-1'
    });
    
    const { movedCount } = await sqsMove(
      sqsInstance,
      'https://sqs.eu-west-1.amazonaws.com/123456789012/some-dead-letter-queue',
      'https://sqs.eu-west-1.amazonaws.com/123456789012/some-process-queue'
    );
    
    console.log(`${movedCount} messages are back in process queue !`);

    Advanced example

    import AWS from 'aws-sdk';
    import sqsMove from '@sagacify/sqs-move';
    // OR
    const AWS = require('aws-sdk');
    const { sqsMove } = require('@sagacify/sqs-move');
    
    const sqsInstance = new AWS.SQS({
      accessKeyId: 'some-aws-id',
      secretAccessKey: 'some-aws-secret',
      region: 'eu-west-1'
    });
    
    const { movedCount, filteredCount } = await sqsMove(
      sqsInstance,
      'https://sqs.eu-west-1.amazonaws.com/123456789012/some-dead-letter-queue',
      'https://sqs.eu-west-1.amazonaws.com/123456789012/some-process-queue', {
        batchSize: 10,
        includes: { 'user.name': 'olivier' },
        excludes: { 'user.country': 'BE' },
        transformBody: (body, messageAttributes) => {
          body.user.country = 'US'
          body.traceId = messageAttributes.traceId
    
          return body;
        },
        transformMessageAttributes: (body, messageAttributes) => {
          // Removes traceId from messageAttributes
          const { traceId, ...newMessageAttributes} = messageAttributes
    
          return newMessageAttributes;
        },
        json: true
      }
    );
    
    console.log(`${movedCount} messages are back in process queue & ${filteredCount} stayed in the deadletter queue !`);

    Install

    npm i @sagacify/sqs-move

    DownloadsWeekly Downloads

    39

    Version

    3.0.2

    License

    MIT

    Unpacked Size

    16.2 kB

    Total Files

    6

    Last publish

    Collaborators

    • sagacify