@defra-fish/sqs-receiver-service

    1.21.0 • Public • Published

    sqs-receiver-service

    This service is configured to poll a set of SQS queues and on receipt of a message from the queue it will notify a subscribing service.

    If the response from the subscriber is successful the message will be removed from the queue. If not the message will become visible again after the visibility timeout and maybe reprocessed subject to the SQS re-drive policy.

    Environment variables

    name description required default valid notes
    NODE_ENV Node environment no development, test, production
    AWS_REGION The AWS region to use yes See AWS Regions
    AWS_SQS_ENDPOINT Custom SQS Endpoint no Region specific Used to override the SQS service endpoint for local development
    RECEIVER_PREFIX Prefix for reading environment variables specific to a receiver instance yes The prefix to use to retrieve further settings via the environment (see below)
    <RECEIVER_PREFIX>_URL The SQS queue URL yes
    <RECEIVER_PREFIX>_WAIT_TIME_MS The amount of time to block while long polling a queue for messages no 20000 (20 seconds)
    <RECEIVER_PREFIX>_VISIBILITY_TIMEOUT_MS The length of time the message is made invisible to other processes yes
    <RECEIVER_PREFIX>_MAX_POLLING_INTERVAL_MS The maximum amount of time to wait between subsequent read requests no 300000 (5 minutes) For each read request returning no messages, an exponential delay is calculated before reading again. This limits that delay.
    <RECEIVER_PREFIX>_ATTEMPTS_WITH_NO_DELAY Number of receive requests with no messages before imposing a delay no 10 Receive message requests may return no messages even if there are messages waiting. Allow this many requests before delaying.
    <RECEIVER_PREFIX>_SUBSCRIBER The location of the subscriber. yes
    <RECEIVER_PREFIX>_SUBSCRIBER_TIMEOUT_MS Prefix for reading environment variables specific to a receiver instance no 180000 (3m)
    DEBUG Enable debug output no sqs:*, sqs:receiver, sqs:read-queue, sqs:process-message, sqs:delete-messages, sqs:queue-stats Enables debug output for the given category, accepts wildcards
    AIRBRAKE_HOST URL of airbrake host no
    AIRBRAKE_PROJECT_KEY Project key for airbrake logging no

    To control via pm2

    pm2 [start|restart|stop|delete] ecosystem.config.js

    The ecosystem.config.js file contains the set of receiver services to run. An example entry is shown:

       - script: ./index.js
         name: 'Sales Queue'
         env:
           RECEIVER_PREFIX: SALES_QUEUE
         output: ./logs/sales-queue.out
         error: ./logs/sales-queue.err
         autorestart: false
    

    Debug

    The service uses the debug package see the DEBUG environment variable guidance

    Failures

    The service will fail early where a general error reading an SQS queue is encountered such as ECONNREFUSED

    In the case of all http errors reading SQS queues the service will log the error continue. See: https://docs.aws.amazon.com/AWSSimpleQueueService/latest/APIReference/CommonErrors.html

    In the case of all errors encountered in the subscriber process queues the service will log the error continue

    Only in the cases where the subscriber return an http success status will the message be removed from the SQS queue

    A development environment can be set up as follows

    AWS_REGION=eu-west-2
    AWS_ACCESS_KEY_ID=local
    AWS_SECRET_ACCESS_KEY=local
    AWS_SQS_ENDPOINT=http://0.0.0.0:9324
    
    DEBUG=sqs:*
    
    TRANSACTION_QUEUE_URL=http://0.0.0.0:9324/queue/TransactionsQueue.fifo
    TRANSACTION_QUEUE_MAX_POLLING_INTERVAL_MS=30000
    TRANSACTION_QUEUE_VISIBILITY_TIMEOUT_MS=300000
    TRANSACTION_QUEUE_SUBSCRIBER=http://0.0.0.0:4000/process-queue/transactions
    
    TRANSACTION_DLQ_URL=http://0.0.0.0:9324/queue/TransactionsDlq.fifo
    TRANSACTION_DLQ_MAX_POLLING_INTERVAL_MS=180000
    TRANSACTION_DLQ_VISIBILITY_TIMEOUT_MS=300000
    TRANSACTION_DLQ_SUBSCRIBER=http://0.0.0.0:4000/process-dlq/transactions
    

    Install

    npm i @defra-fish/sqs-receiver-service

    DownloadsWeekly Downloads

    328

    Version

    1.21.0

    License

    SEE LICENSE IN LICENSE

    Unpacked Size

    39.3 kB

    Total Files

    19

    Last publish

    Collaborators

    • npm-envage
    • defradigitaladmin
    • defradigitalci
    • jaucourt