Narcissistic Pickle Meister

    kafkajs-dlq

    1.0.0 • Public • Published

    KafkaJS Dead Letter Queue

    KafkaJS plugin to handle message processing failures by forwarding problematic messages to a dead-letter queue.

    Dead Letter Queues are message queues that can be produced to when a message on another queue cannot be processed successfully. Dead-letter queues are useful for debugging your application or messaging system because they let you isolate problematic messages to determine why their processing doesn't succeed.

    WIP: This project is not ready for use as of yet

    Usage

    const { Kafka } = require('kafkajs')
    const Dlq = require('kafkajs-dlq')
     
    const kafka = new Kafka({ ... })
    const consumer = kafka.consumer({ ... })
    const producer = kafka.producer()
     
    const topic = 'example-topic'
     
    const { eachMessage } = Dlq.consumer({
      topics: {
        [topic]: 'example-dead-letter-queue'
      },
      producer,
      eachMessage: async ({ topic, partition, message }) => {
        // If eachMessage rejects, the message will be
        // produced on the dead-letter queue
        throw new Error('Failed to process message')
      }
    })
     
    const run = async () => {
      await consumer.connect()
      await consumer.subscribe({ topic })
      await consumer.run({ eachMessage })
    }
     
    run()

    Keywords

    none

    Install

    npm i kafkajs-dlq

    DownloadsWeekly Downloads

    22

    Version

    1.0.0

    License

    MIT

    Unpacked Size

    126 kB

    Total Files

    8

    Last publish

    Collaborators

    • nevon