@data-processing.solutions/grit-events-mongoose

Grit Events Mongoose

Create, Fetch, Acquire and Handle - events being saved as documents in MongoDB.

What is Grit Events Mongoose

Grit Events Mongoose is a class to instantiate event producers and consumers. It can be used to develop all kinds of event-based systems, for example an engine to execute BPMN processes. When initialized with event handlers it functions as an event-consumer, otherwise as an event-producer. In both cases a connection to a MongoDB database is opened.

How does Grit Events Mongoose work under the hood

When created with handlers - once an instance of the class is listening - it creates a backoff function which checks for new events to handle (it is looking for events which were not yet acquired). Whenever a new (not acquired) event is found, the machine tries to acquire it. It could happen that another machine instance is also trying to acquire it but only one will win. Whoever acquires the event will handle it and when handled set it to status: handled. A producer can be used to create new events.

How to install

$ npm install @data-processing.solutions/grit-events-mongoose

or

$ yarn add @data-processing.solutions/grit-events-mongoose

Examples

Require

const GritEvents = require('@data-processing.solutions/grit-events-mongoose');

Producer

const gritEvents = new GritEvents({
  dbUrl: 'mongodb://localhost:27017',
  DEFAULT_PRIORITY: 50, // default: 50
  DEFAULT_MAX_RETRIES: 0 // default: 0
});

gritEvents.listen().catch(err => console.error(err));

for (let i = 0; i <= 1; i++) {
  gritEvents.createNewEvent({
    intent: 'CREATE_SOMETHING',
    time: new Date().getTime() - 1 // optional/default
  });
}

When producing events, note that the intent is the name of the handler method you provide your consumer with. time is the desired time of executing this event, the (current timestamp - 1) should make it being picked up right away (if there are no other events, which are waiting for longer, or have higher priority). You can control how often a failed event gets re-scheduled for retry with maxRetries, by default it is set to 0 so failed events won't get retried.

Consumer

const gritEvents = new GritEvents(
  {
    dbUrl: 'mongodb://localhost:27017',
    DEFAULT_PRIORITY: 50,
    batchSize: 1, // Optional (default: 1), set how many events you want to fetch, acquire and handle at once
    intentScope: ['CREATE_SOMETHING'] // Optional, this worker/consumer only fetches, acquires and handles events with the intent CREATE_SOMETHING, [] means nothing will be handled, not defining it means all handlers will be handled
  },
  {
    CREATE_SOMETHING: event => {
      console.log('create something', event._id);
    }
  }
);

gritEvents.listen().catch(err => console.error(err));

When consuming events, note that you can define batchSize to acquire and handle multiple events at once, the default is 1. You can also define an intentScope so you might have an instance of the Event Machine only handling certain events, even though it is capable of more, handler-wise. This could be used to scale (TODO: elaborate).

Directly access the database schema

Whenever you have the need to read an existing event directly, you can invoke mongoose methods on the Event Model, like described following:

const Event = require('@grit/grit-events-mongoose/lib/schemas/Event');

const specificEvent = await Event.findById('...');

EventEmitter

The GritEventMongoose class extends the native Node.JS EventEmitter (require('events')), there are some before and after events emitted for the process to listen to. Those are side-effects and it is not intended nor possible (?) to interrupt fetching, acquisition or handling of events like this. Such a desire is only possible through the handlers themselves, they are promises and if they fail the event won't get marked as handled.

// before acquiring an event
gritEvents.on('before.acquire.CREATE_SOMETHING', event => {});

// after successfully acquiring an event
gritEvents.on('after.acquire.CREATE_SOMETHING', event => {});

// before handling an event
gritEvents.on('before.handle.CREATE_SOMETHING', event => {});

// after successfully handling an event
gritEvents.on('after.handle.CREATE_SOMETHING', event => {});

// the event got scheduled for a retry
gritEvents.on('retry.handle.CREATE_SOMETHING', event => {});

// when an event never (after retries) was able to be handled
gritEvents.on('failed.handle.CREATE_SOMETHING', event => {});

Retries

When creating events via createNewEvent you can define maxRetries (default: 0), so when handling the event fails, it gets retried n times. By default failed events won't get retried. Note: the handler gets executed 1 + maxRetries times in total.