Share your code. npm Orgs help your team discover, share, and reuse code. Create a free org »

This package has been deprecated

Author message:

is now hawkly-grpc

libharmonia

0.0.3 • Public • Published

libharmonia

An opinionated microservices framework for node.js and docker

Part of harmonia.systems, libharmonia is the foundation of this opinionated system. It's role is to provide a common inter-service messaging platform, deeply integrated with logging and tracing.

The harmonia.systems framework is one that explicity relys on pattern matching messages sent between services over a message queue.

Usage

libharmonia is written in Typescript, and can also be used with ES6/ES7 code. You'll have the most fun with async/await, with either Typescript or Babel. Keep an eye on the NodeJS production releases for when a stable async/await implementation hits NodeJS core.

In harmonia there are two roles that a system can perform. Sender and Listener. By default, all services can send out messsages as a Sender, and with extra configuration they can also listen for messages sent to them as a Listener.

Let's create both in both Typescript and ES7 code.

Sender

Our sender is going to send a math problem to a service called math that can return the solution.

import {
    Harmonia,
    MessageResponse,
} from 'libharmonia';
 
// Initialise Harmonia
 
const harmonia: Harmonia = new Harmonia({
    amqp: {
        host: 'mq',
        port: 5672,
        username: 'admin',
        password: 'secret',
        virtualHost: 'harmonia',
    },
    role: 'math-tester',
});
 
 
// Lets create some numbers for this problem
const x: number = 12;
const y: number = 83;
const z: number = 32;
 
// We wrap our harmonia.message calls in harmonia.request
// This is essential to take advantage of the built in request tracing
// harmonia.request is an async function that takes in a callback,
// and passes a fresh requestId as the only parameter.
// Then, all requests made inside our callback function, will
// be linked together in our logging, allowing you to visualise
// the flow of requests made throughout the system.
const mathSolution: number = await harmonia.request(
  // harmonia.request takes a callback function, which passes a fresh requestId
  // as it's only parameter
  async (requestId: string) => {
    // harmonia.message is another async function that takes our requestId as
    // it's first parameter and then an object that comprises the message as
    // it's second parameter.
    // Our message body needs to have a `service` (where the message is going),
    // an `action` (what action/method to perform on our payload)
    // and `payload` (an object).
    const sumResult: MessageResponse = await harmonia.message(
        requestId,
        {
            service: 'math',
            action: 'sum',
            payload: {
                x,
                y,
            },
        });
 
    // Here we're making another request.
    const productResult: MessageResponse = await harmonia.message(
        requestId,
        {
            service: 'math',
            action: 'product',
            payload: {
                x: sumResult.payload,
                y: z,
            },
        });
    // Finally we're returning the payload of productResult
    // this will become the value of mathSolution
    return productResult.payload;
  }
);
 
// Log our final result
console.log(`(${x} + ${y}) * ${z} = ${mathSolution}`);
 
// This should log:
// (12 + 83) * 32 = 3,040
 

API

Error handling

All of the public functions with harmonia except the constructor new Harmonia() are async functions (Promises). All the public functions will throw an error if something went wrong, thus they must all be wrapped in a try ... catch statement, otherwise your errors may be swallowed.

new Harmonia(options)

The harmonia constructor lets you configure how this instance of harmonia is going to connect to your infrastructure.

Harmonia can connect to either an AMQP message queue like RabbitMQ or a lightweight high performance messaging bus like NATS.

It's also possible to supply your own or (third-party) sender and listener implementations, to connect with a different messaging system. The sender and listener will both receive the same messengerConfig, and use the same messaging system. You can't send messages with AMQP, and listen for them with NATS. At least, not within the same instance, you could setup two harmonia instances with different messaging systems if you needed a bridge.

Harmonia by default will log to Graylog2 with GELF, but again, you can pass through your own logging implementation if you need to use something else. Graylog2 was chosen as it's a great open source logging project, that works out of the both for debugging messages, but can also forward messages to other systems.

Usage
import {
    Harmonia,
} from 'libharmonia';
 
// Always try..catch as errors are thrown not returned
try {
  // Initialise Harmonia
  const harmonia: Harmonia = new Harmonia({
 
      // Define your services name
      // This is a global namespace, only one repository should use it.
      // It's expected that you will run multiple containers with this repo,
      // but that repos will all have a unique service name.
      // Ulitmately this is how meesages are routed.
      service: 'math',
 
      // Provide connection details to the messenger
      messengerConfig: {
        type: 'ampq', // amqp or nats
        host: 'mq',
        port: 5672,
        username: 'admin',
        password: 'secret',
        virtualHost: 'harmonia',
      },
      // (optional) you can provide a custom or third party sender and listener if
      // you need to use something besides amqp or nats
      // Look at  src/modules/messenger/sender.ts and src/modules/messenger/listener.ts for
      // implementation details
      sender: () => {},
      listener: () => {},
 
      // Provide the connection information for the logging service
      loggerConfig: {
        type: 'graylog2', // default implementation is graylog2
        servers: [
            {
              host: 'harmonia-graylog',
              port: 1984
            },
        ],
        hostname: 'dev', // the name of this host
        // (optional, default: os.hostname())
        facility: 'Node.js',     // the facility for these log messages
        // (optional, default: "Node.js")
        bufferSize: 1350,         // max UDP packet size, should never exceed the
        // MTU of your system (optional, default: 1400)
      }
      // (optional) you can pass a custom logger which will be passed loggerConfig
      // Look at src/modules/logger.ts for implementation details
      logger: () => {}
 
      // (optional) Provide the connection details for the database connection
      databaseConfig{
        type: 'rethinkdb',
      }
      // (optional) provide your own or a third party database implementation
      database: () => {}
  });
} catch (harmoniaError){
  // If we could not start for any reason we should crash.
  console.log('Harmonia failed to start', harmoniaError)
  process.exit(1);
}

harmonia.request(callback)

Used to wrap harmonia.message calls. This function ensures a fresh requestId can be passed to harmonia.message, it also clears out the internal state that tracks the previous message -- allowing for detailed tracing of requests and messages.

Usage
// Always try..catch as errors are thrown not returned
try {
  const request: MessageResponse = await harmonia.request(
    // harmonia.request takes a callback function,
    // which passes a fresh requestId as it's only parameter
    async (requestId: string) => { // note it's an async function
      // call harmonia.message Here
    }
  );
} catch(requestError){
  // handle the requestError
}

harmonia.message(requestId, message)

This is an async function (a Promise) that will return the response from the remote service.

Usage
// Always try..catch as errors are thrown not returned
try {
  const messageResult: MessageResponse = await harmonia.message(
      requestId,
      {
          // The service you are sending the message to
          service: 'math',
          // The action you need the service to perform on your payload
          action: 'sum',
          // You payload, in the format required by the action on the chosen service
          payload: {
              x: 12,
              y: 32,
          },
      });
 
  // Use the result
  console.log(messageResult.payload);
 
} catch(messageError){
  // handle messageError
}

The response from harmonia.message will always match MessageResponse which you can see below.

MessageResponse
interface MessageResponse {
    // The response
    payload: any;
 
    // Mainly internal metadata, but you may want some of it
    metadata: MessageMetadata;
 
    // if the data payload contains sensitive information that should not be logged
    // we indicate it here, and it should never be stored, it MUST be zeroed before logging
    sensitiveKeys?: string[];
}

harmonia.addListener(pattern, callback(message, error))

To register a listener you add the action and callback function with a call to harmonia.addListener. The service is set in the constructor of the harmonia instance, and can't be changed here.

First create an async function which takes the message and error as arguments.

Then call harmonia.addListener, specify the pattern to match on, and then the callback function.

Usage

async function sum(message: number, error: any) {
    return message.payload.x + message.payload.y;
}
 
harmonia.addListener({ action: 'sum' }, sum);
 

You can also pass a fat-arrow function callbacks as well. Though these will be much harder to unit test, and therfore are not reccomended.

harmonia.addListener(
  { action: 'sum' },
  async (message: number, error: any) => {
    return message.payload.x + message.payload.y;
  }
);
 

We use patrun for pattern matching. Generally speaking the most specific pattern will always win.

Error handling in listener callbacks

The function that calls your callback will try ... catch your function for any exceptions and pass them back as an error. You don't explicity need to catch errors inside your callback, if you don't need to handle it.

Install

Generally you'll find it easier to start from one of our starterkits, as Harmonia is designed to be a framework to build upon.

TODO build and add a starter kit.

Standalone

With npm installed, run

$ npm install libharmonia

Or with yarn

$ yarn add libharmonia

Acknowledgments

seneca.js is a similar an far less opinionated module, that was the primary inspiration for libharmonia

patrun for an excelent pattern matching library.

See Also

License

ISC License (ISC)

Copyright (c) 2017, Owen Kelly owen@owenkelly.com.au

Permission to use, copy, modify, and/or distribute this software for any purpose with or without fee is hereby granted, provided that the above copyright notice and this permission notice appear in all copies.

THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.

Keywords

none

install

npm i libharmonia

Downloadsweekly downloads

10

version

0.0.3

license

ISC

homepage

gitlab.com

repository

Gitgitlab

last publish

collaborators

  • avatar