Nondeterministic Programming Methodology

    eisenhertz

    1.6.0 • Public • Published

    Eisenhertz

    Build Status

    nodejs module-as-a-process cluster management

    What does it do?

    • Eisenhertz excels at one thing only: keeping a set of dynamic module execution up & running across endless Servers, VMs, Containers
    • You pass in a module to execute and an unlimited amount of config-data (jobs) for each execution and eisenhertz will ensure that the correct amount of modules is constantly running across the all instances of itself, where each module runs in its own process
    • It also gives you controll to manually add or remove such jobs in real-time
    • Additionally you can talk to the processes via ipc and retrieve metrics from all processes running
    • Eisenhertz does not work as a stand-alone "server-setup", its main idea is to build a basis for a project that requires scaling across a lot of machines in stand-alone processes

    Requirements

    • Eisenhertz does heavily rely on async/await therefore you will need at least node >=v 7.0
    • The message cortex and job queue relies on Redis >= 2.8.18

    Install via

    npm i eisenhertz
    

    Server Setup

    const {
        Eisenhertz,
        defaultConfig,
        defaultLogger
    } = require("eisenhertz");
     
    const fetchJobNames = callback => {
        callback(null, [
            "one",
            "two"
        ]);
    };
     
    const fetchJobDetails = (id, callback) => {
     
        let config = {};
     
        switch (id) {
     
            case "one":
                config.port = 1337;
                config.hi = "hi from one";
                break;
     
            case "two":
                config.port = 1338;
                config.hi = "hi from two";
                break;
        }
     
        callback(null, {
            config
        });
    };
     
    const eisenhertz = new Eisenhertz(config, defaultLogger());
    eisenhertz
        .start(fetchJobNames, fetchJobDetails)
        .then(() => {});

    Fork-Module Setup

    const { ForkProcess } = require("eisenhertz");
    const express = require("express");
     
    const fork = new ForkProcess();
    let incomingRequests = 0;
     
    const processCallback = data => {
     
        const app = express();
     
        app.get("/hi", (req, res) => {
            incomingRequests++;
            res.status(200).json({
                message: data.config.hi
            });
        });
     
        app.listen(data.config.port, () => {
            fork.log("ready");
        });
    };
     
    const metricsCallback = cb => {
        cb(null, {
            incomingRequests
        });
    };
     
    fork.connect(processCallback, metricsCallback);

    Example Setup Description

    • The example setup above will give you the possiblity to scale a demo webserver across unlimited instances, by simply deploying the server module to servers, vms or containers.
    • As soon as it starts, it will spawn 2 processes on any of the parent systems that will run one of the two webservers.

    Configuration

    {
        prefix: "eh",
        redis: {
            host: "localhost",
            port: 6379,
            db: 7
        },
        redlock: {
            driftFactor: 0.01,
            retryCount: 2,
            retryDelay: 200,
            retryJitter: 200
        },
        settings: {
            lockDuration: 4500,
            stalledInterval: 4500,
            maxStalledCount: 1,
            guardInterval: 2500,
            retryProcessDelay: 2500
        },
        properties: {
            name: "eh:empty",
            maxJobsPerWorker: 2,
            masterLock: "eh:master:lock",
            masterLockTtl: 2000,
            masterLockReAttempt: 4000,
            maxInstancesOfJobPerNode: 1
        },
        jobOptions: {
            priority: 1, 
            delay: 1000, 
            attempts: 1, //dont touch
            repeat: undefined, //dont touch
            backoff: undefined, //dont touch
            lifo: undefined, //dont touch
            timeout: undefined, //dont touch
            jobId: undefined, // will be set by TaskHandler
            removeOnComplete: true, //dont touch
            removeOnFail: true //dont touch
        },
        fork: {
            module: "./fork/ForkProcess.js"
        }
    }

    Controlling jobs on nodes

    config.properties.maxInstancesOfJobPerNode
    /*
        lets you limit the amount of instances of a job
        that run on a single node, you can define a job instance
        by using ":" as delimiter e.g. jobOne:1, jobOne:2 and jobOne:3
        if the limit is reached, the node will return the job with
        an error back to the queue after a small timeout
    */
     
    config.properties.maxJobsPerWorker
    /*
        lets you limit the amount of jobs per worker
        it is usually a good idea to limit this to the amount
        of cores (* 2 on intel systems) of the node's host
    */

    Install

    npm i eisenhertz

    DownloadsWeekly Downloads

    2

    Version

    1.6.0

    License

    MIT

    Last publish

    Collaborators

    • krystianity