Non-Production Machines

    honeybee-hive

    1.3.0 • Public • Published

    Honeybee-Hive

    Master CircleCI Dev CircleCI

    NPM Version NPM Download

    Gitter

    A node module to allow for volunteer computing, like BOINC.

    Install

    npm install honeybee-hive --save

    Notes

    • Alpha stages, expect breaking changes between versions currently
    • If you want to have clients from the browser, take a look at Honeybee-Web

    Example setups

    Server

    • Inline: Work is processed and created inside of your node application
    • Subprocess: Work is created and processed inside of another application (can be in another language), and information is passed between the node app and your child process
    • Database (not recommended): Work is created and processed by another application, but it is added to a database. The node app finds work in the database and removes it, giving it to the client

    Client

    • Inline: Work is processed inside of the application, and then handed back to the server
    • Subprocess: Work is transferred to and from the server by the node application, but processed by a subprocess which is a different application
    • Database (not recommended): Work is added to a database, and another client periodically checks the database for new work, and then adds processed work back to the database

    Usage

    See examples for working examples, which might be easier to understand

    Server

    Start function

    The server gets called like this:

    let HoneybeeHive = require('honeybee-hive');
    let eventHandler = HoneybeeHive.Hive(settings);

    The settings object is described below

    Settings

    let settings = {
      connection: {
        port: 54321 //Listening port, defaults to 54321
      },
      //All in milliseconds
      timeouts: {
        workTimeout: 60000, //Time to wait until a client is assumed to be dead and not completing the work set, allowing someone else to do so. Set to a value less than 1 to disable. Default: 60000
        sessionTimeout: 30000, //Time to wait until a TCP socket which is idle is assumed to be dead and therefore destroyed. Default: 30000
        checkTimeout: 10000, //How often to check for work timeouts. Default: 10000
      },
      work: {
        groupMax: 10 //How many datasets must be submitted before the workgroup is considered completed. Default: 10
      },
      proofOfWork: {
        strength: 4 //How difficult should the proof of work problem be? Higher values help to prevent spam, but take longer to calculate on average. Set to -1 to be (essentially) off. Default: 4
      },
      encryption: {
        key: "some private key" //NO DEFAULT. YOU MUST SET THIS. The PEM encoded RSA private key for the server
      },
      sections: {
        disableRegistration: false //Disables registration if this value is true. Default: false
      },
      database: {
        hostname: 'localhost', //Sets the hostname for the mongodb database. Default: localhost
        port: '27017', //Optional. Sets the port for the mongodb databse. Default: none
        databaseName: 'hive' //The name for the mongodb database. You could set this to your project name. Default: hive
      },
      websocket: {
        enabled: false, //Sets if we should also run a WebSocket server for WebSocket clients using Honeybee-Web. Default: false
        port: 54322 //What  port we should run the WebSocket server on. Default: 54322
      }
    }

    Event Emitter

    The main function returns an event emitter which we can then listen on, like this:

    eventHandler.on('eventName', function(eventArgs, eventArgs2) {
      //Some code here to process event arguments
    });

    The events are detailed below

    Create work

    We can listen for requests to create work like this:

    eventHandler.on('create_work', function(callback) {
      //We can send the work to the callback like this:
      callback({
        work: 0
      });
      //Or if there's no work remaining, we can send the callback a null value in JavaScript, like this
      callback(null);
    });
    Workgroup complete

    When a set of work is complete, we must verify it. We can do so like this:

    eventHandler.on('workgroup_complete', function(array, callback) {
      //Make sure that all the values of the array are equal
      for(let i = 0; i < array.length - 1; i++) {
        //If they aren't equal
        if(array[i] !== array[i+1]) {
          //Then return null to the callback
          callback(null);
        }
      }
      //Otherwise, we can return the first element, since we just want to make sure that there's a consensus
      //You could also return an average, or work backwards on the solution
      callback(array[0]);
    });
    Datachunk creation

    When a workgroup is validated, we can then bring it together with other validated workgroups, or datachunks, like this:

    let total = 0;
    eventHandler.on('new_datachunk', function(datachunk) {
      //datachunk is the data that we submitted to the callback for workgroup_complete
      //We access the count property and add it to the total, and then log it to the console
      total += datachunk.count;
      console.log(total);
    });

    Remember that if order matters, then you'll need to submit an order with the work, and form a queue type system

    Stopping the server

    To stop the server you can do this:

    eventHandler.stop(function() {
      //Called once all existing connections have finished and the server is actually closed.
    });
    Notes
    • Progress is not saved. It's advisable that when you create work that you somehow store the work that has been created and what hasn't. Once work is created, it will be distributed to clients, but otherwise there is no way for you to know what work needs to be created still.

    Client

    Start function

    The client gets called like this:

    let HoneybeeHive = require('honeybee-hive');
    HoneybeeHive.Honeybee(settings, function(eventHandler) {
      //Wait for us to be registered
      eventHandler.once('registered', function() {
        //Request our first piece of work
        eventHandler.request(workHandler);
      })
    });

    The settings object is described below

    Settings

    let settings = {
      connection: {
        hostname: 'localhost', //The hostname the server is listening on, defaults to localhost
        port: 54321 //Listening port, defaults to 54321
      },
      encryption: {
        key: "some public key" //NO DEFAULT. YOU MUST SET THIS. The PEM encoded RSA public key for the server
      }
    }

    Event Handler

    The main function returns an event handler which we can then call, like this:

    eventHandler.functionName(callback);

    The callbacks and events are detailed below

    Ready

    We know that the client is ready to process work when the registered event is fired. We can listen to this by doing:

    eventHandler.once('registered', function() {
      //Request work here
    });
    Request work

    We can request work like this:

    eventHandler.request(function(error, work) {
      //If error is true, then we can process it as needed
      //work is the work that we specified on the server
      //we should process it and then send it for submission
    });
    Submit work

    We can submit processed work like this:

    eventHandler.submit(work, function(error, success) {
      //If error is true then we can process it as needed
      //success tells us if the submission was successful. You should not retry on failure, rather just request new work
    });
    • work is the processed work that we wish to submit to the server
    Combining the two

    We can combine requesting and submitting like this:

    function workHandler(error, work) {
      eventHandler.submit(work, submitHandler);
    }
    function submitHandler(error, success) {
      eventHandler.request(workHandler);
    }
    //Request first work
    eventHandler.request(workHandler);

    Error handling

    As you can see in the above examples, there are areas where an error object is returned.

    You can do something similar to a try/catch block in Java with if statements, importing the error objects. E.g.:

    const HoneybeeHive = require('honeybee-hive');
    const errorList = HoneybeeHive.errorList;
    const errorGroups = HoneybeeHive.errorGroups;
    function workHandler(error, work) {
      if(error) {
        //Start try/catch-esque if/else if/else block
        //Catch all security errors
        if(error instanceof errorGroups.SecurityError) {
          //Do something here
        }
        //Catch only post handshake errors
        else if(error instanceof errorList.HandshakePostCompleteFailure) {
          //Do something
        }
        //Catch all *other* handshake errors
        else if(error instanceof errorGroups.HandshakeError) {
          //Do something
        }
        //Catch either submit or request errors
        else if(error instanceof errorGroups.SubmitError || error instanceof errorGroups.RequestError) {
          //Do something
        }
        //Catch all other errors, including generic/unknown errors
        else {
          //Do something
        }
      }
    }

    This is the complete list of errors, their description, and their error group:

    Error Name Error Group Error Description
    DatabaseGeneric DatabaseError Generic database errors which do not have a specific error associated
    DatabaseNotFound DatabaseError When a record was not found in the database (e.g. when the worker was not found as being a worker for the submitted data
    GenericPayloadMissing GenericError When the encrypted payload is missing
    GenericParametersMissing GenericError When parameters are missing (e.g. the type of message)
    GenericSecurityInformationMissing GenericError When security information is missing (e.g. the IV used to encrypt the payload)
    HandshakeGeneric HandshakeError When a generic handshake error occurs
    HandshakeKeyMissing HandshakeError When the client doesn't provide an AES key
    HandshakePostCompleteFailure HandshakeError When the handshake fails elsewhere (e.g. someone edits the packets and fails to match the authentication tag)
    VerificationGeneric VerificationError When a generic verification error occurs
    VerificationNotExecuted VerificationError When a user tries to execute an operation which requires verification, but they have not verified themselves yet
    RequestNoWork RequestError When a user tries to request work, but there is none remaining for the server to give out
    RequestPendingWork RequestError When a user tries to request work, but has work which is unsubmitted
    SubmitNoData SubmitError When a user sends no completed work with the submission
    SecurityInvalidKey SecurityError When an invalid key format is sent
    SecurityKeyGenerationFailure SecurityError When an RSA keypair fails to generate
    SecurityEncryptionFailure SecurityError When there's an error encrypting
    SecurityDecryptionFailure SecurityError When there's an error decrypting
    SecuritySigningFailure SecurityError When there's an error signing the message
    SecurityVerificationFailure SecurityError When there's an error when trying to verify a message
    Notes
    • Progress is not saved. It's advisable that when you receive work that you save it for processing later, and also record the time of the request in order to avoid DATABASE_NOT_FOUND errors on the client

    License

    ISC

    Install

    npm i honeybee-hive

    DownloadsWeekly Downloads

    8

    Version

    1.3.0

    License

    ISC

    Last publish

    Collaborators

    • flarp
    • kurimizumi