worker-nodes
    TypeScript icon, indicating that this package has built-in type declarations

    2.3.1 • Public • Published

    view on npm Build Status

    worker-nodes

    A node.js library to run cpu-intensive tasks in a separate processes and to not to block the event loop.

    Installation

    $ npm install worker-nodes

    Node.js greater than 11.7.0 is required

    API Reference

    WorkerNodes

    Kind: global class

    new WorkerNodes(path, [options])

    Param Type Description
    path String An absolute path to the module that will be run in the workers.
    [options] Object See WorkerNodesOptions for a detailed description.

    workerNodes.call : Proxy

    This exposes the api of a module that the worker nodes are working on. If the module is a function, you can call this directly. If the module exports multiple functions, you can call them as they were properties of this proxy.

    Kind: instance property of WorkerNodes

    workerNodes.ready() ⇒ Promise

    A method to check if the minimum required number of workers are ready to serve the calls.

    Kind: instance method of WorkerNodes
    Returns: Promise - resolves with a WorkerNodes instance

    workerNodes.terminate() ⇒ Promise

    Starts the process of terminating this instance.

    Kind: instance method of WorkerNodes
    Returns: Promise - - resolved when the instance is terminated.

    workerNodes.profiler(duration) ⇒ void

    Run CPU Profiler and save result on main process directory

    Kind: instance method of WorkerNodes

    Param Type
    duration number

    workerNodes.takeSnapshot() ⇒ void

    Take Heap Snapshot and save result on main process directory

    Kind: instance method of WorkerNodes

    workerNodes.getUsedWorkers() ⇒ Array.<Worker>

    Return list with used workers in pool

    Kind: instance method of WorkerNodes

    WorkerNodesOptions

    Describes a WorkerNodes options.

    Kind: global class

    options.autoStart : Boolean

    Whether should initialize the workers before a first call.

    If true, depending on the lazyStart option, it will start the min or max number of workers.

    Kind: instance property of WorkerNodesOptions
    Default: false

    options.lazyStart : Boolean

    Whether should start a new worker only if all the others are busy.

    Kind: instance property of WorkerNodesOptions
    Default: false

    options.asyncWorkerInitialization : Boolean

    Enables async initialization of worker. To start handling task over worker, need to invoke sendWorkerMessage('ready') function when it fully initialized. For examples please refer to the test cases

    Kind: instance property of WorkerNodesOptions
    Default: false

    options.minWorkers : Number

    The minimum number of workers that needs to be running to consider the whole pool as operational.

    Kind: instance property of WorkerNodesOptions
    Default: 0

    options.maxWorkers : Number

    The maximum number of workers that can be running at the same time. Defaults to the number of cores the operating system sees.

    Kind: instance property of WorkerNodesOptions

    options.maxTasks : Number

    The maximum number of calls that can be handled at the same time. Exceeding this limit causes MaxConcurrentCallsError to be thrown.

    Kind: instance property of WorkerNodesOptions
    Default: Infinity

    options.maxTasksPerWorker : Number

    The number of calls that can be given to a single worker at the same time.

    Kind: instance property of WorkerNodesOptions
    Default: 1

    options.taskTimeout : Number

    The number milliseconds after which a call is considered to be lost. Exceeding this limit causes TimeoutError to be thrown and a worker that performed that task to be killed.

    Kind: instance property of WorkerNodesOptions
    Default: Infinity

    options.taskMaxRetries : Number

    The maximum number of retries that will be performed over a task before reporting it as incorrectly terminated. Exceeding this limit causes ProcessTerminatedError to be thrown.

    Kind: instance property of WorkerNodesOptions
    Default: 0

    options.workerEndurance : Number

    The maximum number of calls that a single worker can handle during its whole lifespan. Exceeding this limit causes the termination of the worker.

    Kind: instance property of WorkerNodesOptions
    Default: Infinity

    options.workerStopTimeout : Number

    The timeout value (in milliseconds) for the worker to stop before sending SIGKILL.

    Kind: instance property of WorkerNodesOptions
    Default: 100

    Example

    Given /home/joe.doe/workspace/my-module.js:

    module.exports = function myTask() {
        return 'hello from separate process!';
    };

    you can run it through the worker nodes as follows:

    const WorkerNodes = require('worker-nodes');
    const myModuleWorkerNodes = new WorkerNodes('/home/joe.doe/workspace/my-module');
    
    myModuleWorkerNodes.call().then(msg => console.log(msg));  // -> 'hello from separate process!'

    For more advanced examples please refer to the test cases.

    Running tests

    Check out the library code and then:

    $ npm install
    $ npm test

    Benchmarks

    To run tests, type:

    $ npm install
    $ npm run benchmark

    It will run a performance test against the selected libraries:

    • data in: an object that consists of a single field that is a 0.5MB random string
    • data out: received object stringified and concatenated with another 1MB string

    Example results:

    results for 100 executions
    
    name                time: total [ms]  time usr [ms]  time sys [ms]  worker usr [ms]  worker sys [ms]  mem rss [MB]  worker rss [MB]  errors
    ------------------  ----------------  -------------  -------------  ---------------  ---------------  ------------  ---------------  ------
    no-workers                       148            203             37                0                0            98                0       0
    worker-nodes@2.0.0               362            390            143              389              143           213              210       0
    workerpool@6.0.0                 367            495            185              492              182           236              245       0
    worker-nodes@1.6.1              1095            520            207              592              243           216               86       0
    worker-farm@1.7.0               1886            749            276              947              299           221               70       0
    process-pool@0.3.5              2002            847            285              986              309           219               74       0
    worker-pool@3.0.2              13775           7129           5236             1891              952           363               63       0
    
      os : Darwin / 19.5.0 / x64
     cpu : Intel(R) Core(TM) i7-7660U CPU @ 2.50GHz × 4
    node : 14.3.0 / v8: 8.1.307.31-node.33

    See also

    sources of inspiration:

    License

    Copyright Allegro Sp. z o.o.

    Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0
    

    Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

    Install

    npm i worker-nodes

    DownloadsWeekly Downloads

    2,168

    Version

    2.3.1

    License

    Apache-2.0

    Unpacked Size

    58.9 kB

    Total Files

    24

    Last publish

    Collaborators

    • allegrotech
    • bgalek