A cross-platform thread pool add-on for Node.js and io.js
nPool's primary features and benefits include:
nPool is written entirely in C/C++. The thread pool and synchronization frameworks are written in C and the add-on interface is written in C++. The library has no third-party dependencies other than Node.js, V8, and nan.
The cross-platform threading component utilizes
pthreads for Mac and Linux. On Windows, native threads (
CRITICAL_SECTIONS are used. Task based units of work are performed via a FIFO queue that is processed by the thread pool. Each thread within the thread pool utilizes a distinct
uv_async inter-thread communication mechanism.
One thing to note,
unordered_maps are used within the add-on interface, therefore, it is necessary that the platform of choice provides C++11 (Windows and Linux) or TR1 (Apple) implementations of the standard library.
nPool can be easily compiled on Mac, Linux and Windows using
Simply run the following command within a console or terminal
node-gyp clean configure build.
This will automatically configure the environment and produce the add-on module.
Currently during development the following compilers are tested:
A reference implementation is provided with the source in the
nPool provides a very simple and efficient interface. Currently, there are a total of five functions:
// load nPool modulevar nPool = require'npool';// work complete callback from thread poolvar// load files defining object typesnPoolloadFile1 __dirname + '/objectType.js';// create thread pool with two threadsnPoolcreateThreadPool2;// create the unit of work objectvar unitOfWork =workId: 9124fileKey: 1workFunction: "objectMethod"workParam:aProperty: 134532 "xysaf"callbackFunction: callbackFunctioncallbackContext: this// queue the unit of worknPoolqueueWorkunitOfWork;
This function creates the thread pool. At this time, the module only supports one thread pool per Node.js process. Therefore, this function should only be called once, prior to
The function takes one parameter:
numThreadsuint32 - number of threads to create within the thread pool
// create thread pool with two threadsnPoolcreateThreadPool2;
This function destroys the thread pool. This function should only be called once and only when there will be no subsequent calls to the
queueWork function. This method can be called safely even if there are tasks still in progress. At a lower level, this actually signals all threads to exit, but causes the main thread to block until all threads finish their currently executing in-progress units of work. This does block the main Node.js thread, so this should only be executed when the process is terminating.
This function takes no parameters.
// destroy the thread poolnPooldestroyThreadPool;
This function can be called at any time. It should be noted that this is a synchronous call, so the serialization of the file will occur on the main thread of the Node.js process. That being said, it would be prudent to load all necessary files at process startup, especially since they will be cached in memory.
Each thread, on first execution with a unit of work which requires the file referenced by
fileKey, will de-serialize and compile the contents into a V8 function. The function is used to instantiate a new persistent V8 object instance of the object type. The persistent object instance is then cached per thread. Every subsequent unit of work referencing the
fileKey will retrieve the already cached object instance.
This function takes two parameters:
fileKeyuint32 - uniquely identifies a file
Each file must have a unique key.
__dirname in addition to the relative path to the file.
// load files defining object typesnPoolloadFile1 __dirname + '/objectType.js';
Files that are loaded should define an object type (function) that can be instantiated. Keep in mind this object is used as a service and should be stateless.
An example file is given below:
// ./applesOranges.js// utilize a node.js like requirevar _ = require'./underscore.js';var// function that matches the unit of work defined work function// total number of fruitsvar fruitCount = 0;// count number of fruits_eachworkParamfruitArrayfruitCount++;;// return callback objectreturn// return passed in parameterorigFruitArray: workParamfruitArray// using underscore.jsfruitNames: _pluckworkParamfruitArray "name"fruitCount: fruitCount;;// replicate Node.js module loading behaviormoduleexports = ApplesOranges;
This function takes one parameter:
fileKeyuint32 - unique key for a loaded file
There must exist a file for the given key.
// remove file associated with fileKey: 1nPoolremoveFile1;
This function queues a unit of work for execution on the thread pool. This function should be called after
createThreadPool and prior to
The function takes one parameter, a unit of work object which contains specific and required properties.
unitOfWorkObject contains the following named properties:
workId uint32 - This parameter is a unique integer that identifies a unit of work. This integer is later passed as a parameter to the work complete callback function.
fileKey uint32 - This parameter is an integer that is used as a key to reference a file that was previously cached via the
loadFile function. At this time there is no run-time logic to handle the a case when a file key does not reference a previously loaded file. Therefore, ensure that a file exists for the given key.
workFunction string - This parameter is a string that declares the name of a function. This function name will be used in conjunction with the
fileKey in order to reference a specific object instance method. The function name must match a method on the object type that is defined by the file associated with the given
fileKey. The method will be called from within a background thread to process the unit of work object passed to
queueWork. This function should ultimately return an object which is passed to the
workParam object - This is user defined object that is the input for the task. The object will be passed as the only parameter to the object instance method that is executed in the thread pool. Any function properties on the object will not be available when it is used in the thread pool because serialization does not support packing functions.
callbackFunction function - This property specifies the work complete callback function. The function is executed on the main Node.js thread.
The work complete callback function takes the following parameters:
callbackObject object - the object that is returned by the
workId uint32 - the unique identifier,
workId, that was passed with the unit of work when it was queued
exceptionObject object - the object that contains exception information
nullif no exceptions occured during work
messagestring - the exception message (always present)
resourceNamestring - name of the file where the exception occured (not always present depending on error)
lineNumuint32 - line number within the resource where the exception occured (not always present depending on error)
sourceLinestring - line of code within resource where the exception occured (not always present depending on error)
stackTracestring - string format of stack trace (includes '\n's) of the exception (not always present depending on error)
callbackContext context - This property specifies the context (
this) of the
callbackFunction when it is called.
// create the unit of work objectifexceptionObject == null// work was performed successfullyelse// exception or error occured during workconsole.logexceptionObject;var unitOfWork =// unique identifer of unit of workworkId: 34290// object type filefileKey: 1// object instance function to perform unit of workworkFunction: "objectMethodName"// object to be passed to work functionworkParam:arrayProperty:objectProperty:valueProperty: 123stringProperty: "abcd"// function that will be called on main Node.js when the task is completecallbackFunction: myCallbackFunction// context that the callbackFunction will be called incallbackContext: someOtherObject;
// queue the unit of worknPoolqueueWorkunitOfWork;
nPool emulates the Node.js module system for loaded files. The module loading system is emulated because the native functionality is embedded within the Node.js process and is only available within the main Node.js thread.
The emulated module loading system has the following features/limitations:
require(...)syntax as Node.js
../automatically resolve relative to the file performing the
// if the current path is '/home/path/'// this will require module '/home/path/aModule.js'var TheModule = require'./aModule.js';
// if the current path is '/home/path/'// this will require module '/home/aModule.js'var TheModule = require'../aModule.js';
// if the current path is '/home/path/'// this will require module '/home/path/aModule.js'var TheModule = require__dirname + '/aModule.js';
The reference implementation provided with the source (
./example) demonstrates the emulated module loading mechanism.
Copyright (c) 2015, Ivan Hall <firstname.lastname@example.org>All rights reserved.Redistribution and use in source and binary forms, with or withoutmodification, are permitted provided that the following conditions are met:* Redistributions of source code must retain the above copyrightnotice, this list of conditions and the following disclaimer.* Redistributions in binary form must reproduce the above copyrightnotice, this list of conditions and the following disclaimer in thedocumentation and/or other materials provided with the distribution.* Neither the name of nPool nor the names of its contributors may be usedto endorse or promote products derived from this software without specificprior written permission.THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" ANDANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIEDWARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE AREDISCLAIMED. IN NO EVENT SHALL THE AUTHORS BE LIABLE FOR ANYDIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED ANDON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THISSOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.