process-pool
process-pool allows you to maintain a set of sub-processes with a cached state, creating a process pool that can be used to efficiently delegate work over multiple CPU cores.
Using process pool
var moment = var ProcessPool = // Limit number of running processes to two.var pool = processLimit: 2 { return }timestart = var func = pool for var i = 1; i < 4; ++i
This would print:
begin 0: 1
begin 0: 2
end 1: 10
end 1: 20
begin 1: 3
end 2: 30
The process pool is set to run two processes concurrently, this delays the execution of the third call by a second.
Functions past to prepare
are not closures and do not have access to surrounding scope. The following would fail:
var ProcessPool = var global = 5 var pool = var pooled = pool
global
is not available within the call to prepare. To pass context to prepare the two argument version of prepare can be used:
var ProcessPool = var global1 = 2 global2 = 10 var pool = var pooled = pool
Requiring modules in a sub-process
By the the module path data is inherited from module.parent
which is the module that included process-pool
, in many cases this may not be the environment the sup-process should use. In order to use the current module path data the module
option can be used. In most cases the module
global variable provided by node should be passed which will case require
to resolve modules according to module of the current source file.
// In this case the 'pooler' module includes 'process-pool', without using// the `module` argument then require would resolve paths according to the// 'pooler' module rather than this one.var pooler = var pooled = poolerprocPool
Running multiple functions with a single pool
Many functions can be wrapped to run in a subprocess by a single pool via calls to prepare
using the processLimit
option as shown in the previous example. By default processLimit
copies of each prepare
d function are created. Up to processLimit
* number of calls to prepare
can be created but only processLimit
subprocesses will be running code at any given time, the rest will be sleeping. This can be restricted on a per function basis:
var Promise = var ProcessPool = var pool = processLimit: 3 var twoFunc = pool var oneFunc = pool
This would print:
twoFunc 1
twoFunc 2
oneFunc 1
followed by
twoFunc 3
oneFunc 2
a second later.
Future work
- Killing a pooled function should drain the wait queue.