pixl-server-pool

Overview

This module is a component for use in pixl-server. It manages pools of child worker processes, and is designed to integrate with the pixl-server-web component (but can also run independently). Using this you can delegate requests to pools of children, instead of processing everything in the parent process. This can be very useful for CPU-hard operations such as image transformations.

Worker pools can have a fixed number of workers, or grow/shrink automatically based on usage and adjustable headroom. Your worker code can listen for all child life cycle events, including startup, new request (obviously), custom message, maintenance, and shutdown. You choose which requests are delegated to worker pools, either by URI pattern (routing), or by custom API calls from your code.

Features

• Manage multiple simultaneous worker pools with separate configurations.
• Auto-scaling based on busy/idle workers.
• Configurable headroom adjustment for auto-scale events.
• Child life cycle management, max requests per child, etc.
• Custom rolling maintenance sweeps, for things like zero-downtime garbage collection.
• Automatic maintenance sweeps based on time or number of requests, or on-demand.
• Rolling worker restart requests with configurable concurrency.
• Automatic URI routing to worker pools and/or controlled delegation in code.
• Custom child worker requests (i.e. non-web-related).
• Custom messages sent to/from workers.

Table of Contents

• Usage
  • Configuration
    • Child Spawn Options
    • Child Compression
  • Delegating Requests
    • Automatic URI-Based Routing
    • Manual Request Routing
    • Worker Selection Algorithm
  • Writing Workers
    • Startup and Shutdown
    • Handling Requests
      • args
      • Text Responses
      • Binary Responses
      • JSON Responses
      • File Responses
      • Error Responses
      • Performance Tracking
      • Custom Worker URI Routing
  • Auto-Scaling
    • Child Headroom
    • Max Concurrent Requests
    • Request Queue
    • Max Requests Per Child
  • Rolling Maintenance Sweeps
    • Automatic Routine Maintenance
  • Rolling Restarts
  • Sending Custom Requests
    • Custom Request Args
    • Custom Request Errors
  • Sending Custom Pool Messages
    • Custom Worker-Sent Messages
  • Uncaught Exceptions
  • Events
    • message
    • autoscale
    • maint
    • restart
  • API
    • PoolManager
      • PoolManager.getPool
      • PoolManager.getPools
      • PoolManager.createPool
      • PoolManager.removePool
    • WorkerPool
      • WorkerPool.delegateRequest
      • WorkerPool.delegateCustom
      • WorkerPool.sendMessage
      • WorkerPool.requestMaint
      • WorkerPool.requestRestart
      • WorkerPool.getWorkers
      • WorkerPool.getWorker
      • WorkerPool.on
    • WorkerProxy
      • WorkerProxy.delegateRequest
      • WorkerProxy.delegateCustom
      • WorkerProxy.sendMessage
      • WorkerProxy.shutdown
    • Worker
      • Worker.config
      • Worker.addURIHandler
      • Worker.sendMessage
  • Client Errors
    • HTTP 403 Forbidden
    • HTTP 429 Too Many Requests
    • HTTP 500 Internal Server Error
    • HTTP 503 Service Unavailable
    • HTTP 504 Gateway Timeout
  • Logging
• License

Usage

Use npm to install the module:

npm install pixl-server pixl-server-web pixl-server-pool

Here is a simple usage example. Note that the component's official name is PoolManager, so that is what you should use for the configuration key, and for gaining access to the component via your server object.

	var PixlServer = require('pixl-server');
	var server = new PixlServer({
		
		__name: 'MyServer',
		__version: "1.0",
		
		config: {
			"log_dir": "/var/log",
			"debug_level": 9,
			
			"WebServer": {
				"http_port": 80,
				"http_htdocs_dir": "/var/www/html"
			},
			
			"PoolManager": {
				"pools": {
					"MyTestPool": {
						"script": "my_worker.js",
						"uri_match": "^/pool",
						"min_children": 1,
						"max_children": 10
					}
				}
			}
		},
		
		components: [
			require('pixl-server-web'),
			require('pixl-server-pool')
		]
		
	});
	
	server.startup( function() {
		// server startup complete
	} );

Notice how we are loading the pixl-server parent module, and then specifying pixl-server-web and pixl-server-pool as components:

	components: [
		require('pixl-server-web'),
		require('pixl-server-pool')
	]

This example demonstrates a very simple pool setup, which will automatically route incoming URIs starting with /pool, delegating those requests to worker children (up to 10 of them), and proxying their responses back to the client. The workers themselves are spawned as child processes, where your script (specified by the script property) is pre-loaded to handle requests. Example worker script:

// my_worker.js
module.exports = {
	handler: function(args, callback) {
		// handle request in child and fire callback
		callback( 
			"200 OK", 
			{ 'Content-Type': "text/html" }, 
			"Hello this is <b>custom</b> HTML content!\n" 
		);
	}
};

There is quite a bit more you can do in the worker script, including custom URI routing, startup and shutdown handlers, maintenance routine, receiving and sending custom messages, performance tracking, and more. See Writing Workers below for details on all this.

The automatic URI routing via the uri_match property is entirely optional. You can also handle requests in the parent process like a normal single-process app, and then delegate certain requests to your pools at your discretion. You can even intercept and filter the worker responses. See Manual Request Routing for more.

Configuration

The configuration for this component is specified by passing in a PoolManager key in the config element when constructing the PixlServer object, or, if a JSON configuration file is used, a PoolManager object at the outermost level of the file structure. The PoolManager object accepts these properties:

Property Name	Default Value	Description
pools	{}	Define worker pools to launch on startup (see below).
startup_threads	1	How many concurrent threads to use when launching multiple startup pools.
uncatch	false	Set to true to use uncatch for handling uncaught exceptions (see Uncaught Exceptions below).

Inside the pools object you can define one or more worker pools, which will all be launched on startup. Each pool should be assigned a unique ID (property name, used for logging), and the value should be a sub-object with configuration parameters for the pool. Example:

"PoolManager": {
	"pools": {
		"MyTestPool1": {
			"script": "my_worker.js",
			"uri_match": "^/pool1",
			"min_children": 1,
			"max_children": 10
		},
		"MyTestPool2": {
			"script": "my_other_worker.js",
			"min_children": 5,
			"max_children": 5
		}
	}
}

This example would launch two separate worker pools at startup. The first pool, MyTestPool1, would route all ^/pool1 URIs, with requests handled by a my_worker.js script (see Writing Workers below), and would launch 1 worker and auto-scale up to 10 as needed. The second pool, MyTestPool2, performs no URI routing at all (it needs requests explicitly sent in via code, see Manual Request Routing), with requests handled by a my_other_worker.js script, and would launch exactly 5 workers and never scale up or down.

Here is the complete list of available properties for your pool definitions:

Property Name	Default Value	Description
enabled	true	Enable or disable the pool (defaults to enabled).
script	''	Path to your worker script (see Writing Workers).
min_children	1	Minimum number of workers to allow (see Auto-Scaling).
max_children	1	Maximum number of workers to allow (see Auto-Scaling).
max_concurrent_requests	0	Maximum number of concurrent requests to allow (total across all workers, see Max Concurrent Requests).
max_requests_per_child	0	Maximum number of requests a worker can serve before it is cycled out (see Max Requests Per Child).
max_concurrent_launches	1	Maximum number of concurrent children to launch (for both startup and auto-scaling).
max_concurrent_maint	1	Maximum number of concurrent children to allow in a maintenance state (see Rolling Maintenance Sweeps).
child_headroom_pct	0	Percentage of workers to over-allocate, for scaling purposes (see Child Headroom.
child_busy_factor	1	Number of concurrent requests served by one child to consider it to be "busy" (see Auto-Scaling).
child_cooldown_sec	0	Minimum number of child active lifetime seconds before it can be considered for idle shutdown (see Auto-Scaling).
startup_timeout_sec	0	Maximum time allowed for workers to start up. If exceeded the process is killed and an error logged.
shutdown_timeout_sec	10	Maximum time allowed for workers to shut down. If exceeded a SIGKILL is sent and an error logged.
request_timeout_sec	0	Maximum execution time allowed per worker request. If exceeded a HTTP 504 is sent.
maint_timeout_sec	0	Maximum time allowed per workers to complete maintenance. If exceeded the worker is shut down and an error logged.
auto_maint	false	Set to true to automatically perform maintenance sweeps every N requests or N seconds (see Rolling Maintenance Sweeps).
maint_method	'requests'	When auto_maint is enabled this prop can be set to either 'requests' or 'time' (strings).
maint_requests	1000	When maint_method is set to requests this specifies the number of worker requests to count between maintenance sweeps.
maint_time_sec	0	When maint_method is set to time this specifies the number of seconds between maintenance sweeps (tracked per worker).
uri_match	''	Optionally route all incoming web requests matching URI to worker pool (see Delegating Requests).
acl	false	Used in conjunction with uri_match, optionally enable ACL restrictions for routed requests.
exec_opts	n/a	Optionally override child spawn options such as uid and gid. See Child Spawn Options.
compress_child	false	Optionally enable compression in the worker processes. See Child Compression.
compress_regex	.+	Optionally limit which Content-Type values will be encoded. See Child Compression.
gzip_opts	n/a	Optionally override the default Gzip compression settings. See Child Compression.
brotli_child	false	Optionally enable Brotli compression in worker processes. See Child Compression.
brotli_opts	n/a	Optionally override the default Brotli compression settings. See Child Compression.

Child Spawn Options

If you specify an exec_opts object in your pool configuration, you can set properties that are passed directly to the child_process.spawn() call in Node.js. Using this you can set the User ID (UID) and/or Group ID (GID) of your worker processes. Example:

"exec_opts": {
	"uid": 99,
	"gid": 99
}

Child Compression

Normally, all content compression (i.e. content encoding) happens at the web browser level, i.e. in pixl-server-web. However, the web server runs in the parent process, and thus it may become a CPU bottleneck for high traffic applications where all responses are compressed. To solve this, you can opt to have all encoding happen in the worker processes instead. This effectively allows the compression to be parallelized across CPU cores. To enable this feature, set the compress_child property to true in your pool configuration:

"MyTestPool1": {
	"script": "my_worker.js",
	"uri_match": "^/pool1",
	
	"compress_child": true
}

This will encode all pool text responses sent back by your child worker handler code, if and only if all of the following criteria are met:

• The HTTP response code is 200.
• The response body is text-based (i.e. not binary, and not a file), and has a non-zero length.
• The client request included an Accept-Encoding header, and it contains gzip, deflate or br (case-insensitive).
• The response Content-Type matches the regular expression in compress_regex (defaults to all).
• The response payload isn't already encoded (compressed) by your worker code.

The actual compression format is decided automatically, based on the client request Accept-Encoding header, and our list of supported formats (br, gzip, or deflate, matched in that order).

Note that Brotli (br) compression is new as of Node v10.16.0, so it is disabled by default. To enable it, please set the brotli_child property to true in your pool configuration, along with compress_child. Example:

"MyTestPool1": {
	"script": "my_worker.js",
	"uri_match": "^/pool1",
	
	"compress_child": true,
	"brotli_child": true
}

You can optionally configure the Brotli compression options by including a brotli_opts property. Example:

"MyTestPool1": {
	"script": "my_worker.js",
	"uri_match": "^/pool1",
	
	"compress_child": true,
	"brotli_child": true,
	"brotli_opts": {
		"chunkSize": 16384,
		"mode": "text",
		"level": 4,
		"hint": 0
	}
}

See the Node Brotli Class Options for more details on what can be set here. Note that mode is a convenience shortcut for zlib.constants.BROTLI_PARAM_MODE (which can set to text, font or generic), level is a shortcut for zlib.constants.BROTLI_PARAM_QUALITY, and hint is a shortcut for zlib.constants.BROTLI_PARAM_SIZE_HINT.

You can also optionally control the Gzip Compression Flags by setting the gzip_opts property:

"MyTestPool1": {
	"script": "my_worker.js",
	"uri_match": "^/pool1",
	
	"compress_child": true,
	"gzip_opts": {
		"level": 6,
		"memLevel": 8
	}
}

Omit this to accept the default settings, which is compression level 6, and memory level 8. See Gzip Compression Flags for more on these settings.

Finally, you can control exactly which response types are compressed by setting the compress_regex property. This is matched against your Content-Type response headers, so you can limit encoding to certain response types, e.g. text/.

"MyTestPool1": {
	"script": "my_worker.js",
	"uri_match": "^/pool1",
	
	"compress_child": true,
	"compress_regex": "^text\\/"
}

If compress_regex is omitted, it defaults to all content types.

Note: The legacy gzip_child property is still supported, and is treated as an alias to compress_child.

Delegating Requests

For delegating web requests to worker pools, you have two options. You can either use automatic routing based on URI patterns, or manually delegate requests yourself. Both methods are discussed below.

Automatic URI-Based Routing

For automatic routing based on the URI, all you need to do is specify a uri_match property in your pool configuration, and set the value to a regular expression (or a string, which will be interpreted as a regular expression) to match against incoming requests. Example:

"PoolManager": {
	"pools": {
		"MyTestPool1": {
			"uri_match": "^/pool1",
			"script": "my_worker.js"
		}
	}
}

This would route all requests with URIs that start with /pool1 to the worker pool. If you want to route all requests, just set the uri_match property to ".+" (match anything).

If you need to apply ACL restrictions to your worker requests, set the acl property to true (or an array of CIDR blocks). Example:

"PoolManager": {
	"pools": {
		"MyTestPool1": {
			"uri_match": "^/pool1",
			"acl": ['127.0.0.1', '10.0.0.0/8', '172.16.0.0/12', '192.168.0.0/16'],
			"script": "my_worker.js"
		}
	}
}

Manual Request Routing

For more control over your request workflow, you can choose exactly how and when to delegate requests to worker pools. To do this, add a standard URI or method handler via pixl-server-web, which will initially run in the parent process. Then, you can choose to delegate the request over to a worker pool, or not.

This code snippet assumes you have a preconfigured worker pool named MyTestPool1, and your pixl-server instance is in scope and named server.

server.startup( function() {
	// server startup complete, get a ref to our worker pool
	var pool = server.PoolManager.getPool('MyTestPool1');
	
	// add handler for our URI (runs in main process)
	server.WebServer.addURIHandler( /^\/pool1/, 'Pool Or Not', function(args, callback) {
		// custom request handler for our URI
		// choose whether to delegate to worker pool, or not
		if (Math.random() < 0.5) {
			// delegate request to worker pool (handles response as well)
			pool.delegateRequest( args, callback );
		}
		else {
			// handle request in main process
			callback( 
				"200 OK", 
				{ 'Content-Type': "text/html" }, 
				"Hello this is custom content!\n" 
			);
		}
	} );
} );

This is just a silly example that uses Math.random() to randomly delegate about half of all /pool1 requests to the MyTestPool1 worker pool, and serves the other half normally in the main (parent) process. This demonstrates the PoolManager.getPool() and WorkerPool.delegateRequest() APIs.

If you want to include custom data along with the request to your worker, you can put it into args.params. This object is serialized to JSON and passed directly to the worker script, and it can be used however you like. Note that it may already contain data, as it contains HTTP POST params, among other things (see args.params).

If you want to intercept the response coming back from the worker, you can do that as well. Instead of passing along the web callback to delegateRequest(), you can provide your own. Your callback will receive the standard 3 arguments from pixl-server-web URI handlers (i.e. HTTP status, headers, and body). You can manipulate these, perform additional work, and finally execute the original callback to send the response to the client. Example:

server.startup( function() {
	// server startup complete, get a ref to our worker pool
	var pool = server.PoolManager.getPool('MyTestPool1');
	
	// add handler for our URI (runs in main process)
	server.WebServer.addURIHandler( /^\/pool1/, 'My Pool', function(args, callback) {
		// custom request handler for our URI
		// delegate request to worker pool and intercept response
		pool.delegateRequest( args, function(status, headers, body) {
			// got response back from worker, let's add a header
			headers['X-Custom'] = "Added in parent process after worker completed!";
			
			// fire original web callback
			callback( status, headers, body );
		} );
	} );
} );

Worker Selection Algorithm

If multiple workers are active in your pool, the system picks an appropriate one for each request using an algorithm. The selection process is made up of two phases:

• Gather all children serving the least amount of concurrent requests.
• Pick a random child from that sub-group.

So for example, if all your children are idle, it simply picks one at random. But if some of them are serving requests, it will only pick from the least busiest group. In this way you get a nice random spread of workers chosen, but they also tend to fill up equally. You'll never have a situation where one worker is serving 10 requests while another is idle.

Writing Workers

When a request is delegated to a worker, it runs in a child process. The child communicates with the parent process via JSON on STDIN / STDOUT pipes, but all this is abstracted away from your code. All you need to do is specify the path to your Node.js worker script via the script pool configuration property, and then export some key functions:

// my_worker.js

module.exports = {
	startup: function(worker, callback) {
		// child is starting up
		this.worker = worker;
		callback();
	},
	
	handler: function(args, callback) {
		// handle request in child and fire callback
		callback( 
			"200 OK", 
			{ 'Content-Type': "text/html" }, 
			"Hello this is <b>custom</b> HTML content!\n" 
		);
	},
	
	shutdown: function(callback) {
		// child is shutting down
		callback();
	}
};

In this simple example we're exporting three functions, startup(), handler() and shutdown(). All three are discussed below, but here is the full list of supported functions which have special meanings:

Function Name	Required	Description
handler()	Yes	Called once per request, with args and a callback. See Handling Requests below.
startup()	No	Called once upon worker startup, and passed the Worker object. See Startup and Shutdown below.
shutdown()	No	Called once when worker is shutting down. See Startup and Shutdown below.
emergencyShutdown()	No	Called in the event of an uncaught exception. See Uncaught Exceptions below.
custom()	No	Called for each custom request, also with args and a callback. See Sending Custom Requests below.
message()	No	Called when worker receives a custom message. See Sending Custom Pool Messages below.
maint()	No	Called when worker needs to perform maintenance. See Rolling Maintenance Sweeps below.

You can of course add any of your own functions into module.exports, and they will be ignored. The only special functions are listed above.

Startup and Shutdown

You can optionally hook the startup and shutdown events in your worker, and run custom code. Both functions are passed a callback, so you can even perform asynchronous operations. Startup example:

// in my_worker.js
exports.startup = function(worker, callback) {
	// child is starting up, save reference to worker
	this.worker = worker;
	callback();
};

In the case of startup, your code is also passed a Worker object. This can be used for a number of things, including communication with the parent process (Sending Custom Pool Messages), getting the current pool configuration (Worker.config), and adding custom URI handlers in the worker itself (Custom Worker URI Routing).

Here is a shutdown example:

// in my_worker.js
exports.shutdown = function(callback) {
	// perform any necessary shutdown tasks here
	callback();
};

Handling Requests

To handle incoming requests in your worker and to send responses back, export a handler() function. This function will be invoked once per request, and is passed an args object containing everything you need to know about the request, and a callback. Example use:

// in my_worker.js
exports.handler = function(args, callback) {
	// handle request in child and fire callback
	callback( 
		"200 OK", 
		{ 'Content-Type': "text/html" }, 
		"Hello this is <b>custom</b> HTML content!\n" 
	);
};

As you can see the handler callback() accepts the standard 3 arguments from pixl-server-web URI handlers (i.e. HTTP status, headers, and body). However, you have more options available in worker scripts, including the ability to send back JSON (see JSON Responses), binary buffers (see Binary Responses), or entire files (see File Responses).

args

The args object, passed to your worker handler() function, contains almost identical contents to the one in pixl-server-web, with a few notable exceptions:

• args.request is present, but it's not a real http.IncomingMessage object.
  • It has all the essentials, though (see below for details).
• args.response is present, but it's not a real http.ServerResponse object.
  • Responses are handled differently in worker children, but the standard 3-arg callback still behaves as expected. See below.
• args.server is missing (doesn't exist in child land).

The args object should still provide everything you need to serve the request, including:

Property Path	Description
args.cmd	Specifies the type of request. Will be request for normal web requests, or custom for custom ones (see Sending Custom Requests).
args.id	A unique identifier for the request, used internally to match it with the correct socket at response time.
args.ip	The socket IP address of the client connection.
args.ips	An array of all the client IP addresses, including those from the X-Forwarded-For header.
args.url	The fully-qualified request URL, including the HTTP protocol and hostname.
args.request.method	The request method, e.g. GET, POST, etc.
args.request.url	The request URI, sans protocol and hostname.
args.request.headers	An object containing all the request headers, keys lower-cased.
args.request.httpVersion	The HTTP protocol version, e.g. 1.1.
args.socket.remoteAddress	An alias for args.ip.
args.params	All HTTP POST params, parsed JSON, etc. (see args.params).
args.query	The parsed query string as key/value pairs (see args.query).
args.cookies	The parsed cookie as key/value pairs (see args.cookies).
args.files	All uploaded files (see args.files).
args.perf	A pixl-perf object you can use for tracking app performance (see Performance Tracking).
args.response.type	Specifies the response type, e.g. string, buffer, json (see below).
args.response.status	The HTTP response code, e.g. 200 OK, 404 Not Found.
args.response.headers	The response headers (key/value pairs, mixed case).
args.response.body	The response body (String, Buffer, etc.). See below.
args.aborted	Will be set to true if the request was aborted, e.g. from a timeout.

Text Responses

To send a text response from your worker, you can simply fire the callback with the standard 3 arguments from pixl-server-web URI handlers (i.e. HTTP status, headers, and body). Example:

callback( 
	"200 OK", 
	{ 'Content-Type': "text/html" }, 
	"Hello this is <b>custom</b> HTML content!\n" 
);

Alternatively, you can set the following properties in the args.response object, and then fire the callback without any arguments. Example:

args.response.status = "200 OK";
args.response.headers['Content-Type'] = "text/html";
args.response.body = "Hello this is <b>custom</b> HTML content!\n";

callback();

Binary Responses

To send a binary response from your worker, you can use a Buffer object. However, depending on the size of the data, you may want to consider using a file instead (see File Responses below). The reason is memory, as the buffer has to momentarily exist in both the child and the parent. For larger blobs, a file may be faster.

To send a Buffer, fire the callback with the standard 3 arguments from pixl-server-web URI handlers, but pass the Buffer as the body (3rd argument). Example:

var buf = fs.readFileSync('binary-image.gif');

callback( 
	"200 OK", 
	{ 'Content-Type': "image/gif" }, 
	buf // binary buffer
);

Alternatively, you can set the following properties in the args.response object, and then fire the callback without any arguments. Example:

var buf = fs.readFileSync('binary-image.gif');

args.response.status = "200 OK";
args.response.headers['Content-Type'] = "image/gif";
args.response.body = buf; // binary buffer

callback();

JSON Responses

To send a JSON response, you can simply fire the callback and pass the object/array tree to be serialized as the sole argument. Example:

callback({ key1: "Value1", key2: "Value2" });

Alternatively, you can assign the object to the args.response.body property, and then fire the callback without any arguments. Example:

args.response.body = { key1: "Value1", key2: "Value2" };
callback();

These will automatically set response headers like Content-Type: application/json, including support for JSONP (i.e. callback query parameter).

File Responses

If you need to stream an entire file back to the client, there is a special mechanism for doing so. In your worker script you can set the response type to file, and then specify a filesystem path in args.response.body, along with any headers you want. All this metadata is passed to the parent process as JSON, and then the file is opened and streamed from there to the client. This is especially useful for large and/or binary files. Example:

args.response.type = "file";
args.response.status = "200 OK";
args.response.headers['Content-Type'] = "image/gif";
args.response.body = "/path/to/my/image.gif"; // file path

callback();

Error Responses

You can, of course, construct and send back your own custom error responses, but if you would prefer a generic one, simply send an Error object (or any subclass thereof) to your worker handler callback as the sole argument. Example:

var err = new Error("Something went wrong in a worker!");
callback( err );

This will be sent back to the client as an HTTP 500 Internal Server Error, with the response body set to the Error object cast to a string. The error will also be logged to the main pixl-server logging system (see Logging below).

Performance Tracking

If you want to track application performance in your workers, a pixl-perf instance is made available to your handler function, in args.perf. Metrics from this performance object are sent back to the main web server process, where they are logged (if transaction logging is enabled) and also exposed in the getStats() API.

You can track metrics directly on the args.perf object like this:

args.perf.begin('my_engine');
// do some work
args.perf.end('my_engine');

Or you can track metrics independently using your own pixl-perf instances, and import them into args.perf at the very end of the request, just before you fire the callback:

var Perf = require('pixl-perf');
var my_perf = new Perf();

my_perf.begin('my_engine');
// do some work
my_perf.end('my_engine');

// end of request
args.perf.import( my_perf );

callback( "200 OK", {}, "Success!" );

Custom Worker URI Routing

You may want to perform URI routing in the child worker rather than, or in addition to, the parent web server process. For example, your worker may serve multiple roles, activated by different URIs. In this case you'd want to first route all applicable traffic to the worker, but then perform further routing into the correct API function in your worker script.

Here is how you can accomplish this. First, setup your pool to capture all applicable URIs for your application, in the following example any URI that starts with /pool, either by using the uri_match property in your pool configuration, or by calling addURIHandler() in the parent process. Example of the former:

"PoolManager": {
	"pools": {
		"MyTestPool": {
			"uri_match": "^/pool",
			"script": "my_worker.js"
		}
	}
}

Then, in your worker child script, you can further route more specific requests to individual URI handlers, by calling Worker.addURIHandler() in your exported startup() routine:

// in my_worker.js

exports.startup = function(worker, callback) {
	// child is starting up
	this.worker = worker;
	
	// route certain URIs to different methods in child
	worker.addURIHandler( /^\/pool\/route1/, "Route 1", this.myAppRoute1.bind(this) );
	worker.addURIHandler( /^\/pool\/route2/, "Route 2", this.myAppRoute2.bind(this) );
	
	worker.addURIHandler( /^\/pool\/route3/, "Route 3", function(args, callback) {
		// handle this one inline
		callback( "200 OK", {}, "Route 3 completed!" );
	} );
	
	// startup is complete
	callback();
};

exports.myAppRoute1 = function(args, callback) {
	// this will be called for all /pool/route1 URIs
	callback( "200 OK", {}, "Route 1 completed!" );
};

exports.myAppRoute2 = function(args, callback) {
	// this will be called for all /pool/route2 URIs
	callback( "200 OK", {}, "Route 2 completed!" );
};

The idea here is that the Worker.addURIHandler() method provides a similar URI routing setup as the one in pixl-server-web, but performs the routing in the child worker itself.

Please note that if you require ACL restrictions you need to apply them in the parent (web server) process, and not in the child worker.

Worker Logging

You can use any logging system in your worker code that you wish. However, if you happen to use [pixl-logger])(https://github.com/jhuckaby/pixl-logger), you can attach this to the pool worker singleton, to augment your logs with pool-related debug events. To set this up, call attachLogAgent() on the worker object in your startup() function like this:

// in my_worker.js
const Logger = require('pixl-logger');

exports.startup = function(worker, callback) {
	// child is starting up, save reference to worker
	this.worker = worker;
	
	// setup our own logger
	var columns = ['hires_epoch', 'date', 'hostname', 'pid', 'component', 'category', 'code', 'msg', 'data'];
	this.logger = new Logger( 'logs/worker.log', columns );
	
	// attach logger to worker
	this.worker.attachLogAgent( this.logger );
	
	callback();
};

The log entries will include things such as:

• Worker startup
• Worker maintenance
• Worker shutdown
• Sending commands to the parent process (logged at level 9)
• Worker debugger events (see pixl-server-debug)

Auto-Scaling

Auto-scaling is an optional feature that will actively monitor your child workers, and spawn new ones and/or kill off idle ones as needed, based on how busy they are. This behavior is activated by setting the min_children and max_children pool configuration properties to different values. Example:

"PoolManager": {
	"pools": {
		"MyTestPool1": {
			"script": "my_worker.js",
			"uri_match": "^/pool1",
			"min_children": 2,
			"max_children": 10
		}
	}
}

This will spawn 2 children at startup, and always keep at least 2 children alive at all times. Then, based on traffic, it may spawn 8 additional children (up to 10 total) as needed. The system determines how busy children are by sampling them every tick (once per second), and checking if they are serving N or more concurrent requests (N is explained below). If all children are considered busy, and we have less than max_children alive, more are spawned. At the same time, if there are extra idle children sitting around doing nothing, they are killed off.

You can control what makes a child "busy" by adjusting the child_busy_factor property in your pool configuration. This represents the threshold of concurrent requests being served by a single child. If the child is serving this number or more, it is considered to be busy. The default value is 1.

The auto-scaling system follows these rules:

• Always ensure at least min_children workers are active.
• Never exceed max_children workers under any circumstances.
• Try to keep at least one idle worker (+headroom) available at all times.
• Only max_concurrent_launches workers are allowed to start up at one time.
• Only idle workers are considered for shutdown (those not serving any requests).
• Don't touch any workers in a maintenance state.

To disable auto-scaling entirely, simply set min_children and max_children to the same value.

Child Headroom

By default, the system always tries to keep 1 idle child worker ready at all times, while still adhering to things like max_children. So if one child is busy, another is always available to accept the next request (total busy plus one). However, sometimes this is simply not enough. For example, your workers may not start up instantly, so there may be a delay before additional workers can be made available when all current ones are busy.

To better prepare for random spikes of traffic, you can "over-allocate" a certain percentage of workers. The child_headroom_pct pool configuration property adjusts the "number of busy workers" calculation by the specified percentage, so more can always be at the ready. The system then takes this into consideration when deciding whether it needs to auto-scale or not. The basic formula is:

TARGET_CHILDREN = NUM_BUSY + (NUM_BUSY * HEADROOM_PCT / 100) + 1

So TARGET_CHILDREN is the number of children we want to have active, and is calculated by first determining how many children are "busy" (see above), and then raising that number by the headroom percentage, and finally adding 1. This value is then clamped by things like min_children and max_children.

For example, consider this configuration:

"PoolManager": {
	"pools": {
		"MyTestPool1": {
			"script": "my_worker.js",
			"uri_match": "^/pool1",
			"min_children": 1,
			"max_children": 10,
			"child_headroom_pct": 50
		}
	}
}

Here we're asking for 1 to 10 children, with 50% headroom. So if 4 children are busy, then 4 is divided by 2 (50%) and then added to the original 4, making 6. The system would then make sure 7 children were active, because it's total busy plus one. So then every tick it will spawn (or kill) children to arrive at that target number.

Max Concurrent Requests

It is highly recommended that you set the max_concurrent_requests pool configuration property to the maximum number of simultaneous requests your application can serve, across all workers. This value defaults to 0 which is basically unlimited. If additional requests come in and your application is already serving max_concurrent_requests simultaneous requests, an HTTP 429 response is sent.

So if your workers can only serve 1 concurrent request but your max_children is 10, then max_concurrent_requests should probably be 10 as well. However, if your workers can serve multiple concurrent requests, feel free to increase max_concurrent_requests beyond your max children.

This request limit can be somewhat governed by the http_max_connections setting in pixl-server-web, but that is talking about socket connections specifically. A socket may be open but inactive (i.e. keep-alive), and also the pixl-server-pool module can run independently of pixl-server-web, hence the need for its own concurrent request limit.

Request Queue

When a pool is servicing the maximum concurrent requests and more keep coming in, the default behavior is to send back a HTTP 429 response. However, if you would prefer that the extra requests be queued up, and serviced when slots become available, you can set the max_queue_size pool configuration property to any non-zero number:

"max_queue_size": 768

This would allow up to 768 requests to be queued up, before it started rejecting any. The queue system is only used if the max_concurrent_requests ceiling is reached, and additional pool requests are received. Queued requests are serviced in the order in which they were received.

It should be noted that the max_queue_size plus the max_concurrent_requests should never total more than the http_max_connections setting in pixl-server-web. If the latter limit is reached, new sockets are hard-closed. You should always leave some breathing room, for non-pool HTTP requests to be serviced, such as health checks, etc.

Max Requests Per Child

The max_requests_per_child pool configuration property sets the maximum number of requests a child will serve in its lifetime, before it is killed off and a new one spawned in its place. The default is 0 which means infinite lifetime. Increasing this can be used to curb things like memory leaks. If your workers (or any of the libraries they use) leak memory, this allows you to keep them under control.

This property can be a single number, or a range of numbers. The latter will pick a random number within the range for each child spawned. To specify a range, set the property to a two-element array like this:

"max_requests_per_child": [1000, 2000]

This would kill off children between 1000 to 2000 requests, randomly picked once per child. The idea here is that you may not want all your children to cycle out at the same time, and would rather stagger them over a wider period. This is especially important for production scale apps with heavy memory leaks, requiring a short worker lifespan.

Child Cooldown

The child_cooldown_sec pool configuration property sets a minimum amount of time (in seconds) that new children must be active until they can be considered for idle shutdown. Setting this to a non-zero value, such as 60 seconds, reduces child thrashing during periods of choppy traffic. Example:

"child_cooldown_sec": 60

Rolling Maintenance Sweeps

If you need to temporarily take your workers offline to run maintenance on them (i.e. garbage collection or other), you can do that with a rolling maintenance sweep. As long as you have multiple children, this should be a zero-downtime affair, as each child is taken out of rotation safely (well, up to max_concurrent_maint children at a time). When maintenance is completed, the child is put back into live rotation. You can control exactly what happens during maintenance, by declaring a special exported maint() function in your worker script.

You can request a rolling maintenance sweep yourself using the WorkerPool.requestMaint() call, or you can have the system routinely schedule maintenance sweeps every N requests or N seconds. Here is an example which fires it off via a web request (ACL restricted of course):

// in main process
server.startup( function() {
	// server startup complete, get a ref to our worker pool
	var pool = server.PoolManager.getPool('MyTestPool1');
	
	// add URI handler for requesting a rolling maintenance sweep
	server.WebServer.addURIHandler( /^\/pool\/maint$/, 'Pool Maintenance', true, function(args, callback) {
		// custom request handler for our URI
		pool.requestMaint();
		
		// JSON response
		callback({ code: 0, description: "Rolling maintenance sweep scheduled successfully." });
	} );
} );

This would request a rolling maintenance sweep when a request comes in for the URI /pool/maint. You obviously don't have to expose this via a web request -- this is just an example. You can have your own internal code in the parent process decide when to call WorkerPool.requestMaint(), or you can have the system schedule automatic maintenance sweeps (see below).

After a maintenance sweep has been requested, the main ticker chooses one child at a time (up to max_concurrent_maint at once), and places it into maintenance mode. This involves waiting for all its active requests to be completed, and then invoking your worker's maint() function, if defined. Example:

// in my_worker.js
exports.maint = function(user_data, callback) {
	// perform maintenance (garbage collection, etc.)
	callback();
};

You can rest assured that when your maint() function is called, your worker is not serving any other requests, so you are free to block the main thread, etc. The child is essentially "offline" until the maintenance callback is fired.

Notice that the maint() function is passed two arguments: the standard callback, but also a user_data as the first argument. This can be populated with whatever you want, if you pass it to WorkerPool.requestMaint(). It allows you to run different kinds of maintenance routines based on what you pass in.

If you do not declare a maint() function in your worker script, the default action is to run Node.js garbage collection -- that is, if global.gc is exposed via the --expose_gc command-line flag on your main process (which is passed down to all worker children). If not, no action is taken.

Automatic Routine Maintenance

To schedule automatic maintenance sweeps, set the auto_maint pool configuration property to true, and then choose a timing method via maint_method. You can opt to run maintenance every N requests served ("requests"), or every N seconds ("time"). Example of the former:

{
	"auto_maint": true,
	"maint_method": "requests",
	"maint_requests": 1000
}

This would run maintenance every 1,000 requests served (per each child). Or, you can do it via worker elapsed time instead:

{
	"auto_maint": true,
	"maint_method": "time",
	"maint_time_sec": 300
}

This would run maintenance every 300 seconds (5 minutes). Note that both methods are calculated per worker. This is important, because new children can be spawned at any time, and so their maintenance needs will differ from each other, especially if you are performing a task like garbage collection.

Rolling Restarts

If you need to restart all your workers and replace them with new ones, you can do that with a rolling restart request. As long as you have multiple children, this should be a zero-downtime affair, as each child is taken out of rotation individually (well, up to max_concurrent_launches children at a time). You can request a rolling restart using the WorkerPool.requestRestart() call. Here is an example which fires it off via a web request (ACL restricted of course):

// in main process
server.startup( function() {
	// server startup complete, get a ref to our worker pool
	var pool = server.PoolManager.getPool('MyTestPool1');
	
	// add URI handler for requesting a rolling restart
	server.WebServer.addURIHandler( /^\/pool\/restart$/, 'Pool Restart', true, function(args, callback) {
		// custom request handler for our URI
		pool.requestRestart();
		
		// JSON response
		callback({ code: 0, description: "Rolling restart scheduled successfully." });
	} );
} );

This would request a rolling restart when a request comes in for the URI /pool/restart. You obviously don't have to expose this via a web request -- this is just an example. You can have your own internal code in the parent process decide when to call WorkerPool.requestRestart().

If you want to catch the worker shutdown and run your own cleanup code, simply export a shutdown() function in your worker script. Note that it is passed a callback which you must fire, to signal that shutdown is complete and the process can exit. Example:

// in my_worker.js
exports.shutdown = function(callback) {
	// perform any necessary shutdown tasks here
	callback();
};

Sending Custom Requests

Custom requests offer the ability for you to send a completely user-defined request to your worker pool, and then capture the response, all potentially outside of a normal HTTP request workflow. The contents of the request and the response are entirely up to your code. So instead of "handing off" an HTTP request to a worker child, you're just passing it a custom user-defined object, and receiving one in return. Custom requests are sent using the WorkerPool.delegateCustom() method.

One potential use of custom requests is to handle most of your application logic in the parent web process, i.e. parse the HTTP request, perform things like authentication, database queries, etc., but then delegate a smaller side task to a worker pool. For example, a CPU-hard image transformation, or some operation that requires process-level parallelization. Then, handle the HTTP response back in the parent process.

Here is an example. This code snippet runs in the parent (web server) process, and assumes you have a pool variable in scope, which was obtained by calling PoolManager.getPool().

// in main web server process
var user_req = {
	// custom request object, can contain anything you want
	myKey1: "My Value 1",
	myComplexObject: [ 1, 2, "Three", { Four: 4 } ]
};

// send custom request to a worker
pool.delegateCustom( user_req, function(err, user_resp) {
	// got response back from worker
	if (err) {
		// handle error
	}
	// 'user_resp' is custom user-defined object passed to callback from worker
	console.log( user_resp.myRespKey1, user_resp.myRespKey2 );
} );

Please note that your user-defined request object must be able to survive serialization to/from JSON. So please use only JavaScript primitives, like objects, arrays, strings, numbers and/or booleans.

The callback is passed an Error object (see Custom Request Errors below) or null for success, and a user-defined response object, which is entirely dictated by your user code in your worker script.

In your worker script, every custom request arrives by firing your exported custom() function. It is passed a minimal args object and a callback, similar to a web request. Example:

// in my_worker.js
exports.custom = function(args, callback) {
	// handle custom request in child and fire callback
	// 'args.params' is the user_req object
	// in this example: args.params.myKey1 and args.params.myComplexObject
	
	// fire callback with null for no error, and a custom response object
	callback( null, { myRespKey1: "My Response Value 1", myRespKey2: 12345 } );
};

As you can see, args.params contains everything passed in the user_req object demonstrated above. See Custom Request Args for a list of everything available in args. After completing your custom work, fire the callback with an Error if one occurred (or null/false if not) and a custom user-defined response object, which will be passed back to the calling code in the parent web process. Your user-defined response object must also be able to survive serialization to/from JSON. So please use only JavaScript primitives, like objects, arrays, strings, numbers and/or booleans.

Please note that custom requests still count against the worker's Max Requests Per Child, and the pooler still honors things like Max Concurrent Requests. A single worker is still chosen from the pool using the Worker Selection Algorithm, and only idle workers (those not starting up, shutting down or in maintenance mode) are picked. The only real difference here is that a custom request isn't HTTP specific -- it is 100% user defined, in both the request and the response.

Custom Request Args

The custom request version of the args object is pretty minimal, compared to the main args used in web requests. Here is everything that is provided:

Property Path	Description
args.cmd	Specifies the type of request, which will always be custom in this case.
args.id	A unique identifier for the request, used internally to match it up with the correct calling thread.
args.params	A copy of your user-defined request object, which you passed to WorkerPool.delegateCustom().
args.perf	A pixl-perf object you can use for tracking app performance (see Performance Tracking).

When using args.perf for tracking performance in your worker custom requests, please note that the metrics aren't logged or used in the web server process at all, like they are with delegated web requests. For custom requests, you have to explicitly receive the performance object, and log or otherwise use the metrics yourself. args.perf is passed to the WorkerPool.delegateCustom callback as the 3rd argument, after your custom response object:

pool.delegateCustom( user_req, function(err, user_resp, perf) {
	// got response back from worker
	if (err) {
		// handle error
	}
	// 'perf' is a pixl-perf object containing metrics for the custom request
	console.log( perf.metrics() );
} );

The performance object will contain one worker metric, which is the total round-trip time from parent to worker to parent. It will also contain any of your own metrics, if you you added them in your worker script. See Performance Tracking for more details.

Custom Request Errors

A number of errors may be emitted when using custom requests. These will be passed into to your WorkerPool.delegateCustom callback as the first argument. The errors will all have a code property (string), as well as a standard message. Here are the possibilities:

Error Code	Description
429 Too Many Requests	Too many simultaneous requests being served (i.e. max_concurrent_requests).
500 Internal Server Error	An error occurred in the child worker (see below).
503 Service Unavailable	No worker available (should never happen, see HTTP 503 below).
504 Gateway Timeout	The request took too long and timed out (i.e. request_timeout_sec).

Of course, the error may be generated from your worker script (as in, you passed an Error object to the callback as the first argument). In this case, the error is converted to a 500 Internal Server Error, and a string representation of your error is passed as the message property.

Sending Custom Pool Messages

If you need to notify all your workers about something (e.g. configuration file changed, force cache flush, etc.) you can broadcast a custom message to them. Unlike custom requests, messages are broadcast to all workers simultaneously, and it is stateless. A custom message cannot be directly "responded" to like a request can. You can have workers send separate messages back to the parent process (see below), which can be caught by listening for the message event, but those messages are inherently disconnected from any previous message.

To broadcast a custom message to all pool workers, use the WorkerPool.sendMessage() method. Example:

// in main web server process
pool.sendMessage({ myKey1: 12345, myKey2: "Custom!" });

This code snippet runs in the parent (web server) process, and assumes you have a pool variable in scope, which was obtained by calling PoolManager.getPool().

You can pass any user-defined object as the message, as long as it is able to survive serialization to/from JSON. So please use only JavaScript primitives, like objects, arrays, strings, numbers and/or booleans. Note that there is no callback here (messages are fire-and-forget).

In your worker script, custom messages arrive via your exported message() function, with the user-defined message object as the sole argument. Note that all workers in the pool are sent the same message simultaneously. Example:

// in my_worker.js
exports.message = function(user_data) {
	// received custom message
	// 'user_data` is whatever object was passed to pool.sendMessage()
	// in this example: user_data.myKey1, user_data.myKey2
};

Again, notice that there is no callback here. Messages are one-way deals. That being said, there is an API for sending separate messages from workers back to the parent web server process. See below for details on this.

Please note that messages do not care what state the worker is in. Even if the child is in the middle of maintenance, or startup or shutdown, the message will still be sent, and the worker's message() function will be called, as soon as the Node.js event loop has an available thread in the child process. You can, of course, choose to ignore the message or delay acting on it in your own code.

Custom messages also do not count against the Max Requests Per Child counter, and Max Concurrent Requests is ignored entirely as well.

Custom Worker-Sent Messages

In addition to broadcasting messages from the pool to all workers, you can send messages in the reverse direction as well, i.e. from a worker to the parent web process. This is done by calling the Worker.sendMessage() method in your worker script.

To use this API, first make sure you store a copy of the Worker object initially passed to your exported startup() function:

// in my_worker.js
exports.startup = function(worker, callback) {
	// child is starting up, save reference to worker
	this.worker = worker;
	callback();
};

Now you can invoke methods on this.worker whenever you want, even outside of a request workflow (i.e. from a timer or other event). Here is an example of sending a custom message from the worker to the parent web process:

// in my_worker.js
this.worker.sendMessage({ myWorkerKey1: 12345, myWorkerKey2: "Hello" });

As with Sending Custom Pool Messages, the Worker.sendMessage() accepts any user-defined object as its sole argument. There is no callback (messages are one-way and fire-and-forget).

Back in the main (web server) process, worker-sent messages are received in the WorkerPool object and a message event is emitted (WorkerPool is an EventEmitter). Here is how to listen for it:

// in main web server process
pool.on('message', function(message) {
	// received message sent from worker
	// message.pid is the PID of the sender
	// message.data is the raw user-defined data
	// in this example: message.data.myWorkerKey1, message.data.myWorkerKey2
});

This code snippet runs in the parent (web server) process, and assumes you have a pool variable in scope, which was obtained by calling PoolManager.getPool().

So here we're using WorkerPool.on() to register an event listener for the message event. Note that it is passed a single object which contains both the PID of the worker process which sent the message, and the raw message object itself (user-defined).

The PID can be useful because you can pass it to WorkerPool.getWorker() to retrieve the actual worker object itself.

Uncaught Exceptions

You can opt-in to allow the uncatch module to manage uncaught exceptions for you. This is done by setting the global uncatch configuration property to true, and will cause the pooler to register its own listener that shuts down all pools and all workers on any uncaught exception.

The idea with uncatch is that multiple modules can all register listeners, and that includes your application code. Example:

// in main web server process
require('uncatch').on('uncaughtException', function(err) {
	// run your own sync shutdown code here
	// do not call process.exit
});

On an uncaught exception, this code would run in addition to the pooler performing its own emergency shutdown routine. Uncatch then emits the error and stack trace to STDERR and calls process.exit(1) after all listeners have executed.

When the feature is enabled, Uncatch is propagated to your child worker code as well. On the child side, uncaught exceptions will trigger an emergencyShutdown() method call if exported from your worker script. Example:

// in my_worker.js
exports.emergencyShutdown = function(err) {
	// emergency crash, perform last second operations
	// do not call process.exit here
};

There are two cases where emergencyShutdown() would be called: An uncaught exception in the child, and also an uncaught exception in the parent. For the latter, the worker children are all sent a SIGTERM signal, which is caught and emergencyShutdown() is also invoked, just prior to the child exiting. You can tell which one happened by looking at the Error object passed to emergencyShutdown(). For a termination signal due to parent crash, the error code property will be set to the string SIGTERM. Otherwise, the error object is the raw uncaught exception from the child.

Note that if you enable the uncatch feature but do not export an emergencyShutdown() method in your worker script, your regular shutdown() method is called. If neither method is exported then the process simply exits.

Events

The WorkerPool class is an EventEmitter, and can emit the following events:

message

The message event is emitted when a worker sends a custom message back to the parent (web server) process. The message itself is an object containing the PID of the sender (pid) and the user data itself (data). See Custom Worker-Sent Messages above for details.

autoscale

The autoscale event is emitted whenever the worker pool is scaling up (adding a worker) or scaling down (removing a worker). The event object will contain:

Property	Description
cmd	Will be set to either "add" or "remove".
pid	The PID of the worker being added or removed.

Example:

pool.on('autoscale', function(event) {
	// an auto-scale event is taking place.
	// event.cmd will be either "add" or "remove".
	// event.pid is the PID of the worker being added or removed.
});

maint

The maint event is emitted whenever maintenance is starting on a worker. For a Rolling Maintenance Sweep this event will be emitted once for every worker. The event object is the WorkerProxy representing the worker. Example:

pool.on('maint', function(worker) {
	// maintenance is being performed on a worker
	console.log( worker.pid );
});

restart

The restart event is emitted whenever a worker is being restarted, or simply shut down. This can happen if a worker reaches the end of its lifespan (see Max Requests Per Child), or upon request. For a Rolling Restart request this event will be emitted once for every worker. The event object is the WorkerProxy representing the worker. Example:

pool.on('restart', function(worker) {
	// worker is being restarted or shut down
	console.log( worker.pid );
});

Note that the worker is actually shut down entirely, and may be replaced with a new worker, with a new PID. The autoscale event should also fire in this case.

API

This section is a reference for all classes and methods.

PoolManager

The PoolManager class is a singleton, and is the main pixl-server component which runs the pool show. It manages all worker pools, and provides the main entry point for API calls. You can gain access via the PoolManager property in the main server object. Example:

// in main web server process
var poolmgr = server.PoolManager;

This code snippet assumes your pixl-server instance is in scope and named server.

PoolManager.getPool

The getPool() method retrieves a WorkerPool object given its ID. The ID is user-defined, from your pool configuration. Example:

// in main web server process
var pool = server.PoolManager.getPool('MyTestPool1');

This code snippet assumes your pixl-server instance is in scope and named server.

PoolManager.getPools

The getPools() method retrieves all the current active WorkerPool objects in a hash, keyed by their pool IDs. Example:

// in main web server process
var pools = server.PoolManager.getPools();
var pool = pools['MyTestPool1'];

PoolManager.createPool

The createPool() method allows you to dynamically create a new WorkerPool at runtime. This is an advanced method and should only be used if you know exactly what you are doing. The method accepts an ID (string), a configuration object for the new pool, and a callback to fire once all workers are launched and ready. The method also returns the new WorkerPool object. Example use:

// in main web server process
var config = {
	"script": "my_new_worker.js",
	"min_children": 1,
	"max_children": 2
};
var pool = server.PoolManager.createPool( 'MyNewPool', config, function(err) {
	if (err) throw err;
	// if no error then pool has started up successfully and is ready to work
} );

This would dynamically add a new worker pool with ID MyNewPool, and immediately launch 1 child (i.e. min_children). The callback would fire once the child was started up and ready to receive requests.

PoolManager.removePool

The removePool() method shuts down and removes a WorkerPool at runtime. This is an advanced method and should only be used if you know exactly what you are doing. It should only be used to remove pools that you yourself added via PoolManager.createPool(), i.e. do not use this to remove any startup pools in the main configuration file. The method accepts a Pool ID (string), and a callback which is fired once the pool and all workers are completely shut down. Example:

// in main web server process
server.PoolManager.removePool( 'MyNewPool', function(err) {
	if (err) throw err;
	// if no error then pool is completely shut down and removed
});

WorkerPool

The WorkerPool class represents one pool of workers. It can be retrieved by calling PoolManager.getPool().

WorkerPool.delegateRequest

The WorkerPool.delegateRequest() method delegates a web request to a worker pool. It picks a suitable worker using the Worker Selection Algorithm. It should be passed the args object from pixl-server-web, and the web callback. Example:

// in main web server process
server.startup( function() {
	// server startup complete, get a ref to our worker pool
	var pool = server.PoolManager.getPool('MyTestPool1');
	
	// add handler for our URI (runs in main process)
	server.WebServer.addURIHandler( /^\/pool1/, 'My Pool', function(args, callback) {
		// custom request handler for our URI
		// delegate request to worker pool (handles response as well)
		pool.delegateRequest( args, callback );
	} );
} );

This code snippet assumes you have a preconfigured worker pool named MyTestPool1, and your pixl-server instance is in scope and named server.

You can also intercept the response from the worker by providing your own callback. See Manual Request Routing for more details.

WorkerPool.delegateCustom

The WorkerPool.delegateCustom() method sends a custom request to a worker pool (i.e. not web-related). It picks a suitable worker using the Worker Selection Algorithm. You can pass any custom user-defined object, as long as it can be JSON serialized. Example:

// in main web server process
// send custom request to a worker
pool.delegateCustom( { custom1: 12345, custom2: "Hello" }, function(err, user_resp) {
	// got response back from worker
	if (err) {
		// handle error
	}
	// 'user_resp' is custom user-defined object passed to callback from worker
} );

This code snippet assumes you have a pool variable in scope, which was obtained by calling PoolManager.getPool().

See Sending Custom Requests for more details.

WorkerPool.sendMessage

The WorkerPool.sendMessage() method broadcasts a custom message to all workers simultaneously. Message sending is a stateless system with no callbacks (messages are one-way and fire-and-forget). Example:

// in main web server process
pool.sendMessage({ myKey1: 12345, myKey2: "Custom!" });

This code snippet assumes you have a pool variable in scope, which was obtained by calling PoolManager.getPool().

See Sending Custom Pool Messages for more details.

WorkerPool.requestMaint

The WorkerPool.requestMaint() method requests a rolling maintenance sweep across all workers. The pooler will then choose up to max_concurrent_maint workers per tick, take them out of service, run maintenance, and then put them back in. Example:

// in main web server process
pool.requestMaint();

You can optionally pass in custom user-defined object, which is sent all the way to the exported maint() function in your worker script (note that it must be JSON-safe). Example:

// in main web server process
pool.requestMaint({ myKey1: 12345, myKey2: "Maint!" });

These code snippets assume you have a pool variable in scope, which was obtained by calling PoolManager.getPool().

Note that if you only have one active worker child, this call has no effect. The system will only send children into maintenance mode if there are more available to service requests.

See Rolling Maintenance Sweeps for more details.

WorkerPool.requestRestart

The WorkerPool.requestRestart() method requests a rolling restart of all workers. The pooler will choose up to max_concurrent_launches workers per tick, shut them down, and the auto-scaler will replace them with new ones. Example:

// in main web server process
pool.requestRestart();

This code snippet assumes you have a pool variable in scope, which was obtained by calling PoolManager.getPool().

Note that if you only have one active worker child, this call has no effect. The system will only restart children if there are more available to service requests.

See Rolling Restarts for more details.

WorkerPool.getWorkers

The WorkerPool.getWorkers method returns an object containing all current workers in the pool. The object's keys are the worker PIDs, and the values are WorkerProxy objects. Example:

// in main web server process
var workers = pool.getWorkers();

This code snippet assumes you have a pool variable in scope, which was obtained by calling PoolManager.getPool().

WorkerPool.getWorker

The WorkerPool.getWorker() method fetches a single worker from the pool, given its PID. The response is a WorkerProxy object. To determine the PIDs of all workers, see WorkerPool.getWorkers() above. Example:

// in main web server process
var worker = pool.getWorker( 1234 ); // pid

This code snippet assumes you have a pool variable in scope, which was obtained by calling PoolManager.getPool().

WorkerPool.on

The WorkerPool.on() method is inherited from the Node.js EventEmitter class. It allows you to add listeners for events emitted on your WorkerPool objects. Example:

// in main web server process
pool.on('maint', function(worker) {
	// maintenance is being performed on a worker
	console.log( worker.pid );
});

This code snippet assumes you have a pool variable in scope, which was obtained by calling PoolManager.getPool().

See Events for a list of all available events you can listen for.

WorkerProxy

The WorkerProxy class represents one single worker in a pool, but runs in the main (web server) process. It links to the actual child process via STDIN/STDOUT pipes. You can fetch all the WorkerProxy objects for all workers in a pool by calling WorkerPool.getWorkers().

WorkerProxy.delegateRequest

The WorkerProxy.delegateRequest() method delegates a web request to a specific worker. It bypasses the Worker Selection Algorithm and targets the exact worker you call it on. This is an advanced function and should be used with great care. It should be passed the args object from pixl-server-web, and the web callback. Example:

// in main web server process
server.startup( function() {
	// server startup complete, get a ref to our worker pool
	var pool = server.PoolManager.getPool('MyTestPool1');
	
	// add handler for our URI (runs in main process)
	server.WebServer.addURIHandler( /^\/pool1/, 'My Pool', function(args, callback) {
		// custom request handler for our URI
		// delegate request to specific worker in pool (handles response as well)
		var worker = pool.getWorker( 1234 ); // PID
		worker.delegateRequest( args, callback );
	} );
} );

This code snippet assumes you have a preconfigured worker pool named MyTestPool1, and your pixl-server instance is in scope and named server.

WorkerProxy.delegateCustom

The WorkerProxy.delegateCustom() method sends a custom request to a specific worker. It bypasses the Worker Selection Algorithm and targets the exact worker you call it on. This is an advanced function and should be used with great care. You can pass any custom user-defined object, as long as it can be JSON serialized. Example:

// in main web server process
var worker = pool.getWorker( 1234 ); // PID

// send custom request to a specific worker
worker.delegateCustom( { custom1: 12345, custom2: "Hello" }, function(err, user_resp) {
	// got response back from worker
	if (err) {
		// handle error
	}
	// 'user_resp' is custom user-defined object passed to callback from worker
} );

This code snippet assumes you have a pool variable in scope, which was obtained by calling PoolManager.getPool().

WorkerProxy.sendMessage

The WorkerProxy.sendMessage() method sends a custom message to a single worker. Message sending is a stateless system with no callbacks (messages are one-way and fire-and-forget). Example:

// in main web server process
var worker = pool.getWorker( 1234 ); // PID
worker.sendMessage({ myKey1: 12345, myKey2: "Custom!" });

This code snippet assumes you have a pool variable in scope, which was obtained by calling PoolManager.getPool().

WorkerProxy.shutdown

The WorkerProxy.shutdown() method shuts down a worker (kills the child process and removes the worker from the pool once it has exited). This is an advanced / internal method used by the auto-scaler, and should only be called if you know exactly what you are doing. Note that the auto-scaler may spawn a new child as soon as one is shut down, to maintain the desired number of workers in the pool. Example:

// in main web server process
var worker = pool.getWorker( 1234 ); // PID
worker.shutdown();

Worker

The Worker object is a singleton that exists in every child worker process. It handles all the communication with the parent process, and calls your worker script's exported functions when necessary. It also provides a few API calls and properties you can access.

The object is passed to your exported startup() function in your worker script. It is recommended that you save this so you can access it later. Example:

// in my_worker.js
exports.startup = function(worker, callback) {
	// child is starting up, save reference to worker
	this.worker = worker;
	callback();
};

Worker.config

The Worker.config property is a copy of the worker pool's configuration object (containing properties like max_children, max_requests_per_child, etc.). It also contains an id property which is the Pool's ID (configuration object key).

Worker.addURIHandler

The Worker.addURIHandler() method allows you to add URI routing in your worker script, as opposed to (or in addition to) URI routing in the main (web server) process. The calling convention is similar to pixl-server-web. Example:

// in my_worker.js
exports.startup = function(worker, callback) {
	// child is starting up
	this.worker = worker;
	
	// route specific URI to method in worker script
	worker.addURIHandler( /^\/pool\/route1/, "Route 1", this.myAppRoute1.bind(this) );
	
	// startup is complete
	callback();
};

exports.myAppRoute1 = function(args, callback) {
	// this will be called for all /pool/route1 URIs
	callback( "200 OK", {}, "Route 1 completed!" );
};

See Custom Worker URI Routing for more details.

Worker.sendMessage

The Worker.sendMessage() method sends a custom user-defined message from your worker script back to the main (web server) process. It is captured by the worker pool object and emitted as a message event. Example:

// in my_worker.js
this.worker.sendMessage({ myWorkerKey1: 12345, myWorkerKey2: "Hello" });

As with Sending Custom Pool Messages, the Worker.sendMessage() accepts any user-defined object as its sole argument. This is no callback (messages are one-way and fire-and-forget).

See Custom Worker-Sent Messages for more details.

Client Errors

The following HTTP errors may be sent to your clients in certain situation. Here is an explanation of when and why each can occur.

HTTP 403 Forbidden

The HTTP 403 Forbidden error is sent back to clients if an incoming request fails the ACL check. This is only applicable if you use the uri_match and acl features in your pool configuration, and an incoming request bound for your worker pool is from an IP address outside the ACL.

HTTP 429 Too Many Requests

The HTTP 429 Too Many Requests error is sent back to clients if too many simultaneous requests are being served by your worker pool. This limit is set via the max_concurrent_requests pool configuration property, and possibly the the max_queue_size property, if set.

HTTP 500 Internal Server Error

The HTTP 500 Internal Server Error error is sent back to clients if one of the following situations occur:

• A worker (child process) crashes with active pool requests.
• A worker attempts to proxy a file response and the specified file cannot be read.
• A worker explicitly passes an Error object to the request callback.

HTTP 503 Service Unavailable

The HTTP 503 Service Unavailable error should theoretically never happen. It means that an available worker could not be found to service a pool request. The only way this can ever happen is if every single worker is otherwise tied up in states such as maintenance, startup or shutdown.

Great care has been taken to ensure that these situations never occur, even with auto-scaling and rolling maintenance / restarts. However, if you are only running a single child for some reason, or too many children exit at once and the system cannot spawn new ones quickly enough, a 503 error may occur.

Make sure you always have plenty of workers available, and use the child_headroom_pct feature to over-allocate as well.

HTTP 504 Gateway Timeout

The HTTP 504 Gateway Timeout error is sent back to clients if a worker takes too long to service a request. This timeout is set via the request_timeout_sec pool configuration property.

Logging

The pooler uses the logging system built into pixl-server. The component column will be set to either PoolManager or Pool-[ID] (where [ID] is the ID of your worker pool). Most debug messages are pool-specific.

Here is an example log excerpt showing a typical startup with one pool (TestPool) and 2 workers. In all these log examples the first 3 columns (hires_epoch, date and hostname) are omitted for display purposes. The columns shown are component, category, code, msg, and data.

[PoolManager][debug][3][pixl-server-pool v1.0.0 starting up][]
[Pool-TestPool][debug][2][Starting up pool][{"enabled":true,"script":"my_child.js","uri_match":"^/pool","min_children":2,"max_children":10,"max_concurrent_requests":10,"max_requests_per_child":0,"startup_timeout_sec":10,"id":"TestPool","exec_path":"","exec_args":[],"exec_env":{},"exec_opts":{},"max_concurrent_launches":1,"max_concurrent_maint":1,"child_headroom_pct":0,"child_busy_factor":1,"shutdown_timeout_sec":10,"request_timeout_sec":0,"maint_timeout_sec":0,"auto_maint":false,"maint_method":"requests","maint_requests":1000,"maint_time_sec":0,"acl":false}]
[Pool-TestPool][debug][4][Worker starting up][]
[Pool-TestPool][debug][4][Spawned new child process: 6334][{"cmd":"/usr/local/bin/node","args":["/Users/jhuckaby/node_modules/pixl-server-pool/worker.js"],"script":"my_child.js"}]
[Pool-TestPool][debug][5][Current worker states][{"startup":1}]
[Pool-TestPool][debug][5][Worker 6334 startup complete, ready to serve][]
[Pool-TestPool][debug][5][Worker 6334 changing state from 'startup' to 'active'][]
[Pool-TestPool][debug][5][Current worker states][{"active":1}]
[Pool-TestPool][debug][4][Worker starting up][]
[Pool-TestPool][debug][4][Spawned new child process: 6335][{"cmd":"/usr/local/bin/node","args":["/Users/jhuckaby/node_modules/pixl-server-pool/worker.js"],"script":"my_child.js"}]
[Pool-TestPool][debug][5][Current worker states][{"active":1,"startup":1}]
[Pool-TestPool][debug][5][Worker 6335 startup complete, ready to serve][]
[Pool-TestPool][debug][5][Worker 6335 changing state from 'startup' to 'active'][]
[Pool-TestPool][debug][5][Current worker states][{"active":2}]
[Pool-TestPool][debug][2][Pool startup complete][]

Here is a single delegated web request (most of this is logged by pixl-server-web):

[WebServer][debug][8][New incoming HTTP connection: c8][{"ip":"::ffff:127.0.0.1","num_conns":1}]
[WebServer][debug][8][New HTTP request: GET /pool/json?foo=bar (::ffff:127.0.0.1)][{"socket":"c8","version":"1.1"}]
[WebServer][debug][9][Incoming HTTP Headers:][{"host":"127.0.0.1:3012","user-agent":"curl/7.54.0","accept":"*/*"}]
[WebServer][debug][6][Invoking handler for request: GET /pool/json: TestPool][]
[Pool-TestPool][debug][9][Chose worker: 6535 for request: /pool/json?foo=bar][]
[WebServer][debug][9][Sending HTTP response: 200 OK][{"Content-Type":"application/json","X-JoeTest":1234,"Server":"Test Server 1.0","Content-Length":90}]
[WebServer][debug][9][Request complete][]
[WebServer][debug][9][Response finished writing to socket][]
[WebServer][debug][9][Request performance metrics:][{"scale":1000,"perf":{"total":2.744,"read":0.011,"process":2.022,"worker":1.649,"write":0.685},"counters":{"bytes_in":95,"bytes_out":201,"num_requests":1}}]
[WebServer][debug][9][Closing socket: c8][]
[WebServer][debug][8][HTTP connection has closed: c8][{"ip":"::ffff:127.0.0.1","total_elapsed":10,"num_requests":1,"bytes_in":95,"bytes_out":201}]

Here is an example of a rolling maintenance sweep:

[Pool-TestPool][debug][4][Beginning rolling maintenance of all workers in pool][]
[Pool-TestPool][debug][3][Peforming maintenance on worker: 6631][]
[Pool-TestPool][debug][4][Worker 6631 entering maintenance mode][]
[Pool-TestPool][debug][6][Sending 'maint' command to process: 6631][]
[Pool-TestPool][debug][5][Worker 6631 changing state from 'active' to 'maint'][]
[Pool-TestPool][debug][5][Current worker states][{"active":1,"maint":1}]
[Pool-TestPool][debug][4][Maintenance complete on child: 6631][]
[Pool-TestPool][debug][5][Worker 6631 changing state from 'maint' to 'active'][]
[Pool-TestPool][debug][5][Current worker states][{"active":2}]
[Pool-TestPool][debug][3][Peforming maintenance on worker: 6630][]
[Pool-TestPool][debug][4][Worker 6630 entering maintenance mode][]
[Pool-TestPool][debug][6][Sending 'maint' command to process: 6630][]
[Pool-TestPool][debug][5][Worker 6630 changing state from 'active' to 'maint'][]
[Pool-TestPool][debug][5][Current worker states][{"maint":1,"active":1}]
[Pool-TestPool][debug][4][Maintenance complete on child: 6630][]
[Pool-TestPool][debug][5][Worker 6630 changing state from 'maint' to 'active'][]
[Pool-TestPool][debug][5][Current worker states][{"active":2}]

Here is a auto-scale event, showing what is logged when a new worker is added to the pool:

[Pool-TestPool][debug][4][Auto-Scale: Adding worker to pool][{"num_busy":0}]
[Pool-TestPool][debug][4][Worker starting up][]
[Pool-TestPool][debug][4][Spawned new child process: 6606][{"cmd":"/usr/local/bin/node","args":["/Users/jhuckaby/node_modules/pixl-server-pool/worker.js"],"script":"my_child.js"}]
[Pool-TestPool][debug][5][Current worker states][{"maint":1,"active":1,"startup":1}]
[Pool-TestPool][debug][5][Worker 6606 startup complete, ready to serve][]
[Pool-TestPool][debug][5][Worker 6606 changing state from 'startup' to 'active'][]
[Pool-TestPool][debug][5][Current worker states][{"maint":1,"active":2}]

Here is another auto-scale event, removing an idle worker:

[Pool-TestPool][debug][4][Auto-Scale: Removing idle worker: 6535][{"num_busy":0}]
[Pool-TestPool][debug][4][Worker 6535 shutting down (6 requests served)][]
[Pool-TestPool][debug][6][Sending 'shutdown' command to process: 6535][]
[Pool-TestPool][debug][5][Worker 6535 changing state from 'active' to 'shutdown'][]
[Pool-TestPool][debug][5][Current worker states][{"shutdown":1,"active":2}]
[Pool-TestPool][debug][4][Child 6535 exited with code: 0][]
[Pool-TestPool][debug][4][Worker 6535 has been removed from the pool][]
[Pool-TestPool][debug][5][Current worker states][{"active":2}]

Here is a rolling restart sequence:

[Pool-TestPool][debug][4][Beginning rolling restart of all workers in pool][]
[Pool-TestPool][debug][3][Restarting worker 6606 upon request][]
[Pool-TestPool][debug][4][Worker 6606 shutting down (0 requests served)][]
[Pool-TestPool][debug][6][Sending 'shutdown' command to process: 6606][]
[Pool-TestPool][debug][5][Worker 6606 changing state from 'active' to 'shutdown'][]
[Pool-TestPool][debug][5][Current worker states][{"shutdown":1,"active":1}]
[Pool-TestPool][debug][4][Auto-Scale: Adding worker to pool][{"num_busy":0}]
[Pool-TestPool][debug][4][Worker starting up][]
[Pool-TestPool][debug][4][Spawned new child process: 6615][{"cmd":"/usr/local/bin/node","args":["/Users/jhuckaby/node_modules/pixl-server-pool/worker.js"],"script":"my_child.js"}]
[Pool-TestPool][debug][5][Current worker states][{"shutdown":1,"active":1,"startup":1}]
[Pool-TestPool][debug][4][Child 6606 exited with code: 0][]
[Pool-TestPool][debug][4][Worker 6606 has been removed from the pool][]
[Pool-TestPool][debug][5][Current worker states][{"active":1,"startup":1}]
[Pool-TestPool][debug][5][Worker 6615 startup complete, ready to serve][]
[Pool-TestPool][debug][5][Worker 6615 changing state from 'startup' to 'active'][]
[Pool-TestPool][debug][5][Current worker states][{"active":2}]
[Pool-TestPool][debug][3][Restarting worker 6607 upon request][]
[Pool-TestPool][debug][4][Worker 6607 shutting down (0 requests served)][]
[Pool-TestPool][debug][6][Sending 'shutdown' command to process: 6607][]
[Pool-TestPool][debug][5][Worker 6607 changing state from 'active' to 'shutdown'][]
[Pool-TestPool][debug][5][Current worker states][{"shutdown":1,"active":1}]
[Pool-TestPool][debug][4][Auto-Scale: Adding worker to pool][{"num_busy":0}]
[Pool-TestPool][debug][4][Worker starting up][]
[Pool-TestPool][debug][4][Spawned new child process: 6616][{"cmd":"/usr/local/bin/node","args":["/Users/jhuckaby/node_modules/pixl-server-pool/worker.js"],"script":"my_child.js"}]
[Pool-TestPool][debug][5][Current worker states][{"shutdown":1,"active":1,"startup":1}]
[Pool-TestPool][debug][4][Child 6607 exited with code: 0][]
[Pool-TestPool][debug][4][Worker 6607 has been removed from the pool][]
[Pool-TestPool][debug][5][Current worker states][{"active":1,"startup":1}]
[Pool-TestPool][debug][5][Worker 6616 startup complete, ready to serve][]
[Pool-TestPool][debug][5][Worker 6616 changing state from 'startup' to 'active'][]
[Pool-TestPool][debug][5][Current worker states][{"active":2}]

And here is what the log looks like for shutdown:

[PoolManager][debug][3][Worker Pool Manager shutting down][]
[Pool-TestPool][debug][2][Shutting down pool: TestPool][]
[Pool-TestPool][debug][4][Worker 6334 shutting down (0 requests served)][]
[Pool-TestPool][debug][6][Sending 'shutdown' command to process: 6334][]
[Pool-TestPool][debug][5][Worker 6334 changing state from 'active' to 'shutdown'][]
[Pool-TestPool][debug][5][Current worker states][{"shutdown":1,"active":1}]
[Pool-TestPool][debug][4][Worker 6335 shutting down (1 requests served)][]
[Pool-TestPool][debug][6][Sending 'shutdown' command to process: 6335][]
[Pool-TestPool][debug][5][Worker 6335 changing state from 'active' to 'shutdown'][]
[Pool-TestPool][debug][5][Current worker states][{"shutdown":2}]
[Pool-TestPool][debug][4][Child 6334 exited with code: 0][]
[Pool-TestPool][debug][4][Worker 6334 has been removed from the pool][]
[Pool-TestPool][debug][5][Current worker states][{"shutdown":1}]
[Pool-TestPool][debug][4][Child 6335 exited with code: 0][]
[Pool-TestPool][debug][4][Worker 6335 has been removed from the pool][]
[Pool-TestPool][debug][2][All workers exited, pool shutdown complete][]

Here is an example of an error (in this case an HTTP 500):

[Pool-TestPool][error][500][Error: Test Error Message][{"request_id":"rj5hr4brm01","worker":6697}]

If you have http_log_requests enabled in your pixl-server-web configuration, all HTTP errors are also logged as transactions:

[WebServer][transaction][HTTP 500 Internal Server Error][/pool?error=1][{"proto":"http","ips":["::ffff:127.0.0.1"],"host":"127.0.0.1:3012","ua":"curl/7.54.0","perf":{"scale":1000,"perf":{"total":12.017,"read":0.37,"process":5.741,"worker":3.837,"write":4.419},"counters":{"bytes_in":90,"bytes_out":104,"num_requests":1}}}]

License

The MIT License (MIT)

Copyright (c) 2017 - 2018 Joseph Huckaby.

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.