2.1.1 • Public • Published


Postable Task Service

A simple task distribution and result collection service.

NPM Version NPM License Build Test Coverage

Postable can receive tasks and distribute them to listeners. Results from listeners are then forwarded back to the original caller as line-delimited JSON.

Postable can be clustered, but instances must share the same Redis master.


  1. Listeners connect to Postable specifying the buckets to listen to.
  2. Tasks are submitted to Postable under a specific bucket.
  3. Listeners on that bucket receive the task as a line-delimited JSON entry.
  4. Listeners respond to Postable with the result for that specific task.
  5. Postable streams all results for the task back to the caller as line-delimited JSON.


Server Installation

  • Install Redis
  • Install Postable npm install postable
  • Set configuration as environment variables
  • Run service bin/postable

Listener Installation

  • Install Postable npm install postable
  • Set configuration as environment variables
  • Run client bin/postable-listener


Environment Variables (Server)

    Optional (defaults to 3000). The port to listen on.

    Optional (defaults to none). A semicolon-delimited set of base URIs of other postable services for task broadcasting (see Task Broadcasting below).

    Optional (defaults to The redis host to use.

    Optional (defaults to 6379). The redis port to connect to.

    Optional (defaults to none). The auth password for redis.

    Optional (defaults to postable_). The prefix for redis keys and channels.

    Optional (defaults to 5 seconds, 5000). How often to send a heartbeat message so client connections don't close.

    Optional (defaults to 20 seconds, 20). How long to keep listener data in redis without an active client connection.

    Optional (defaults to 30 mins, 1800). How long to keep listener set data in redis.

    Optional (defaults to 7 days, 604800). How long to keep the last task per bucket.

Environment Variables (Server and Listener)

    Optional (defaults to console). Where to log data.

    Optional (defaults to info). The minimum level to log.

    Optional (defaults to none). A username for basic HTTP authentication to the service.

    Optional (defaults to none). A password for basic HTTP authentication to the service.

Environment Variables (Listener)

    Required. The base URL for the Postable service.

    Required. A URL to get the listener buckets from. The buckets will be refreshed periodically.

    Required. A URL to forward the task to. Will forward as a JSON body:

      "task":  { /* task data */ }
    Optional (defaults to 50). The max number of concurrent task forwards to send to the forward URL.

    Optional (defaults to 5 seconds, 5). How soon to reconnect to postable in the case of an error.

    Optional (defaults to 1 minute, 60). How often to refresh buckets.

    Optional (defaults to 5 seconds, 5). How soon to reattempt to get buckets in the case of an error.

    Optional (defaults to 2). How many times to attempt forwarding the data to the HTTP URL.

    Optional (defaults to an empty object). Listener data (as JSON) to send to Postable.


Listening for Tasks

POST /listeners/
  "buckets": [ "bucket-1", "bucket-2", ... ],
  ... additional listener data ...
=> 200
{ "id": <taskId>, "started": <dateString>, "listenerId": <listenerId>, "data": <taskData> }

To listen for tasks on buckets, a client will POST /listeners/ with a body containing the buckets to listen to (as an array of strings).

This will be a long-poll request and as tasks come in they will be streamed to the client as line-delimited JSON. The connection is never closed by the service.

Each task will contain an id, started, listenerId, and the data from the task.

Sending a Task

POST /buckets/<bucket>/tasks/
  ... task data ...
=> 200
{ "meta": { "listenersPending": [ ... ] } }
{ "listener": { "buckets": [...], "id": <listenerId>, "started": <dateString>, ... }, "timeout": false, "data": <result> }

To send a task to a bucket, simply POST /buckets/<bucket>/tasks/ with the task data as a JSON object.

The response will be a stream of line-delimited JSON. The first entry will be a meta entry containing listenersPending (an array of listener IDs).

This task will be given a unique task ID and sent to all listeners. As listeners respond to the task with results, those results will be streamed back to this response. Each entry will contain the listener ID sending the result. Once all results have been received, the connection will close.


If the timeout is reached the connection will close with additional entries for each timed out listener with a property timeout set to true. This timeout can be configured using ?timeout=<seconds>.

Task Send Options

timeout Query-string value. Defaults to 30. Specify the listener timeout (in seconds)
max Query-string value. Defaults to no maximum. Specify the maximum number of listeners to receive the task.

Task Broadcasting

A task can be broadcast across data-centers using by calling the broadcast endpoint. Postable must be configured to use this feature (see the POSTABLE_BROADCAST option under Configuration).

This endpoint is similar to Sending a Task, but begins with /broadcast/:

POST /broadcast/buckets/<bucket>/tasks/
  ... task data ...
=> 200
{ "clusterId": "a...", "meta": { "listenersPending": [ ... ] } }
{ "clusterId": "a...", "listener": { "buckets": [...], "id": <listenerId>, "started": <dateString>, ... }, "timeout": false, "data": <result> }
{ "clusterId": "b...", "meta": { "listenersPending": [ ... ] } }

The response will be a stream of line-delimited JSON. Results from various clusters will be streamed back. One meta item containing listenersPending will be sent from each cluster.

Task Broadcasting Options

timeout Same as the above Task Send option; forwarded to each cluster.
max Same as the above Task Send option; forwarded to each cluster. Specifies the maximum number of listeners (per cluster) to receive the task. That means each cluster will send the task to the number of listeners specified by ?max.
cluster Defaults to nothing. Only clusters with the ID specified will respond. This can also be an array (?cluster[0]=<id>&cluster[1]=<id>&...).

Responding to a Task

POST /tasks/<taskId>/results/<listenerId>
  ... task result ...
=> 200

To respond to a task from a listener, simply POST /tasks/<taskId>/results/<listenerId> with the task result as a JSON object.

The <taskId> and <listenerId> should come from the initial task sent (see Listening for Tasks).

Getting the Last Task

GET /buckets/<bucket>/tasks/last
=> 200
  ... task data ...

Return the last task submitted to the given bucket as JSON, or null if there was none.

Getting Bucket Listeners

GET /buckets/<bucket>/listeners/
=> 200
    { ... listener 1 info ... },
    { ... listener 2 info ... },

Return the last task submitted to the given bucket as JSON, or null if there was none.


Postable works by using redis for pub/sub. See below for a high-level sequence diagram:

In the diagram below, Potable 1 and Postable 2 can either be the same or different boxes in the same cluster.

Sequence Diagram




npm i postable

Downloadsweekly downloads









last publish


  • avatar
Report a vulnerability