0.4.4 • Public • Published

    Cloudant Envoy (beta)

    Build Status npm version

    Beta software

    Note: this is beta; it's not battle tested or supported in any way. If you find bugs (of which there will be plenty), do let us know – or better, consider a pull request. You know what beta means, right?


    Cloudant Envoy is a microservice that acts as a replication target for your PouchDB web app or Cloudant Sync-based native app. Envoy allows your client side code can adopt a "one database per user" design pattern, with a copy of a user's data stored on the mobile device and synced to the cloud when online, while invisibly storing all the users' data in one large database. This prevents the proliferation of database that occurs as users are added and facilitates simpler backup and server-side reporting.

    Envoy implements a subset of the CouchDB API and can be used as a replication target for PouchDB or Cloudant Sync, or used with custom replicators such as pouchdb-envoy.

    Envoy includes a demo web app (hosted at the /demo/ path) which demonstrates how a basic offline-first, progressive web app can be deployed that scales easily as users are added.

    Why Cloudant Envoy?

    Database-per-user is a common pattern with CouchDB when there is a requirement for each application user to have their own set of documents which can be synced (e.g. to a mobile device or browser). On the surface, this is a good solution - Cloudant handles a large number of databases within a single installation very well. However, there are some problems:

    1. Querying data across many databases (e.g. for analytics) can be difficult. Cloudant does not have native support for a cross-database query and aggregating the data into a single database using e.g. replication can be resource intensive or require manual scheduling.
    2. There is a high management cost of many (thousands of) databases. Consider backup, replication in cross-region scenarios, design document management. These all require coordination / resource management outside of Cloudant.

    Envoy aims to work around these problems by emulating database-per-user using a single backend database and a proxy.

    When should I consider Envoy instead of a db-per-user pattern?

    • you want to perform queries across all per-user databases.
    • you want to replicate all user data to a remote target (e.g. another Cloudant account or IBM dashDB).
    • you want to take advantage of additional HTTP features (e.g. compression) that are not natively supported by Cloudant.

    When should I consider Envoy instead of a single database with filtered replication?

    Cloudant Envoy is essentially performing a filtered replication under the hood (using the new Mango/Erlang native filtering supported in CouchDB 2.0). However, you may want to consider it if:

    • you do not want to give all users read/write access to other users data.
    • you want to take advantage of additional HTTP features (e.g. compression) that are not natively supported by Cloudant.

    TODO / known limitations

    • multipart requests
    • document ownership is non-transferable (essentially the same as db per user)
    • _design documents cannot be created through the proxy (they will get saved as normal documents)


    Via npm

    Cloudant Envoy is published to npm. It can be installed and run if you have Node.js and npm installed:

    npm install -g cloudant-envoy
    export COUCH_HOST=''

    Deploy to Bluemix

    Deploy Cloudant Envoy to Bluemix by clicking the Deploy to Bluemix button below.

    Deploy to Bluemix

    Don't have a Bluemix account? If you haven't already, you'll be prompted to sign up for a Bluemix account when you click the button. Sign up, verify your email address, then return here and click the the Deploy to Bluemix button again. Your new credentials let you deploy to the platform and also to code online with Bluemix and Git. If you have questions about working in Bluemix, find answers in the Bluemix Docs.

    Manual installation

    To install the code yourself, clone the repo and run npm install. The Envoy server needs admin credentials for the backing Cloudant database, and it expects a COUCH_HOST environment variable to be set:

    export COUCH_HOST=''

    After those variables are set, you can start the Envoy server with npm start. Note that the port is the port that Envoy will listen to, not the port of the Cloudant server.


    Environment variables

    • PORT - the port number Envoy will listen on. When running in Bluemix, Envoy detects the Cloud Foundry port assigned to this app automatically. When running locally, you'll need to provide your own e.g. export PORT=8001
    • COUCH_HOST - The URL of the Cloudant service to connected to. Not required in Bluemix, as the attached Cloudant service is detected automatically. COUCH_HOST is required when running locally e.g. export COUCH_HOST=''
    • ENVOY_DATABASE_NAME - the name of the Cloudant database to use. Defaults to envoy
    • LOG_FORMAT - the type of logging to output. One of combined, common, dev, short, tiny, off. Defaults to off. (see
    • DEBUG - see debugging section
    • ENVOY_AUTH - which authentication plugin to use. One of default, couchdb_user
    • ENVOY_ACCESS - which access control plugin to use. One of default, id, meta
    • PRODUCTION - when set to 'true', disables the POST /_adduser endpoint
    • ENVOY_STATIC - when set to a full path, this directory is served out as static content from the Envoy express app.
    • ENVOY_USERS_DATABASE_NAME - the name of the database to store users when using the default auth plugin. Defaults to envoyusers.

    Using Envoy in your own app

    You can install Envoy and run it from your own Node.js application by install the module:

    npm install --save cloudant-envoy

    And then "require" it into your app:

        var envoy = require('cloudant-envoy')();

    where it will pick up its configuration from environment variables. You may also pass in object to Envoy on startup with your runtime options e.g.:

        var opts = {
          couchHost: '',
          databaseName: 'myenvoy',
          port: 9000,
          logFormat: 'dev',
          production: true
        var envoy = require('cloudant-envoy')(opts);'listening', function() {
          console.log('[OK]  Server is up');


    By default Envoy uses the Express Session session handler. This is an in-memory store and is only designed for test deployments. If you want to use any of the compatible session stores you may pass in a sessionHandler option at startup e.g.:

        var session = require('express-session');
        var RedisStore = require('connect-redis')(session);
        var path = require('path');
        var options = {
          url: 'redis://'
        var sessionHandler = session({
          store: new RedisStore(options),
          secret: 'oh my'
        var opts = {
          sessionHandler :sessionHandler,
          port: 9000
        var envoy = require('cloudant-envoy')(opts);

    Static files

    You can get Envoy to server out static files for you. Simply pass in a 'static' option or use the ENVOY_STATIC environment variable:

        var opts = {
          port: 9000,
          static: path.join(__dirname, './public')
        var envoy = require('cloudant-envoy')(opts);

    Custom routes

    You can supply your own custom routes to Envoy too:

    var express = require('express'),
      router = express.Router();
    // my custom route
    router.get('/myroute', function(req, res) {
    var opts = {
        databaseName: dbName,
        port: appEnv.port,
        logFormat: 'dev',
        production: true,
        static: path.join(__dirname, './public'),
        router: router
    var envoy = require('envoy')(opts);

    Custom middleware

    You can supply your own custom middleware to Envoy

    var myLogger = function (req, res, next) {
    var opts = {
        databaseName: dbName,
        port: appEnv.port,
        logFormat: 'dev',
        production: true,
        static: path.join(__dirname, './public'),
        router: router,
        middleware: [myLogger]
    var envoy = require('envoy')(opts);

    The middleware property expects an array of Express middleware functions.


    Debugging messages are controlled by the DEBUG environment variable. To see detailed debugging outlining the API calls being made between Envoy and Cloudant then set the DEBUG environment variable to cloudant,nano e.g

    export DEBUG=cloudant,nano
    node app.js


    DEBUG=cloudant,nano node app.js

    Envoy API end points

    Envoy supports a subset of the CouchDB API, sufficient to support replication. Additionally, a new Envoy-specific endpoint _add_user is there to facilitate testing; see below.

    Replication CRUD Utility
    GET /db/_all_docs GET /db/id POST /_adduser
    POST /db/_all_docs POST /db GET /_auth
    POST /db/_bulk_docs POST /db/id GET /_logout
    POST /db/_bulk_get PUT /db/id OPTIONS /*
    GET /db/_bulk_get DELETE /db/id
    GET /db/_changes GET / Search
    POST /db/_revs_diff GET /db POST /db/_find

    Envoy-specic APIs

    GET /_auth

    Allows a remote client to either check whether it is logged in or to establish a login session (credentials are passed using an HTTP Basic Authorization header):

    # not logged in - Envoy returns 403 response 
    > curl
    # log in - Envoy returns 200 response and saves cookie 
    > curl --user glynn:password

    POST /_auth

    Allows a remote client to either check whether it is logged in or to establish a login session (credentials are passed via a JSON-formatted HTTP POST):

    # not logged in - Envoy returns 403 response 
    > curl
    # log in - Envoy returns 200 response and saves cookie 
    > curl -H "Content-Type: application/json" -X POST -d '{"name":"glynn","password":"password"}'

    GET /_logout

    Allows a remote client to logout of a session.

    > curl

    POST /_adduser

    Allows the creation of new Envoy users. Supply a username and password parameter in a form-encoded post e.g.

    > curl -X POST -d 'username=rita&password=password'

    This API is only for evaluation purposes and should be disabled in production, by setting the PRODUCTION environment variable.


    At startup, Envoy can load in plugin modules to modify Envoy's behaviour. The plugin source code can be found in the lib/plugins directory, with two plugin types supported:

    • auth – controls how authentication occurs in Envoy.
    • access – controls how the ownership of a document is stored.

    Default auth plugin

    The default auth plugin uses the envoyusers database (by default) to store a database of users who are allowed to authenticate. Add a user to your envoyusers database with a document like this:

      "_id": "glynn",
      "_rev": "1-58bbb25716001c681febaccf6d48a9b8",
      "type": "user",
      "name": "glynn",
      "roles": [],
      "username": "glynn",
      "salt": "monkey",
      "password": "9ef16a44310564ecd1b6894c46d93c58281b07af"

    Where the password field is the sha1 of the "salt" field concatenated with the user's password e.g.

    > sha1('monkey' + 'password')

    Additional plugins can be selected using the ENVOY_AUTH environment variable. e.g. a value of "couchdb_user" will use the CouchDB "_users" database.

    Default access plugin

    The default access plugin stores the ownership of a document in the _id field of the document. If the owner of the document is glynn, then a document whose _id is test will actually get stored with an _id of




    is the sha1 hash of the username.

    The replication client doesn't see this additional meta data – it is transparently added and stripped out on its way through Envoy – but it allows Envoy to efficiently store many users data in the same database and retrieve the data each user owns efficiently.

    Additional plugins can be selected using the ENVOY_ACCESS environment variable.

    Frequently Asked Questions


    If you have an issue with Envoy, then please check to see if someone else has raised it already in our issues page. Please raise an issue for any problems you encounter and we'll see if we can help.



    npm i cloudant-envoy

    DownloadsWeekly Downloads






    Last publish


    • cloudant-sdks-automation
    • ricellis
    • emlaver
    • smithsz
    • glynnbird
    • mikerhodes
    • willholley
    • xpqz