This package has been deprecated

Author message:

couchxray is no longer supported

couchxray

couchxray is a command-line tool which outputs the CouchDB features used by a database and was inspired by the Python project xray by Will Holley. On a per database basis, couchxray outputs whether it is using:

• global MapReduce views and which reducers (built-in or bespoke) are being used.
• Lucene-based search indexes or Cloudant-specific Geospatial indexes.
• so called "Mango" (aka Cloudant Query) indexes backed by MapReduce or Lucene.
• partitions, and which form of per-partition secondary indexes are used.
• CouchDB features such as update/show/list/vdu functions.
• dbcopy, a now-deprecated Cloudant-only feature.

This information is useful when moving data between major versions of CouchDB. couchxray outputs a "compatibility" object which shows which versions of CouchDB are suitable for hosting the database e.g. if the database is using Lucene-based indexes then CouchDB 3+ would be required.

The tool outputs data in JSON for single-database requests or as a CSV when profiling all the databases in a CouchDB instance.

Installation

Installation requires Node.js/npm to be installed:

npm install -g couchxray

Usage

The URL containing the CouchDB service, including service credentials, should be supplied as a COUCH_URL environment variable or as the only command-line parameter.

e.g.

export COUCH_URL="https://username:[e-mail]"
couchxray --db cities

or

couchxray --url "https://username:[e-mail]" --db cities

or to use IBM IAM authentication

export COUCH_URL="https://host.cloudant.com"
export IAM_API_KEY="MY_IAM_API_KEY"
couchxray --db mydb

Use couchxray --help for assistance with command-line options.

Examining a single database

When running couchxray on a single database, it outputs a JSON object with its findings:

> couchxray --url http://admin:admin@localhost:5984/cities
{
  "databaseName": "cities",
  "partitioned": false,
  "numDocs": 23514,
  "numDeletions": 9,
  "numDesignDocs": 8,
  "totalDocs": 23531,
  "diskSize": 13688612,
  "billableSize": 2683923,
  "q": 16,
  "recommendedQDocs": 1,
  "recommendedQBytes": 1,
  "indexes": {
    "global": {
      "mapReduce": 7,
      "search": 1,
      "geo": 0,
      "mangoJSON": 0,
      "mangoText": 0
    },
    "partitioned": {
      "mapReduce": 0,
      "search": 0,
      "mangoJSON": 0,
      "mangoText": 0
    },
    "viewGroups": {
      "mapReduce": 8,
      "search": 1,
      "mango": 0
    },
    "updates": 0,
    "shows": 0,
    "lists": 0,
    "vdus": 0,
    "dbcopy": 0,
    "reducers": {
      "_count": 2,
      "_sum": 4,
      "_stats": 1,
      "_approx_count_distinct": 0,
      "none": 0,
      "custom": 0
    }
  },
  "compatibility": {
    "couchDB1": {
      "ok": false,
      "features": [
        "search"
      ]
    },
    "couchDB2": {
      "ok": false,
      "features": [
        "search"
      ]
    },
    "couchDB3": {
      "ok": true
    },
    "couchDB4": {
      "ok": true
    }
  }
}

Full scan mode

Couchxray is usually quick to run because it can proceed with only a handful of API calls per database. It passed the --scan flag, couchxray will spool through each document in the database and check that:

• the document ids are not greater than 512 bytes
• the MapReduce keys are not greater than 8000 bytes
• the MapReduce values are not greater than 64000 bytes

These are pertinent if a user is wishing to move to CouchDB 4 (or Cloudant on Transaction Engine) which imposes some size limits on document keys and index key/values.

Running in full scan mode will produce a scan object in the JSON output and will influence the compatibility.couchDB4 object.

  "compatibility": {
    ...
    "couchDB4": {
      "ok": false,
      "features": ["large document ids", "large index keys", "large index values"]
    }
  },
  "scan": {
    "docIdTooBig": true,
    "errors": {
      "_design/count/byCity": {
        "indexKeyTooBig": true,
        "indexValueTooBig": true
      }
    }
  }
}

Don't use --scan unless you are interested in a thorough examination of whether a database is CouchDB 4 compatible.

Examining all databases

If the URL points to the "top level" of the CouchDB service, all of the databases are examined and the output data is presented as a CSV file:

> couchxray http://admin:admin@localhost:5984
databaseName,partitioned,numDocs,numDeletions,numDesignDocs,totalDocs,diskSize,billableSize,q,recommendedQDocs,recommendedQBytes,indexes.global.mapReduce,indexes.global.search,indexes.global.geo,indexes.global.mangoJSON,indexes.global.mangoText,indexes.partitioned.mapReduce,indexes.partitioned.search,indexes.partitioned.mangoJSON,indexes.partitioned.mangoText,indexes.viewGroups.mapReduce,indexes.viewGroups.search,indexes.viewGroups.mango,indexes.updates,indexes.shows,indexes.lists,indexes.vdus,indexes.dbcopy,indexes.reducers._count,indexes.reducers._sum,indexes.reducers._stats,indexes.reducers._approx_count_distinct,indexes.reducers.none,indexes.reducers.custom,compatibility.couchDB1.ok,compatibility.couchDB2.ok,compatibility.couchDB3.ok,compatibility.couchDB4.ok,scan.docIdTooBig
aaa,false,3,1,1,5,388179,173,16,1,1,1,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,1,0,true,true,true,true,false
aa,false,2,0,0,2,2722750,2489192,16,1,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,true,true,true,true,false
a,false,22,3,2,27,4353575,250017,16,1,1,0,0,0,1,1,0,0,0,0,0,0,2,0,0,0,0,0,1,0,0,0,0,0,false,mango,true,true,true,false
_replicator,false,4,22,1,27,2297477,14447,16,1,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,true,true,true,true,false
a1,false,1,0,0,1,154619,39,16,1,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,true,true,true,true,false
ab,false,3,0,1,4,388147,387,16,1,1,1,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,1,0,0,0,0,true,true,true,true,false
abb,false,1,0,0,1,154507,20,16,1,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,true,true,true,true,false

What does the output data mean?

databaseName	The name of the database
partitioned	If the database is partitioned or not
numDocs	The number of regular documents
numDeletions	The number of deleted documents
numDesignDocs	The number of design documents
totalDocs	Total number of documents
diskSize	Disk space used
billableSize	Data size used in Cloudant billing
q	The number of shards
recommendedQDocs	Recommended value of q given the document count
recommendedQBytes	Recommended value of q given the data size
indexes.global.mapReduce	The number of global MapReduce indexes
indexes.global.search	The number of global Lucene-based search indexes
indexes.global.geo	The number of geo-spatial indexes (Cloudant only)
indexes.global.mangoJSON	The number of global Mango/MapReduce indexes
indexes.global.mangoText	The number of global Mango/Lucene indexes
indexes.partitioned.mapReduce	The number of partitioned MapReduce indexes
indexes.partitioned.search	The number of partitioned Lucene-based search indexes
indexes.partitioned.mangoJSON	The number of partitioned Mango/MapReduce indexes
indexes.partitioned.mangoText	The number of partitioned Mango/Lucene indexes
indexes.viewGroups.mapReduce	The number of MapReduce view groups (design docs with one or more MapReduce definitions)
indexes.viewGroups.search	The number of search view groups
indexes.viewGroups.mango	The number of Mango view groups
indexes.updates	Number of update functions
indexes.shows	Number of show functions
indexes.lists	Number of list functions
indexes.vdus	Number of vdu functions
indexes.dbcopy	Number of indexes defined to output their data to another database
indexes.reducers._count	Number of indexes using the _count reducer
indexes.reducers._sum	Number of indexes using the _sum reducer
indexes.reducers._stats	Number of indexes using the _stats reducer
indexes.reducers._approx_count_distinct	Number of indexes using the approximate count reducer
indexes.reducers.none	Number of indexes using no reducer
indexes.reducers.custom	Number of indexes using a user-defined reducer
compatibility.couchDB1.ok	Compatible with CouchDB 1.x
compatibility.couchDB1.features	Array of features that make the database incompatible
compatibility.couchDB2.ok	Compatible with CouchDB 2.x
compatibility.couchDB2.features	Array of features that make the database incompatible
compatibility.couchDB3.ok	Compatible with CouchDB 3.x
compatibility.couchDB3.features	Array of features that make the database incompatible
compatibility.couchDB4.ok	Compatible with CouchDB 4.x
compatibility.couchDB4.features	Array of features that make the database incompatible