NASA Planetary Mission


    1.2.1 • Public • Published

    NPM Version NPM Downloads Test Coverage Coverage

    Streaming XML->JS parser that extracts selected components from XML documents one-by-ony as they are found and pipes them downstream. This method ensures a low memory footprint as neither the XML file nor the corresponding JSON have to be fully loaded into memory at any point in time.


    // returns a stream

    The selector can be any of the following:

    • String => a selected tag has to match the supplied selector exactly
    • RegExp => the fullpath (space delimited) has to pass the regex
    • Number => pipes down all element (with any child objects) that reside at a particular depth inside the document.
    • Function => element selected when function returns true. The function will be called with the following object:
      • tag : name of the tag
      • path : an array of the tags of the full path
      • attr : an object with the attributes of the element
      • depth : the depth of the object within the xml document
    • Array => Each element is a criteria - will be evaluated as or

    Each selected element will include all children as a json object. Selecting depth 0 for example will just result in the whole document piped down as one json record. Each object (+ children) will contain a property named __attr__ that contains any attributes defined in the XML document. By default the __attr__ object is non-enumerable, meaning that its not visible to screen or Object keys but can still be accessable.

    The piped record will include the following properties: tag, path,depth and value with the element+children placed in the value property.

    The following options can be defined

    • arrayNotation : default false - enforces that all children are within an array, even when there is only one child node
    • coerce : default false - coerces numbers and booleans if turned on, both in text and attribute values. If coerce is an object then custom coercion methods can be defined for each element name / attribute name
    • textObject : default false - Enforces that each text element is placed under .text property. The default behavior is: if text object the only child of a node:|{text:'xxxx'} is simplified to `'xxxx'.
    • valuesOnly : pipes down just the matched record (i.e. the .value of the default record)
    • showAttr : default false - show the __att__ object instead of hiding it behind the non-enumerable definition
    • mergeAttr : default false - merges the attributes with the children objects (risking collisions when attribute name == children tag name)

    Simple example

    Given a sample file.xml


    Running the following:

        transform: (d,e,cb) => return console.log(d) && cb();

    will console.log the following records one by one:

      tag: 'item',
      path: ['xml','item'],
      depth: 1,
      value: {
        description: 'First',
        price: 4.32
      tag: 'item',
      path: ['xml','item'],
      depth: 1,
      value: {
      description: 'Second',
        price: 4.73

    Using .pipe(xmler('item'),{coerce:true}) will extract the same records. Using .pipe(xmler('price',{coerce:true})) will return an array where the values will be 4.32 and 4.37 and the path will be ['xml','item','price']

    Real life example

    This example streams the full Open Streetmap XML for North America into JSON records:

    var request = require('request');
    var bz2 = require('unbzip2-stream');
    var etl = require('etl');
    var xmler = require('xmler');
    var fs = require('fs');
    // Keep track of linecount and report every second
    var count = 0;
    setInterval( () => console.log(count),1000);
      .pipe( => {
        return JSON.stringify(d);

    Same example with the records piped to mongo (bulk = 100 records at a time and max concurrent connections = 5)

    var mongo = require('mongodb');
    var collection = mongo.connect('mongodb://localhost:27017/osm')
          .then(db => db.collection('osm'));
      .pipe( => {
        d._id =;
        return d;
      .pipe(etl.mongo.update(collection,['_id'],{upsert: true, concurrency: 5}));




    npm i xmler

    DownloadsWeekly Downloads






    Unpacked Size

    153 kB

    Total Files


    Last publish


    • zjonsson