read-json-stream

    1.1.0 • Public • Published

    read-json-stream

    Node library for reading and parsing JSON files as streams.

    Why?

    Node does not like parsing huge JSON files. In fact, it completely crashes! One alternative is to stream the file and parse it as it comes in. This not only removes any limits on how large a JSON file you can open & parse, it also is better for memory management.

    What?

    read-json-stream allows you to easily load and parse JSON files, large or small. I do not provide hooks for doing other transformations during the stream. This lib is simply an alternative to using fs.loadFile and then JSON.parse. The one extra feature this does provide is the option of registering a progress callback which is run while streaming and is passed an integer from 1 to 100, showing the percentage complete.

    How?

    npm install read-json-stream
    

    First, import/require the lib:

    import ReadJSONStream from 'read-json-stream';
    

    Or, if you are using older syntax:

    var ReadJSONStream = require('read-json-stream').default;
    

    Then load a JSON file.

    ReadJSONStream(filePath)  // filePath is a file path string e.g. 'data/myData.json' or 'path.join('data', 'myData.json');
      .done((err, data) => {
        if(err) {
          // handle error
        } else {
          // do something with the freshly-parsed data!
        }
      });
    

    To keep track of progress, register a progress callback.

    ReadJSONStream(filePath)
      .progress(p => {
        console.log(`${p}% done so far!`);
      }),
      .done((err, data) => {
        if(err) {
          // handle error
        } else {
          // do something with the freshly-parsed data!
        }
      });
    

    If you are not used to ES2015 syntax, you can write it as:

    ReadJSONStream(filePath)
      .progress(function(p) {
        console.log(p + '% done so far!');
      }),
      .done(function(err, data) {
        if(err) {
          // handle error
        } else {
          // do something with the freshly-parsed data!
        }
      });
    

    npm Scripts

    Run the tests:

    npm test
    

    To re-compile the source code:

    npm run build
    

    To watch the src directory and automatically recompile on changes:

    npm run watch
    

    Contributions

    Contributions are welcome! If you have any issues and/or contributions you would like to make, feel free to file an issue and/or issue a pull reuqest.

    License

    Apache License Version 2.0

    Copyright (c) 2016 by Ryan Burgett.

    Install

    npm i read-json-stream

    DownloadsWeekly Downloads

    20

    Version

    1.1.0

    License

    Apache-2.0

    Last publish

    Collaborators

    • rburgett