big-json-streamer
NodeJS: Parse big JSON files using streams.
Parse big files with the following format streaming to the output the choosen jsons:
{
"collection1" : [
{"key1": "value",
"key2": "value"
},
{"key1": "value",
"key2": "value"
},
...
],
"collection2" : [
{"key1": "value",
"key2": "value"
},
{"key1": "value",
"key2": "value"
},
...
],
}
Streams JSON one by one saying from which collection is the json node
/*** Return the string that will be pushed to the writable stream. If you want to ignore the json node just return null.*/jsonParser;
Example
var fs = ;var jsonParser = ; var file = processargv2;var output = file ? fs : processstdout;var input = fs; //Set readableStream from where read the jsonjsonParser;//Set writableStream from where write the jsonjsonParser;//Set callback to get each json foundjsonParser; //Set callback to do something when the big json finish.jsonParser; //Start the parsingjsonParser;