Update: you should use csv-parser instead, it has the same API as this but is faster
Buffer in node or
Uint8Array in the browser (thanks to bops). Whereas most CSV parsers parse
String data, this library never converts binary data into non-binary data. It's fast because it never creates Numbers, Strings, Arrays or Objects -- only binary representations of the line and cell values in the CSV, meaning the JS VM spends less time doing things like decoding UTF8 strings and going back and forth between C++ and JS.
By default it will only split lines, but you can use the provided
.cell methods to parse the cells and cell values.
Parses a 55 million line, 5.18GB CSV in a little over 1 minute.
See a demo running in the browser on RequireBin:
You can also load any CSV on the internet via querystring, e.g.:
Huge CSVs might be slow to render because of the terminal renderer used in the demo.
You can use it two ways: programmatically in Node programs, or from the command line.
var binaryCSV = require'binary-csv'var parser = binaryCSV
parser is a duplex stream -- you can pipe data to it and it will emit a buffer for each line in the CSV
separator: ','newline: '\n'detectNewlines: truejson: false
json is truthy then the parser stream will emit fully decoded JSON objects representing each row of the csv (combined with the header row)
Parses cells from a line buffer. Returns an array of cell buffers.
var cells = parserline'hello,world'// returns equivalent of [new Buffer('hello'), new Buffer('world')]
Parses a single cell buffer, returns the unescaped data in a buffer.
var cell = parsercell'"this is a ""escaped"" csv cell value"'// returns equivalent of new Buffer('this is a "escaped" csv cell value")
test/test.js for more examples.
To use on the command line install it globally:
$ npm install binary-csv -g
This should add the
bcsv command to your
Then, you either pipe data into it or give it a filename:
# pipe data in$ cat some_data.csv | bcsv# pass a filename$ bcsv some_data.csv# tell bcsv to read from + wait on stdin$ bcsv -
$ npm install$ npm test