Nothing Prevents Misery

    json2lines
    TypeScript icon, indicating that this package has built-in type declarations

    1.0.1 • Public • Published

    json2lines

    This package is designed to take JSON files of any size with multiple sub-documents and produce a JSONL or JSON Lines file from the data. Take the example document:

    {
        "fruits": [
            {
                "name": "Apple",
                "color": "red"
            },
            {
                "name": "Orange",
                "color": "red"
            }
        ]
    }

    Run it through json2lines package:

    # npx json2lines --input input.json --output output.jsonl --filter fruits.*

    Will result in the following JSON Lines to the output.jsonl file:

    {"name":"Apple","color":"red"}
    {"name":"Orange","color":"orange"}
    

    Filtering

    In the example above, you can see we passed the filter foods.*. This tells us to extract the object in the foods array in the root object. For this functionality we make use of JSONStream. For more information on filters, you can visit the documentation here:

    https://github.com/dominictarr/JSONStream#jsonstreamparsepath

    Performance

    Memory wise the program stays near baseline for Node.JS, with a 2.6GB file with 374k entries running at <50MB in the Task Manager on a Windows 10 x64 machine. The file took roughly 6m40s on a AMD FX-8120 and Budget SSD doing both reads and writes.

    Install

    npm i json2lines

    DownloadsWeekly Downloads

    1

    Version

    1.0.1

    License

    MIT

    Unpacked Size

    10 kB

    Total Files

    12

    Last publish

    Collaborators

    • dudeami