Normal Polygamous Marriage

    mongoimporter

    1.0.4 • Public • Published

    mongoimporter

    Import multiple files into mongo DB. The existing mongoimport binary is great, but this is a flexible library built for programmatic use. Currently supported filetypes: csv, json.

    This library can import multiple files or an entire directory at once. If importing the entire directory, it will only import files in that directory (not subdirectories).

    The destination collection names can either be explicitly stated or inferred from each filename using the useFilename option. See test/index.js for examples.

    Examples

    1. Importing multiple files (mixed filetypes, csv or json) into mongodb:
    /* Requires */
    const MongoImporter = require('mongoimporter')
     
    /* Instantiate the mongoImporter */
    const mongoImporter = new MongoImporter({
        dbName: 'test'
        // can also specify dbHost and/or dbPort options here. If none specified,
        // mongo defaults will be used; 127.0.0.1:27017
    })
     
    mongoImporter
    .connect({ // Connect to the db first (provide the connection options)
        "auth": { // Auth, if any
            "user": "test1",
            "password": "test1"
        },
        // Optional: if mongodb requires SSL connection...must provide appropriate options
        // (Here, simply supply the paths, not the buffers; this makes it easy for you to stash
        // these options in a config.json file)
        "ssl": true,
        "sslKey": "~/.ssl/mongoClient.key",
        "sslCert": "~/.ssl/mongoClient.crt",
        "sslCA": "/etc/ssl/mongo_dev/mongod.pem"
    })
    .then(() => {
        var files = [
            './datasets/example1.csv',
            './datasets/example2.csv',
            './datasets/example3.json',
            './datasets/example4.json',
        ]
        return mongoImporter.importFiles(files, { // import files
            csvDelimiter: ',', // Delimiter to use for csv files
            collectionName: { // Set collectionName to an object
                useFilename: true // The filename will be used as the collectionName
            },
            headerline: true // Use the headerline as fields
        })
    })
    .then((filesImported) => {
        console.log('filesImported', filesImported) // Count of files imported
        mongoImporter.disconnect(); // disconnect from the db
    })
    1. Import a directory into mongodb. Suppose the directory "datasets" consists of the files from example 1. Then the code would be simplified to this:
    /* Requires */
    const MongoImporter = require('mongoimporter')
     
    /* Instantiate the mongoImporter */
    const mongoImporter = new MongoImporter({
        dbName: 'test'
    })
     
    var dir = 'datasets';
     
    mongoImporter
    .connect({ // Connect to the db first (provide the connection options)
        "auth": { // Auth, if any
            "user": "test1",
            "password": "test1"
        }
    })
    .then(() => {
        return mongoImporter.importDir(dir, { // import directory
            csvDelimiter: ',', // Delimiter to use for csv files
            collectionName: dir, // Setting the collectionName to directory name explicitly (but it can be any string you want)
                    // You can also set this to { useDirectoryName: true } to use the directoryName as the collectionName.
            headerline: true // Use the headerline as fields
        })    
    })
    .then((resp) => {
        console.log('dirImported', resp) // If we got here, we succeeded; resp is count of files imported
    })
    .catch((err) => {
        console.log('dirImport failed', err) // If we got here, we failed...read the err.
    })
    .finally(() => mongoImporter.disconnect());

    (This is still in it's early stage...Tests pass and this code works, but patience requested for more advanced tasks.)

    Install

    npm i mongoimporter

    DownloadsWeekly Downloads

    5

    Version

    1.0.4

    License

    MIT

    Unpacked Size

    18.6 kB

    Total Files

    12

    Last publish

    Collaborators

    • richardeschloss