npm-metadata-raw

Retrieve the metadata for every package on NPM and store it in a level* database

npm-metadata-raw

Retrieve the metadata for every package on NPM and store it in a level* database. Once you've acquired the data, you can then run your own manual queries over it very quickly and without hitting the registry.

The initial sync takes a long time, but this module only downloads the most recent changes not already stored - so after the first run it should be relatively speedy.

Takes a level* database db and returns a modified one for querying. Once synced, each package is stored in metadb by their name - so you could do this to get the metdata for the disc module:

var db = require('level')(__dirname + '/npm-meta')
var metadata = require('npm-metadata-raw')
var sublevel = require('level-sublevel')
 
var metadb = metadata(sublevel(db))
 
metadb.sync(function(err) {
  if (err) throw err
  metadb.get('disc', function(errmeta) {
    if (err) throw err
    console.log(JSON.parse(meta).name) // "disc" 
  })
})

Sync with the NPM registry, optionally passing the finished callback for when the process is complete. This must be done at least once before you can access any data.

When syncing, an initial pass is done over a complete list of packages in the registry. The found event is emitted once for each module in the list.

After the first pass, a second pass will retrieve and store the metadata for each out of date package. When retrieved, the module data is emitted under the data event.

Emitted when the whole process is complete.

MIT. See LICENSE.md for details.