mongolian

Mongolian DeadBeef is an awesome Mongo DB node.js driver

Mongolian DeadBeef

Mongolian DeadBeef is an awesome Mongo DB node.js driver that attempts to closely approximate the mongodb shell.

Introduction

Mongolian DeadBeef and its documentation is super under construction! Go check out examples/mongolian_trainer.js and the rest of the source!

Unlike other MongoDB node.js drivers, Mongolian DeadBeef is built from the ground up for node.js, using node-buffalo for BSON/message serialization.

v0.1.15 Upgrade notes

0.1.15 uses node-buffalo instead of mongodb-native for serialization, this means a few incompatibilities:

The helper methods on ObjectId are removed, use the ObjectId constructor to parse hex strings
Code type is removed, use vanilla function instances instead
DBRef is not supported
Error messages may be different

Installation

DISCLAIMER: The API is experimental (but stabilizing). I will be adding, removing, and changing the API in the interest of a solid API. Use at your own risk

You can either clone the source and install with npm link, or install the latest published version from npm with npm install mongolian.

Motivation

Not a fan of existing asynchronous mongodb apis for node.js, I set out to write my own. To avoid completely reinventing the wheel, much of the Mongolian DeadBeef API is inspired by the mongodb shell.

High level principles:

Less is more
- Nothing is added without careful consideration
- Remove everything but the essentials
- Each refactor should remove as much unnecessary lines of code as possible
Fail early and often
- If I can easily detect a programmer error, an exception will be thrown

Notes:

mongodb is pretty simple, much of its functionality is defined as queries on special databases
- This allows for lots of code reuse
Avoid callbacks unless they are absolutely necessary

Basics

Most of the work in MongolianDeadBeef doesn't occur until a query is actually made. This means that simple operations are fast and synchronous. Currently there is one connection per server.

Examples

var Mongolian = require("mongolian")

// Create a server instance with default host and port
var server = new Mongolian

// Get database
var db = server.db("awesome_blog")

// Get some collections
var posts = db.collection("posts")
var comments = db.collection("comments")

// Insert some data
posts.insert({
    pageId: "hallo",
    title: "Hallo",
    created: new Date,
    body: "Welcome to my new blog!"
})

// Get a single document
posts.findOne({ pageId: "hallo" }, function(err, post) {
    ...
})

// Document cursors
posts.find().limit(5).sort({ created: 1 }).toArray(function (err, array) {
    // do something with the array
})
posts.find({ title: /^hal/ }).forEach(function (post) {
    // do something with a single post
}, function(err) {
    // handle errors/completion
})

Connections and Authentication

// Create a server with a specific host/port
var server = new Mongolian("mongo.example.com:12345")


// Authenticate a database
db.auth(username, password)


// Supported connection url format: [mongo://][username:password@]hostname[:port][/databasename]
// Use uri-encoding for special characters in the username/password/database name

// Database/auth shorthand (equivalent to calling db() and auth() on the resulting server)
var db = new Mongolian("mongo://username:password@mongo.example.com:12345/database")

// Connecting to replicasets:
var server = new Monglian(
    "server1.local",
    "server2.local",
    "server3.local:27018"
)

Logging

By default, Mongolian logs to console.log, but you can override this by specifying your own log object (any object that provides debug, info, warn, and error methods):

var server = new Mongolian({
    log: {
        debug: function(message) { ... },
        info: function(message) { ... },
        warn: function(message) { ... },
        error: function(message) { ... }
    }
})

var server = new Mongolian('server1.local', 'server2.local', {
    log: { ... }
})

BSON Data Types

Mongolian DeadBeef uses node-buffalo's BSON serialization code. Most BSON types map directly to JavaScript types, here are the ones that don't:

var Long =      require('mongolian').Long       // goog.math.Long - http://closure-library.googlecode.com/svn/docs/class_goog_math_Long.html
var ObjectId =  require('mongolian').ObjectId   // new ObjectId(byteBuffer or hexString)
var Timestamp = require('mongolian').Timestamp  // == Long
var DBRef =     require('mongolian').DBRef      // not supported yet

The Mongo shell doesn't support gridfs, so Mongolian DeadBeef provides a custom Stream-based GridFS implementation. It consists of two main classes, MongolianGridFS and MongolianGridFile. You can get a MongolianGridFS object from a database with the gridfs([gridfs name]) function.

// Get a GridFS from a database
var gridfs = db.gridfs() // name defaults to 'fs'

// Writing to GridFS consists of creating a GridFS file:
var file = gridfs.create({
    filename:"License",
    contentType:"text/plain"
})
// And getting writable Stream (see http://nodejs.org/docs/v0.4/api/streams.html#writable_Stream )
var stream = file.writeStream()

// You can then pipe a local file to that stream easily with:
fs.createReadStream('LICENSE').pipe(stream)

// Reading a file from GridFS is similar:
gridfs.findOne("License", function (err, file) {
    if (!err && file) {
        // Get the read stream:
        var stream = file.readStream()

        // You could then pipe the file out to a http response, for example:
        stream.pipe(httpResponse)
    }
})

// You can access metadata fields from the file object:
file.length // might be a Long
file.chunkSize
file.md5
file.filename
file.contentType // mime-type
file.uploadDate
// These two are optional and may not be defined:
file.metadata
file.aliases

// If you make any changes, save them:
file.save()

Mongodb Shell Command Support

Nearly all commands are identical in syntax to the mongodb shell. However, asynchronous commands that go to the server will have an optional node.js style callback parameter.

Currently most commands starting with get are named without the get. Some of the getters are implemented as values instead of functions.

Bold functions are supported
Italicized functions are supported with different syntax
Everything else is currently unsupported

There will likely be methods below that are never supported by Mongolian DeadBeef, since I'm targetting a slightly different use case.

Databases

From http://api.mongodb.org/js/1.8.1/symbols/src/shell_db.js.html

db.addUser(username, password[, readOnly=false][, callback])
db.auth(username, password)
db.cloneDatabase(fromhost)
db.commandHelp(name) returns the help for the command
db.copyDatabase(fromdb, todb, fromhost)
db.createCollection(name, { size : ..., capped : ..., max : ... } )
db.currentOp() displays the current operation in the db
db.dropDatabase() - see callback note below
db.eval(func[, arg1, arg2, ...][, callback]) run code server-side - see callback note below
db.getCollection(cname) implemented as db.collection(cname)
db.getCollectionNames() implemented as db.collectionNames(callback)
db.getLastError() - just returns the err msg string
db.getLastErrorObj() implemented as db.lastError(callback) - return full status object
db.getMongo() get the server connection object implemented as db.server
db.getMongo().setSlaveOk() allow this connection to read from the nonmaster member of a replica pair
db.getName() implemented as db.name
db.getPrevError() (deprecated?)
db.getProfilingStatus() - returns if profiling is on and slow threshold
db.getReplicationInfo()
db.getSiblingDB(name) get the db at the same server as this one
db.isMaster() check replica primary status
db.killOp(opid) kills the current operation in the db
db.listCommands() lists all the db commands
db.printCollectionStats()
db.printReplicationInfo()
db.printSlaveReplicationInfo()
db.printShardingStatus()
db.removeUser(username[, callback]) - see callback note below
db.repairDatabase()
db.resetError()
db.runCommand(cmdObj[, callback]) run a database command. if cmdObj is a string, turns it into { cmdObj : 1 }
db.serverStatus()
db.setProfilingLevel(level,) 0=off 1=slow 2=all
db.shutdownServer()
db.stats()
db.version() current version of the server

Collections

From http://api.mongodb.org/js/1.8.1/symbols/src/shell_collection.js.html

collection.find().help() - show DBCursor help
collection.count(callback)
collection.dataSize()
collection.distinct(key[, query], callback) - eg. collection.distinct( 'x' )
collection.drop([callback]) drop the collection - see callback note below
collection.dropIndex(name[, callback]) - see callback note below
collection.dropIndexes()
collection.ensureIndex(keypattern[,options][, callback]) - options is an object with these possible fields: name, unique, dropDups - see callback note below
collection.reIndex()
collection.find([query],[fields]) - query is an optional query filter. fields is optional set of fields to return. e.g. collection.find( {x:77} , {name:1, x:1} ) - returns a cursor object
collection.find(...).count()
collection.find(...).limit(n)
collection.find(...).skip(n)
collection.find(...).sort(...)
collection.findOne([query][callback])
collection.findAndModify( { update : ... , remove : bool [, query: {}, sort: {}, 'new': false] } ) ex: finds document with comment value 0, increase its 'count' field by 1, and return the updated document. collection.findAndModify( {query: {comment:'0'}, update : {"$inc":{"count":1}}, 'new': true}, function (err, doc) { console.log(doc) })
collection.getDB() get DB object associated with collection implemented as collection.db
collection.getIndexes() implemented as collection.indexes(callback)
collection.group( { key : ..., initial: ..., reduce : ...[, cond: ...] } )
collection.mapReduce( mapFunction , reduceFunction , [optional params][, callback])
collection.remove(query[, callback]) - see callback note below
collection.renameCollection( newName , [dropTarget] ) renames the collection.
collection.runCommand( name , [options][, callback]) runs a db command with the given name where the first param is the collection name
collection.save(obj[, callback]) - see callback note below
collection.stats()
collection.storageSize() - includes free space allocated to this collection
collection.totalIndexSize() - size in bytes of all the indexes
collection.totalSize() - storage allocated for all data and indexes
collection.update(query, object[, upsert_bool, multi_bool][, callback]) - see callback note below
collection.validate() - SLOW
collection.getShardVersion() - only for use with sharding

Cursors

From http://api.mongodb.org/js/1.8.1/symbols/src/shell_query.js.html

cursor.sort( {...} )
cursor.limit( n )
cursor.skip( n )
cursor.count() - total # of objects matching query, ignores skip,limit
cursor.size() - total # of objects cursor would return, honors skip,limit
cursor.explain([verbose])
cursor.hint(...)
cursor.showDiskLoc() - adds a $diskLoc field to each returned object
cursor.toArray(callback) - unique to Mongolian DeadBeef
cursor.forEach(func, callback) - calls func for each document, and callback upon completion or error
cursor.print() - output to console in full pretty format
cursor.map( func ) - map documents before they're returned in next, toArray, forEach
cursor.hasNext()
cursor.next([callback]) - returns the next document or null if there are no more

Callbacks

Callbacks take the standard node.js format: function(error, value)

Mongodb handles write operations (insert, update, save, drop, etc.) asynchronously. If you pass a callback into one of these methods, this is equivalent to immediately calling db.lastError(callback) on the same server/connection. Passing a null value will not send a getLastError command to the server.

Currently there is no way to specify the write concern on these inlined callbacks.

Todo

Connection pooling
Various utility methods
More unit tests
Documentation
Cleanup

Contributing

Try it out and send me feedback! That's the best help I could use right now. Unit tests are good, too.

License

Mongolian DeadBeef is open source software under the zlib license.