populator

populator: for application-level JOINs in a node.js-NoSQL world

The trend towards NoSQL datastores has led to an introduction of complexity in the application layer, particularly with patterns that are common in the SQL world. JOINs are something which generally run against the methodologies of many modern datastores but are a necessary evil (through real-time JOINs or denormalization of some sort) and populator has been created to address that need.

With populator, you can define a series of fields on an object (or objects) and functions used to asynchronously populate those fields with actual data (instead of ids). When populate is called, all of the replacement functions will be ran in parallel and loaded onto the objects provided to populator

To use populator:

1. provide an object (or objects) to populator which you would like to populate data onto:

// load populator
var populator = require('./populator')
 
// some objects we want to get data for
var messages = [
  {text: "Hello", user: 'user-a'}
, {text: "Hello to you, good sir", user: 'user-b', votes:{up: ['user-a']}}
, {text: "Thanks!", user: 'user-a'}
]
 
// create a populator instance for the objects
var p = populator(messages)

1. provide a set of replacement rules

// pretend these are defined somewhere else (like a database)
var users = {
    'user-a': {id: 'user-a', name: "Jeremy"},
    'user-b': {id: 'user-b', name: "Jean"}
}
 
// take a list of user ids and retrieve from the hashmap defined above
function getUsersByIds(ids, callback) {
  var returnUsers = []
  for (var i = 0; i < ids.length; i += 1) {
    if (users[ids[i]]) returnUsers.push(users[ids[i]])
  }
  callback(null, returnUsers)
}
 
// we want to replace the votes (up and down) and users fields on messages with user objects
p.replace(['user', 'votes.up', 'votes.down'], getUsersByIds, function (id, obj) {
    return obj.id === id
})

1. populate the data and provide a callback

// populate the specified fields
p.populate(function (err, data) {
  console.log(JSON.stringify(data))
})

Your returned objects should now look like:

[
  {
    "text": "Hello",
    "user": {"id": "user-a", "name": "Jeremy"}
  },
  {
    "text": "Hello to you, good sir",
    "user": {"id": "user-b", "name": "Jean"},
    "votes": {
      "up":[
        {"id": "user-a", "name": "Jeremy"}
      ]
    }
  },
  {
    "text": "Thanks!",
    "user": {"id": "user-a", "name": "Jeremy"}
  }
]

And you know how to use populator!

Contributing

Questions, comments, bug reports, and pull requests are all welcome. Submit them at the project on GitHub.

Bug reports that include steps-to-reproduce (including code) are the best. Even better, make them in the form of pull requests that update the test suite. Thanks!

Author

Jeremy Stanley supported by The Obvious Corporation.

License

Copyright 2012 The Obvious Corporation.

Licensed under the Apache License, Version 2.0. See the top-level file LICENSE.txt and (http://www.apache.org/licenses/LICENSE-2.0).