Nonstop Pet Mewing

    gg

    0.1.3 • Public • Published

    gg: Generator General

    gg manages generator execution in a simple, declarative manner that allows for both serial and parallel execution of asynchronous requests.

    Platform Compatibility

    gg has the same requirements as co:

    When using node 0.11.x or greater, you must use the --harmony-generators flag or just --harmony to get access to generators.

    When using node 0.10.x and lower or browsers without generator support, you must use gnode and/or regenerator.

    When using node 0.8.x and lower or browsers without setImmediate, you must include a setImmediate polyfill.

    Installing

    via npm

    npm install gg
    

    from source

    git clone git://github.com/candu/gg.git
    cd gg
    npm install
    

    Usage

    These examples are adapted from test/gg.js, which you can run via

    npm test
    

    Waiting

    function* foo(value) {
      return value;
    }
    
    function* main() {
      var result;
      
      // use gg.wait() to wait for a single generator
      result = yield gg.wait(foo('test'));
      expect(result).to.equal('test');
      
      // use gg.waitAll() to wait for several generators (in parallel)
      result = yield gg.waitAll(foo('baz'), foo('frob'));
      expect(result).to.deep.equal(['baz', 'frob']);
      
      // you can also just pass an Array to gg.waitAll()
      result = yield gg.waitAll([foo('baz'), foo('frob')]);
      expect(result).to.deep.equal(['baz', 'frob']);
    
      return true;
    }
    
    gg.run(main(), function(err, result) {
      expect(result).to.be.true;
    });
    

    Error Handling

    function* bar(msg) {
      // just throw an error as you would normally
      throw new Error(msg);
    }
    
    function* main() {
      var threwException = false;
      try {
        var result = yield gg.wait(bar('zow'));
      } catch (err) {
        // exceptions are thrown into the waiting generator
        expect(err.message).to.equal('zow');
        threwException = true;
      }
      expect(threwException).to.be.true;
      yield gg.wait(bar('biff'));
    });
    
    gg.run(main(), function(err, result) {
      // gg.run() collects any uncaught exceptions
      expect(err.message).to.equal('biff');
    });
    

    Thunks and Promises

    var fs = require('fs'),
        Q = require('q');
    
    function sizeThunk(file) {
      return function(fn){
        fs.stat(file, function(err, stat){
          if (err) return fn(err);
          fn(null, stat.size);
        });
      }
    }
    
    function size(file, fn) {
      fs.stat(file, function(err, stat) {
        if (err) return fn(err);
        fn(null, stat.size);
      });
    }
    var sizePromise = Q.denodeify(size);
    
    function* main() {
      // you can also wait on thunks or promises
      var result = yield gg.waitAll(
        sizeThunk('test/gg.js'),
        sizePromise('test/gg.js'));
      expect(result[0]).to.equal(result[1]);
    }
    

    Dispatching

    In pseudocode, the core loop of gg looks like this:

    while (!main.isFinished()) {    // the same main passed to gg.run()
      dispatch();                   // see below
      runOneStep();                 // run everything we can
    }
    

    gg.onDispatch() allows you to attach your own functions to be called during that dispatch() step.

    Why would you want to do this? Suppose you're building a web page, and you need to fetch a bunch of users:

    function* navbar(req) {
      var user = yield gg.wait(fetchUser(uid1));
      // ...
    }
    
    function* feed(req) {
      var users = yield gg.waitAll([uid2, uid3, uid4].map(fetchUser));
      // ...
    }
    
    function* home(req) {
      var parts = yield gg.waitAll(navbar(req), feed(req)); 
      return combinePartsIntoPage(parts);
    }
    

    Ideally, we'd fetch uid1, ..., uid4 in one database query. dispatch() allows you to do exactly that, by providing a hook for batched operations to execute during the core loop:

    var Users = {
      _idsToFetch: {},
      _cache: {},
      gen: function*(id) {
        if (!(id in cache)) {
          this._idsToFetch[id] = true;
        }
        gg.wait();
        return this._cache[id];
      },
      dispatch: function(done) {
        var ids = Object.keys(this._idsToFetch);
        if (ids.length === 0) return done();
        DB.getUsers(ids, function(err, results) {
          if (err) return done(err);
          ids.forEach(function(id) {
            this._cache[id] = results[id];
          }.bind(this));
          done();
        }.bind(this));
      }
    };
    
    var fetchUser = Users.gen;
    gg.onDispatch(Users.dispatch);
    

    The control flow is as follows:

    • all fetchUser(uid) calls hit gg.wait() and pause;
    • on the next iteration of the core loop, Users.dispatch() is called;
    • Users.dispatch() performs a batched DB fetch, and stores the results in Users._cache;
    • the fetchUser(uid) calls resume, and read their return values from Users._cache.

    In this way, all fetchUser calls across all generators can be batched into a single DB request.

    This is extremely powerful! Now that all our user fetching is centralized, we can add caching, logging, etc. to Users without changing the fetchUser() callsites.

    For another example of batched access operations in gg, see DT in test/gg.js.

    Install

    npm i gg

    DownloadsWeekly Downloads

    2

    Version

    0.1.3

    License

    MIT

    Last publish

    Collaborators

    • qw3rtman