Norq

A loosely-ordered random-access queue for JSON documents implemented with Node and Redis.

Features

• Use Norq as a library, command line tool, preloaded in a repl, or via HTTP
• Optionally validate your data with json-schema.
• Flexible logging from HTTP with Winston

Installation

Install globally to use the norq executable:

$ npm install -g norq

Install locally to use norq as a dependency:

$ npm install norq

Configuration

Norq looks for a config file called norq.json in the working directory. The command $ norq init will generate a file with the default configuration. It looks like this:

{
  "redis": {
    "host": "127.0.0.1",
    "port": 6379,
    "db": 9,
    "options": {}
  },
  "http": {
    "port": 5150
  },
  "logs": [
    "Console",
    { "transport": "File",
      "options": { "filename": "logs/http.log" }}
  ],
  "model": {
    "default": {
      "name": "default"
    }
  }
}

You can define any number of queues in the model, each with its own json-schema.

// ...
"model": {
  "resources": {
    "name": "resources",
    "schema": {
      "type": "object",
      "properties": {
        // ...  
      }
    }
  }
}
// ...

Usage

// this creates norq.conf and log/
// you probably want to do it in a new directory
$ norq init 

// help
$ norq help
$ norq --help
$ norq -h

// version
$ norq version
$ norq --version
$ norq -v

Norq commands:

* push <queue> <json> 
* peek <queue>
* pop <queue>
* size <queue>
* range <queue> <start> <end>
* head <queue> <length>
* tail <queue> <length>
* page <queue> <page> <size>
* get <queue> <id>
* set <queue> <id> <json>
* remove <queue> <id>
* flush <queue|all>

Examples with default config:

$ norq push default '{ "some": "data" }'
$ norq peek default

Use Norq as a library

var norq = require('norq')
  , client = norq.createClient({...});

// all norq commands are methods that take a callback as their last argument
client.push('queue-name', {...}, function (err, result) {
  // do your thing 
});

Use Norq in a repl

$ norq repl
norq> 

// Your configuration will be loaded for you along with 
// norq and client, and a standard callback called norq.print
norq> norq.peek('queue', norq.print)
...

Use Norq from the command line (Unix/Linux/Mac)

$ norq help
$ norq version
$ norq push queue '{...}' 
$ norq peek queue 
etc.

Use Norq over HTTP

In your norq project directory start a HTTP server

$ norq http

Then access with curl or whatever client you want. Content-Type of HTTP PUT requests has to be application/json. For example:

$ curl -XPUT http://localhost:5150/queue -d '{...}' -H 'Content-Type: application/json'

HTTP API

Instance information

server    GET /
model     GET /model
schema    GET /<queue>/schema

Queue operations

push      PUT /<queue>
peek      GET /<queue>
pop       DEL /<queue>

List operations

head      GET /<queue>/head?limit=100     // limit defaults to 10
tail      GET /<queue>/tail?limit=100
range     GET /<queue>/25..49

Random access

set       PUT /<queue>/<id>
get       GET /<queue>/<id>
remove    DEL /<queue>/<id>

Running the Tests

Get the source from GitHub, install nodeunit and expresso globally, then in the root directory:

$ nodeunit test/test-norq.js test/test-logger.js test/test-config.js $ expresso test/test-http.js

Acknowledgements

Thanks to the open source software community for sharing your work. Norq is built with:

• Node.js and Redis (of course)
• node_redis
• node-uuid
• express
• winston
• json-schema
• ...

Support Norq

Norq is free software. I hope it serves you well. If you are using Norq in production, or feel that you derive value from it in some way, please consider sponsoring my work with a donation via PayPal.

License

Copyright (c) 2011 Christian Smith <[e-mail]>

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE.