Jam all of your logs into an event-stream


Jam all of your logs into an even-stream.

logjam turns your logfiles into a stream of events that you can access over HTTP. It hijacks your file system commands using FUSE and redirects them into an event-stream.

This means when you write files to a directory that's been logjammed, you're actually writing to a stream.


  • Ubuntu: sudo apt-get install libfuse-dev
  • CentOS / RedHat: yum install fuse-devel
  • OSX osxfuse
$ npm install --save logjam
$ jam --help
    jam tail [--host=localhost] [--port=3000] [--raw] [--html]
    jam up --logdir=<dirname> [--port=3000]
    Jam all of your logs into an event-stream
    jam up --logdir /path/to/your/logs
    jam tail --port 3000
# serve up some logs
$ jam up --logdir /path/to/logs --port 3000
# connect the logs
$ jam tail --host localhost --port 3000
$ curl localhost:3000/
$ curl localhost:3000?raw=true
$ curl localhost:3000?html=true

Open http://localhost:3000/

Jamming your friends up isn't cool, but jamming up your logs is. jam up hijacks a directory's file operations and puts them all into an event stream. It's sort of like a log pirate.

For example, let's say you have 3 jobs running on a server. Their logs will show up in 3 different places. For example:

  • /tmp/log/app1.log (sudo start jam_job name="app1.log" logdir="/tmp/log/")
  • /tmp/log/app2.log (sudo start jam_job name="app2.log" logdir="/tmp/log/")
  • /tmp/log/app3.log (sudo start jam_job name="app3.log" logdir="/tmp/log/")

One way to monitor all of the logs would be to use tail

$ tail -f /tmp/logs/app*.log

That's fine and all but it's a little annoying to keep track of. Especially if you even want to get into the business of dynamically adding jobs.

$ for i in `seq 1 100`
  sudo start jam_job name="app${i}.log" logdir="/tmp/logs/"

Not quite as much fun. So instead you can use jam up to redirect all of those logs into a stream!

So when you're apps/jobs write to any file in /tmp/logs, logjam is actually turning this into a stream. The file doesn't actually get written. It's a virtual file!

You can take that stream anywhere. And it's easy to access via curl or any other HTTP client.

Super simple, almost unneccessary. jam tail hooks up with a jam up stream and then writes any data back to stdout.

  • html (true/false): Flag for whether to send back HTML in stream.
  • raw (true/false): Flag for whether or not to use event-stream protocol.
  • pattern (glob): Pattern for matching a filename.

This is the main endpoint for the app. All of the logs will get streamed here.

There are a few options for formatting and determining which files you want to seein your logs. Since this is a one way street (you're not writing anything back to the server), it's setup as an event stream and is compatible with EventSource.

What's great about this is that you can also just CURL the endpoint and it will give you some nice looking output.

$ curl http://localhost:3000/
data: {"filename":"/hi.txt","content":"Hello!\n"}
data: {"filename":"/hi.txt","content":"My name is, Greg.\n"}
$ curl http://localhost:3000?html=true
data: {"filename":"/hi.txt","content":"Hello!\n"}
data: {"filename":"/hi.txt","content":"My name is, Greg.\n"}
data: {"filename":"/hi.txt","content":"<span style=\"color:#0AA\"> My favorite color is BLUE\n</span>"}
$ curl http://localhost:3000?raw=true
/hi.txt> Hello!
/hi.txt> My name is, Greg.
$ curl http://localhost:3000?pattern=*.txt
data: {"filename":"/hi.txt","content":"Hello!\n"}
$ node demo/color-spitter.js >> /tmp/logs/colors.yay

  • fusermount -u /path/to/stuff/
  • FUSE kind of sucks
  • echo "abcd" > /tmp/logdir/filename doesn't work

PROTIP: Jamming your friends up isn't cool