node based job execution engine


node.js cloud engine Apart from all the buzz words, key aspect of this software is to specify set of tasks that are interdependent in a config file (similar to DAG input on condor) and run them.


  • Task - is the smallest unit of execution (preferrable should be an idempotent task)
  • Job - collection of tasks with dependencies (acyclic)


  • job execution engine
  • Task could be a javascript unit or executable that would be exec'ed
  • Command line to check the status of jobs, tasks and current running processes
  • Command line Submit of jobs
  • instantiate jobs cron style
  • Good unit test coverage
  • Failure of tasks to be handled
  • Job load balancing among cloudd hosts
  • Persistence of state
  • clustering among cloudd hosts


  • Lets see how to define a Hello My World Job
    1. Job Hello My World comprise of three tasks
    • Hello Task
    • My Task
    • World Task
    1. Dependencies
    • My would be run only after Hello
    • World would be run only after My
  • Sample yaml config file would look like (see in examples/hellomyworld.yaml)
name: Hello World
  Sample 'hello my world' job
    executable: echo "my"
    executable: echo "hello"
    executable: echo "world"
     parent: [my]
     child: [world]
     parent: [hello]
     child: [my]

Have two terminals, one for the server and another one for submitting the job

# terminal 1
cloudd server

# terminal 2
cloudd submit HW examples/hellomyworld.yaml

To run the server

cloudd server

Command line utilities

# to show current running tasks
cloudd ps

# to show total tasks that are ready to run (ordered list)
cloudd tasks

# to show total jobs in the system (non-completed)
cloudd jobs

# to submit a new job
cloudd submit job-id sample-job-file.yaml

# to submit a cron job (say every 5 seconds)
cloudd cron "*/5 * * * * *" job-id sample-job-file.yaml
  • id's for the job and task cannot have '#'
  • input is a field that would be passed to task