Miss any of our Open RFC calls?Watch the recordings here! »

clussh

2.5.0 • Public • Published

Clussh

Build Status JavaScript Style Guide Clussh on npm

Stream task execution, through ssh, to one or many host, one or many times, in parallel or in series.

clussh

Features:

  • Distribute tasks upon many hosts
  • Stream task from stdin
  • Stream task outputs and progression stats to ldjson stream
  • Support scaling, parallel execution and retry, network failure and host unavailability

Installation

npm install -g clussh

This will install three command-line cli:

  • clussh - The main command line
  • clussh-board - A streamable clussh progress board
  • clussh-log - Pretty print clussh logs

Pre-requirement: NodeJS and its awesome universe

Usages & exemples

# Run "Hello $(hostname)" on the default worker ssh://yourusername@localhost 
clussh
 
# Run given command on default worker 
clussh --cmd 'echo "Hello world"'
clussh --cmd 'echo "Hello world"' --scale 10 # Teen times 
clussh --cmd 'echo "Hello world"' --scale 10 --concurrency 2 # Two in parallel 
 
# Run given bash script on default worker 
clussh ./install.sh
 
# Run given bash script on to agents (with a concurrency of 2 for agent-b) 
# (NB: for ssh cooking & baking reasons, the shell script must ends with an exit statement) 
clussh -w ssh://user@agent-a.local -w ssh://user@agent-b.local?concurrency=2 ./runme.sh
 
# Pipe task from line delimited json stream 
echo '
{ "id": "task-a", "cmd": "echo hello; sleep 1" }
{ "id": "task-b", "cmd": "echo hello; sleep 2", "scale": 10 }
{ "id": "task-c", "script": "./runme.sh", args: ["--option", "value"] }
{ "id": "task-d", "script": "./runme.sh", "worker": "ssh://only@on.host" }
' | clussh -w ssh://user@agent-a -w ssh://user@agent-b
 
# You can pipe infinite line delemited json stream (eg. event logs for a file wacher...) 
infinite-stream | clussh
 
# Print human readable logs 
clussh | clussh-log
 
# Print nice-looking progress board 
clussh | clussh-board
 
# Print human readable logs 
clussh | clussh-log
 
# Save logs for further exploration (and still display the board) 
clussh | tee output.log | clussh-board
tail -f output.log | clussh-log   # Another console... 
tail -f output.log | clussh-board # Another console... 
tail -f output.log | ndjson-filter 'd.type === "fail"' | prettyldjson # Another console... 

Command line usage and configuration

clussh [options] [script filepath]
 
Options:
  --help             Show help                                         [boolean]
  --version          Show version number                               [boolean]
  --retry, -r        How many retry for each task          [number] [default: 0]
  --timeout, -t      Task timeout in millisecond or parseable ms time (eg. 1h,
                     2d or `3 days`)                   [string] [default: "10d"]
  --worker, -w       Repeatable worker uri list
                                   [array] [default: "ssh://jponchon@localhost"]
  --concurrency, -c  Concurrency per worker                [number] [default: 1]
  --scale, -s        Default scale                         [number] [default: 1]
  --cmd              Command to execute                                 [string]
  --script           Script to execute (please ensure proper exit)      [string]
  --identity, -i     List ssh identity files to use        [array] [default: []]

Clussh can be configured with a .clusshrc file (see https://www.npmjs.com/package/rc)

{
  "worker": [
    "ssh://master@agent-a.local",
    "ssh://master@agent-b.local",
    "ssh://master@agent-b.local?concurrency=2"
  ],
  "concurrency": 4,
  "retry": 10
}

Above, one worker uri overide the default clussh configuration using url query string (?concurrency=2 for host agent-b.local). This will result in two workers for agent-b.local: one with the default concurrency (4) and another with a concurrency of 2.

Log message format

Clussh output line delemited json stream consumable by clussh-board or clussh-log or any other line delemited json tools (see prettyldjson for pretty print and ndjson-cli for map, reduce, filtering, etc.)

TODO: Fields documentation & messages types

API

const clussh = require('clussh')
 
const clusshStream = clussh({
  cmd: 'hello',
  worker: [ 'ssh://foo@bar.com', 'ssh://bar@foo.com' ],
  retry: 3,
  concurrency: 2
})
 
clusshStream.pipe(process.stdout)

License: MIT - Novadiscovery

Install

npm i clussh

DownloadsWeekly Downloads

21

Version

2.5.0

License

MIT

Unpacked Size

43.5 kB

Total Files

21

Last publish

Collaborators

  • avatar