Alinex Exec: Readme
This module may be used to call external commands. It is an extended
wrapper arround the core process.spawn
command or runs them on a remote machine
through SSH. It's benefits are:
- fully configurable
- automatic error control
- automatic retry in case of error
- prioritized calls with load control
- supports remote execution
It is one of the modules of the Alinex Namespace following the code standards defined in the General Docs.
Read the complete documentation under https://alinex.github.io/node-exec.
Install
The easiest way is to let npm add the module directly to your modules (from within you node modules directory):
npm install alinex-exec --save
And update it to the latest version later:
npm update alinex-exec --save
Always have a look at the latest changes.
Usage
You may connect to the process using a callback method on the run()
call or
use the events.
First you have to load the class package.
Exec = require 'alinex-exec'Execinit # this will setup the config class
Run an external command
Now you may setup an external process like:
proc =cmd: 'date'
You may also change the configuration afterwards like:
proc.setup.cmd = 'date'
To run this simple process call the run-method:
procrun# work with the results
After the process has completed it's task the callback will get only an error
obect and the process instance which you don't need. You may access all details
through the proc.result
and proc.process
objects.
You may also run the same command but only after the first run has finished.
Simplified run
You may call all of this directly using:
Execruncmd: 'date'args: '--iso-8601'# work with the results within the process instance
Remote Execution
To execute the command on a remote machine you only have to specify the remote machine. This can be done in various ways like also described in {@link alinex-ssh}.
Execrunremote:server:host: '65.25.98.25'port: 22username: 'root'#passphrase: 'mypass'privateKey: require'fs'readFileSync '/home/alex/.ssh/id_rsa'#localHostname: "Localost"#localUsername: "LocalUser"#readyTimeout: 20000keepaliveInterval: 1000#debug: trueretry:times: 3intervall: 200cmd: 'date'# work with the results within the process instance
This may also be called with a list of alternative server
connections.
Configured
But to make it easier you can use a reference to the server configuration under
/ssh/server
in your {@link alinex-config}.
Execrunremote:server: 'server2'retry:times: 3intervall: 200cmd: 'date'# work with the results within the process instance
The retry part can also be kept away to use the defaults (from config).
The following is a short form, only possible if no special retry times are used:
Execrunremote:group: 'appcluster'cmd: 'date'# work with the results within the process instance
Alternatively you can give the group as an array of server names or configurations:
Execrunremote:group: 'node1''node2''node3'cmd: 'date'# work with the results within the process instance
And also the short version is possible which will first try to use the given name as group else as server:
Execrunremote: 'appcluster'cmd: 'date'# work with the results within the process instance
Cluster/Group
You can also reference a group of servers defined under /ssh/group
in your
{@link alinex-config}. If you do so the command will be execute on the host with
the lowest load if load based executing through priority is set.
Ending
If you used remote executions, there may be some open server connections in the pool which prevent your program from ending. To end them call:
Execclose
Best Practice in Module
If you use this module in your app call the Exec.setup()
method before initialization
of the config module:
config = require 'alinex-config'Exec = require 'alinex-exec'schema = require './configSchema':# setup module configs firstasynceach Execmodsetup cbreturn cb err if err# add schema for app's configurationconfigsetSchema '/app'schema# extend module search pathconfigregister 'app'fspathdirname __dirnamecb:configinitreturn cb err if err@conf = configget '/app'cb
Configuration
To configure this to your needs, please make a new configuration file for /exec
context. To do so you may copy the base settings from src/config/exec.yml
into
var/local/config/exec.yml
and change it's values or put it into your applications
configuration directory.
Like supported by Config you only have to write the settings which differ from the defaults.
The configuration contains the following three parts:
Retry if failed
In this part you define the number of retries and timeouts used for rechecking in all parts which prevent the execution to start or run successfully.
exec:retry:# connection retriesconnect:# number of attemptstimes: 3# time to sleep till next tryinterval: 1s# check for host vital signsvital:# time to recheck host vital signsinterval: 5s# maximum load% per CPU core per second in usage to startstartload: 80%# too much processes opened on systemulimit:# time to sleep till next tryinterval: 1s# a queue will be created if host is overloadedqueue:# specify the time after that a retry for queued executions should runinterval: 3s# process failed retry checkerror:# number of attemptstimes: 3# time to sleep till next tryinterval: 3s
All interval
settings can be set as a time range in milliseconds or with the
unit appended. Use something like '1.5s' (possible units: ms, s, m, h, d).
Priority Levels
This defines the possible priority levels to be used. These are used to prioritize some tasks using nice levels and also run them only if the load and cpu usage on the machine is below the defined level for each priorities. Jobs are held back till the machine load allows them.
The following configuration shows the default.
exec:priority:# specify the default prioritydefault: medium# specify the possible priorities with their checkslevel:anytime:maxCpu: 20%maxLoad: 20%nice: 19low:maxCpu: 40%maxLoad: 60%nice: 10medium:maxCpu: 60%maxLoad: 100%nice: 5high:maxCpu: 90%maxLoad: 150%immediately:nice: -20
You may add other values or remove some of them to get your very own set of priorities:
exec:priority:# specify the possible priorities with their checkslevel:low: nullmedium: nullhigh: null1:maxCpu: 40%maxLoad: 100%nice: 152:maxCpu: 50%maxLoad: 120%nice: 53:maxCpu: 60%maxLoad: 140%nice: 0
Remote Servers
This section shows you how to setup remote servers:
# Define remote hosts or host pools to be used through ssh.ssh:server:# virtual name for serverserver1:# hostname or ip to connect tohost: localhost# connection portport: ssh# user to login asusername: alex# (optionally) login using password#password: 'geheim'# (optionally) private key for loginprivateKey: <<<file:///home/alex/.ssh/id_rsa>>># the passphrase for the key (if necessary)#passphrase: '.....'# the host used for hostbased authentication#localHostname: '...'# the username for host based authentcation#localUsername: '...'# time for sending keepalive packetskeepaliveIntervall: 1s# the maximum time for the ssh handshakereadyTimeout: 20s# maximum load to start here per each cpustartload: 2.4# maximum number of parallel sessions on this connection#maxSessions: 4# debug also server communicationdebug: falseserver2:host: 127.0.0.1port: sshusername: alexpassword: dontknowmaxConnections: 5exec:group:testpool:- server1- server2
As seen above there are two servers defined to access as server1
and server2
both are the localhost for test purpose.
Setup Execution
To specify your executable you give an object with the following settings.
Execution
The basic settings needed are which command to start and with which parameter. Keep in mind that here no shell execution is possible so don't use bash basics without specifying 'sh' or 'bash' as the command.
- cmd - (string) giving the command to call with path if needed
- args - (array) list of arguments to use in the given order
You can give both separately but you may also give attributes in the command string. If you do so the arguments will be extracted automatically so you need to use proper quoting.
Remote
To run on a remote machine you only need the virtual name which has to correspond to the configuration (see above).
- remote - (string) reference to configuration
Environment
- cwd - (absolute path) the current working directory
- uid - (integer) the userid under which to run
- gid - (integer) the group id under which to run
- env - environment object for key-value-pairs
If you want to use different user and/or group id you have to be root first.
Priority
The priority defines which command to run first in case of multiple commands.
- priority - (string) name of configured priority to use
By default the following priorities are possible: anytime, low, medium, high, immediately.
Failure Management
- timeout - time in milliseconds after that the process will be killed
- check - name of the check with
- 'true' as value if no args
- args - (array) list of arguments for the check if possible
- retry - (boolean) should retry on this failure
- retry - specify if not use the default
- times - (integer) maximum number of retries
- interval - (milliseconds)
The following checks may be used:
noExitCode
- check that the exit code is 0exitCode code
- check that the exit code is in allowed listnoStderr
- check that STDERR is emptynoStdout
- check that STDOUT is emptystderr
- check that STDERR isn't emptystdout
- check that STDOUT isn't emptymatchStdout ok, report
- check that the givenok
RegExp succeeds, if not output the result of the report RegExpmatchStderr ok, report
- check that the givenok
RegExp succeeds, if not output the result of the report RegExpnotMatchStderr fail
- check that thefail
RegExp don't matchstdoutLines min, max
- check that the output has the correct number of lines
This are only the general checks. There may be more command specific matches which you may use.
As an example you may use:
Execruncmd: 'ffmpeg'args: '001.wma''001.mp3'priority: mediumcheck:noExitCode: truematchStdErr:args: /Succeeded//Failed: /
Events
The following events are possible to use:
wait
- (ms) wait some milliseconds before checking againstdout
- (line) output linesstderr
- (line) error output linesend
- (exitCode) if execution stopped
To use events to immediately see what goes on you will call the execution like:
Execinitreturn cb err if errproc =cmd: 'ffmpeg'args: '001.wma''001.mp3'priority: mediumcheck:noExitCode: truematchStdErr:args: /Succeeded//Failed: /procon 'stdout' consolelog lineprocon 'stderr' consoleerror lineprocon 'done' consolelog "Exit with code "procrunreturn cb err if err
This will always printout a line immediately if a full line comes from the command. Empty lines will be ignored.
Access Results
Process
With this attribute you may get the following information:
- exec.process.host - remote host identifier
- exec.process.pid - process id
- exec.process.start - start time
- exec.process.end - end time
- exec.process.error - system error
Tries
If multiple tries are done you may access the information of each try:
- exec.tries[n].process - first calls
- exec.tries[n].result . first calls
- exec.process - last run
- exec.result - last run
Results
The result code and maybe error of the defined checks may be red directly:
- exec.result.code
- exec.result.error
But to get one of the outputs better use the functions:
- exec.stdout()
- exec.stderr()
They will give you a complete text out of the line data which lists all channels in the proper order. If order of messages matters better directly access them:
- exec.result.lines
- 0 - stdin
- 1 - stdout
- 2 - stderr
You may also use the above methods on previous tries:
- exec.stdout(exec.tries[0].result)
- exec.stderr(exec.tries[0].result)
License
(C) Copyright 2015-2016 Alexander Schilling
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.