nodejs hook for htcondor
node-htcondor contains various nodejs hooks for htcondor.
npm install htcondor
You can submit a condor job and watch for joblog.
var htcondor = require'htcondor';//create condor submit objectvar submit_options =universe: "vanilla"executable: "test.sh"arguments: "hello"notification: "never"//transfer_output_files: 'bogus',requirements: '(Arch == "INTEL") || (Arch == "X86_64") && (machine == "some_ce")'shouldtransferfiles: "yes"when_to_transfer_output: "ON_EXIT"output: "stdout.txt"error: "stderr.txt"queue: 1;//submit to start the jobhtcondorsubmitsubmit_optionsthenconsole.log"Submitted";//you can dump the condor submit property for your new job//console.dir(job.props);//job.id contains cluster/proc ids//console.dir(job.id);//you can *watch* job logjoboneventswitcheventMyType//normal status type events (just display content)case "SubmitEvent":case "ExecuteEvent":case "JobImageSizeEvent":console.logeventMyType;break;//critical eventscase "ShadowExceptionEvent":console.logeventMyType;consoledirevent;//I now stop the watcher (ends this submission)jobunwatch;break;//job ended normallycase "JobTerminatedEvent":console.logeventMyType;consoledirevent;//Do something based on the ReturnValue (resubmit, submit different job, etc..)console.log"return value:"+eventReturnValue;//If you submitted queue != 1, then you can look for Event.Proc to get the Process ID (0, 1, etc..)//If you are happy, just unwatchjobunwatch;break;default:console.logeventMyType;console.log"unknown event type.. ignoring for now";;;
You can do the usual condor stuff.
jobremoveiferr throw err;console.log"successfully removed job";;
jobholdiferr throw err;console.log"successfully held job";;
jobreleaseiferr throw err;console.log"successfully released job";;
You can also call via htcondor object
htcondorremovejobiferr throw err;console.log"successfully removed job";;
Also, if you are modern, you can use "then" .. on most htcondor methods.
htcondorremovejobthenconsole.log"successfully removed job";fail//handle err;;
Wrapper for condor_q
htcondor = require"htcondor";htcondorqid: "59794891.0"console.logJSONstringifyjob null 4;;
Outputs job entry..
You can set various condor_q options.. like constraint, or attributes.
htcondor = require"htcondor";htcondorqconstraint: "JobStatus==5"attributes: "Iwd" "Owner" "JobStatus"console.logJSONstringifyjobs null 4;;
In above, node-htcondor will "stream" output from htcondor q output - so callbacks will be called as soon as each job info is parsed - instead of waiting until all jobs info is received.
Alternatively, then() will not stream but wait until it receives all job entries in a single array.
htcondor = require"htcondor";htcondorqconstraint: "JobStatus==5"thenconsole.logJSONstringifyjobs null 4;;
This module allows you to subscribe to condor event log (usually at /var/log/condor/EventLog), and receive callbacks so that you can monitor job status or any other attribute changes.
If you are monitoring jobs that you submit, then you can just watch the job log instead(See above). eventlog watcher is to monitor the entire cluster.
Obviously though.. you need to have EventLog enabled on your condor installation. You need to have something like following in your condor config.d if you don't see your EventLog generated already.
EVENT_LOG=$(LOG)/EventLogEVENT_LOG_JOB_AD_INFORMATION_ATTRS=Owner,CurrentHosts,x509userproxysubject,AccountingGroup,GlobalJobId,QDate,JobStartDate,JobCurrentStartDate,JobFinishedHookDone,MATCH_EXP_JOBGLIDEIN_Site,RemoteHostEVENT_LOG_MAX_SIZE = 1000000EVENT_LOG_MAX_ROTATIONS = 5
var eventlog = require'htcondor'eventlog//you can start watching on your htcondor eventlogeventlogwatch"/var/log/condor/EventLog";//and receive eventseventlogonconsoledirads;;
eventlog.on() will be called for each classads posted. ads will look like following.
Why didn't I implement this more like fs.watch()? 2 reasons... You often need to start watch before you know which job id to watch (see submit sample below). Another reason is that, you usually don't have more than 1 EventLog, so there is no point of having multiple watcher watching the same log.
_jobid: '49563264.000.000'_timestamp: '12/15 19:12:25'_updatetime: Sun Dec 15 2013 19:12:25 GMT+0000 UTCProc: 0EventTime: '2013-12-15T19:12:25'TriggerEventTypeName: 'ULOG_SUBMIT'SubmitHost: '<126.96.36.199:9615?sock=8287_a430_1068600>'QDate: 1387134745TriggerEventTypeNumber: 0MyType: 'SubmitEvent'Owner: 'donkri'CurrentHosts: 0GlobalJobId: 'osg-xsede.grid.iu.edu#49563264.0#1387134745'Cluster: 49563264AccountingGroup: 'group_xsedelow.donkri'Subproc: 0EventTypeNumber: 28CurrentTime: 'time()'
Call unwatch() to stop watchin on eventlog
(This is a prototype feature)
You can dump all condor configs (in key/value dictionary).
You may optionally configure the module by setting
config variable if HTCondor binaries or configuration are located in a non-standard location. Current options are:
node-htcondor watches job logsi for all jobs submitted using Tail - which uses inotify kernel hook. Don't submit too many jobs at once. You need to throttle it so that you won't exeed your inotify limit imposed by your OS. (See /proc/sys/fs/inotify/max_user_watches) I might implement such throttling capability built into this module in the future..
MIT. Please see License file for more details.