ISDP HSI download service
sudo yum install nodejs npm git
git clone firstname.lastname@example.org:hayashis/isdp.git cd isdp npm install
First, update the latest ISDP service
Then update ISDP dependencies..
Copy ./config/config_sample.js to ./config/config.js
See comments within the config file for more info, but if you aren't sure, feel free to contact me at email@example.com
All production node apps should run under pm2. If you agree, install pm2
sudo npm install pm2 -g
Finally, start the ISDP service
pm2 start isdp.js
To auto-start pm2 (and isdp app) during the next reboot,
pm2 save sudo pm2 startup redhat
- Use "systemd" instead of redhat if you are installing on systemd enabled host.
- If you don't want to run the app as root, specify the user ID you want to run pm2 under (like -u hayashis)
pm2 stores stdout/stderr from ISDP to ~/.pm2/logs . you can monitor it by
pm2 logs isdp
You can check the status of ISDP server via /health endpoint
If this doesn't return, or return non-200, then something is wrong.
You can also check the runtime information from pm2
pm2 show isdp
- Currently it receives request via web and immediately start processing request. This means all simultaneously requests will be executed in parallel... instead, I should do following.
- When a request comes, post to AMQP
- Write a separate handler that pulls request from AMQP one at a time and handle request
Currently doesn't handle invalid file path gracefully. Maybe I should skip missing ones and output a text file stating that files couldn't be downloaded?
Make sure an appropriate error message is generated (to who?) if incoming message is bogus.
Add a sensu check to make sure web-receiver and request handlers are running.
what happens if stage / publish directory can't be written? (doesn't exist, no access, not directory, disk is full)
add timeout mechanism for each task? also, should I async.retry?
what happens to error messages? stored in configured location (user can specify level)
there isn't much in the mocha test.. I need to add more unit testsing
- If caller send broken JSON, will it be logged in the error message?
what happens if amqp is down? I am not sure if it's pooling messages to be published locally, but as soon as amqp server comes back online, it reconnects seemslessly
what happens if 2 request comes in simultaneously? I am afraid "var job = this;" on async.series on sca-datamover will point to 2nd jobs while processing the 1st job? I've created a test script and it seems to work as expected. both job eventually finished successfully and correctly
What happens if unzip, tar, etc.. commands doesn't exists in path?