Hawk
The Hawk is a web crawler tool developed for high performance, supports pages mapping and updating information already collected.
Install Hawk
$ sudo apt-get install cpulimit$ sudo npm install -g pm2$ git clone https://github.com/andrehrf/hawk.git$ npm installStart application
$ cd hawk$ pm2 start ecosystem.jsonhttp://localhost:5800