PhantomJS Console Crawler
- Currently this only works on unix systems
- You need to have
pkillinstalled, try typing
which pkill, if not you can try
brew install pkill
You can easily install the package via npm
npm install consolecrawler -g
As long as your NPM modules is in your $PATH you should be able to run
consolecrawlerfrom your command line.
Usage: consolecrawler [options] Crawls a given website using PhantomJS - and logging any console errors Options: -h, --help output usage information -V, --version output the version number -u, --url <string> Define the URL you want to crawl -m, --max <n> Defines max pages to scan (defaults to 25) -i, --id <string> Defines the ID of a scan, useful for debugging logs -l, --log Writes the raw output to file -v, --verbose Gives more verbose logging messages
- For each webpage crawled, a new process gets created on the machine, these should get cleaned up after shutting down, but if they don't you can run
pkill -f phantomjs
- Your computer may take off into space.