zappos-product-crawler

Gets Zappos product list from a search link with breadcrumb attributes included.

First, do:

npm install

then

for Zappos:

node zappos.js <zapposSearchLink> <searchPageLimit>

If you do not put search page limit, it defaults to 5 pages.

To run:

• If you want the result pages to be returned faster, go to setTimeout( next, 500 ); line and change 500 to some other number in milliseconds
• If you want to change how many product pages are simultaneously crawled at any given time, in the line with async.mapLimit( merged, 10, function ( item, callback ) change 10 to some other integer.