zenrows
    TypeScript icon, indicating that this package has built-in type declarations

    1.2.1 • Public • Published

    ZenRows Node.js SDK

    SDK to access ZenRows API directly from Node.js. ZenRows handles proxies rotation, headless browsers, and CAPTCHAs for you.

    Installation

    Install the SDK with npm.

    npm install zenrows

    Usage

    Start using the API by creating your API Key.

    The SDK uses axios for HTTP requests. The client's response will be an AxiosPromise if using TypeScript, a regular Promise otherwise.

    It also uses axios-retry to automatically retry failed requests (status code 429 and 5XX). Retries are not active by default; you need to specify the number of retries, as shown below. It already includes an exponential back-off retry delay between failed requests.

    const { ZenRows } = require('zenrows', { retries: 1 });
    
    const apiKey = 'YOUR-API-KEY';
    const url = 'https://www.zenrows.com/';
    
    (async () => {
        const client = new ZenRows(apiKey);
    
        const { data } = await client.get(url, {
            // Our algorithm allows to automatically extract content from any website
            autoparse: false,
    
            // CSS Selectors for data extraction (i.e. {"links":"a @href"} to get href attributes from links)
            css_extractor: '',
    
            // Enable Javascript with a headless browser (5 credits)
            js_render: false,
    
            // Use residential proxies (10 credits)
            premium_proxy: false,
    
            // Make your request from a given country. Requires premium_proxy
            proxy_country: '',
    
            // Wait for a given CSS Selector to load in the DOM. Requires js_render
            wait_for: '.content',
    
            // Wait a fixed amount of time in milliseconds. Requires js_render
            wait: 2500,
    
            // Block specific resources from loading, check docs for the full list. Requires js_render
            block_resources: 'image,media,font',
    
            // Change the browser's window width and height. Requires js_render
            window_width: 1920,
            window_height: 1080,
    
            // Will automatically use either desktop or mobile user agents in the headers
            device: 'desktop',
    
            // Will return the status code returned by the website
            original_status: false,
        }, {
            headers: {
                'Referrer': 'https://www.google.com',
                'User-Agent': 'MyCustomUserAgent',
            },
        });
    
        console.log(data);
    
        /* <!doctype html> <html... */
    
        // With the CSS selector {"links":"a @href"}
        /*
            {
                links: [
                    'https://www.zenrows.com',
                    'https://www.zenrows.com/blog',
                    ...
                ]
            }
        */
    })();

    You can also pass optional parameters and headers; the list above is a reference. For more info, check out the documentation page.

    Sending headers to the target URL will overwrite our defaults. Be careful when doing it and contact us if there is any problem.

    Concurrency

    To limit the concurrency, it uses fastq, which will simultaneously send a maximum of requests. The concurrency is determined by the plan you are in, so take a look at the pricing and set it accordingly. Take into account that each client instance will have its own limit, meaning that two different scripts will not share it, and 429 (Too Many Requests) errors might arise.

    We use Promise.allSettled() in the example below, available from Node 12.9. It will wait for all the promises to finish, and the results are objects with a status marking them as fulfilled or rejected. The main difference with Promise.all() is that it won't fail if any requests fail. It might make your scraping more robust since the whole list of URLs will run, even if some of them fail.

    const { ZenRows } = require('zenrows');
    
    const apiKey = 'YOUR-API-KEY';
    
    (async () => {
        const client = new ZenRows(apiKey, { concurrency: 5, retries: 1 });
    
        const urls = [
            'https://www.zenrows.com/',
            // ...
        ];
    
        const promises = urls.map(url => client.get(url));
    
        const results = await Promise.allSettled(promises);
        console.log(results);
        /*
        [
            {
                status: 'fulfilled',
                value: {
                    status: 200,
                    statusText: 'OK',
                    data: `<!doctype html> <html lang="en"> <head> ...
                
            ...
        */
    
        // separate results list into rejected and fulfilled for later processing
        const rejected = results.filter(({ status }) => status === 'rejected');
        const fulfilled = results.filter(({ status }) => status === 'fulfilled');
    })();

    Examples

    Take a look at the examples directory for Javascript and TypeScript files using the SDK. It has its own package.json file and includes zenrows SDK ready to use. Each file makes two requests, the first with CSS selectors and the second with CSS selectors and premium proxies in the US.

    cd examples
    npm install
    node index.js # JS example
    npx ts-node index.ts # TS example

    Contributing

    Pull requests are welcome. For significant changes, please open an issue first to discuss what you would like to change.

    License

    MIT

    Install

    npm i zenrows

    DownloadsWeekly Downloads

    41

    Version

    1.2.1

    License

    MIT

    Unpacked Size

    12.7 kB

    Total Files

    9

    Last publish

    Collaborators

    • anderrv
    • aurken