Negotiable Paleobotanist Missions

    parallel-http-request

    1.1.3 • Public • Published

    parallel-http-request

    NPM

    npm version Build Status Coverage Status Known Vulnerabilities NPM download/month NPM download total
    Simplicity to create multiple HTTP Request in Parallel for NodeJS.

    Install using NPM

    $ npm install parallel-http-request

    Usage

    Set Config

    var ParallelRequest = require('parallel-http-request');
     
    var config = {
        response: "simple"    // [optional] detail|simple|unirest, if empty then the response output is simple
    };
     
    var request = new ParallelRequest(config);
     
    //or without config
    var request = new ParallelRequest();

    The config.response options value is :

    • simple : minimalist output response.
    • detail : output response very detail.
    • unirest : output response with unirest format.

    Make Request

    request.add('https://jsonplaceholder.typicode.com/posts/1')
        .add('https://jsonplaceholder.typicode.com/posts/2')
        .send(function(response){
            console.log(response);
        });
     
    // or
    request.add({url:'https://jsonplaceholder.typicode.com/posts/1',method:'get'})
        .add({url:'https://jsonplaceholder.typicode.com/posts/2',method:'get'})
        .send(function(response){
            console.log(response);
        });
     
    // or mixed it
    request.add({
            url:'https://jsonplaceholder.typicode.com/posts/1',
            method:'get'
        })
        .add({
            url:'https://jsonplaceholder.typicode.com/posts/2',
            method:'post',
            headers:{'Content-Type':'application/json'},
            body: {
                fullname:'value',
                address:'value'
            }
        })
        .add({
            url:'https://jsonplaceholder.typicode.com/posts/3',
            method:'delete',
            headers:{'Content-Type':'application/json'},
            body: {
                fullname:'value',
                address:'value'
            }
        })
        .send(function(response){
            console.log(response);
        });

    Documentation

    This Parallel HTTP Request is created based on unirest-nodejs library, so request.options is almost similar with unirest.

    Note:

    • This library is intended to create multiple request in parallel, so not all unirest feature is worked. Please see Limitation.

    request.add(options)

    To make a multiple http request, you have to use .add() for each request.
    request.add(string|object) this will add your request into collection.

    • request.add(string) this request will use default method GET.
    • request.add(object) this request will use object options.

    request.send(callback)

    This will execute your multiple request.
    Return output is always array object.

    request.send(function(response){
            console.log(response)
        });

    Options object in request.add(options)

    • url: (String) - Url to send the request.
    • method: (String) - Default GET; HTTP Method.
    • headers: (Object) - Default {}; HTTP Headers.
    • query: (Object) - HTTP URI Parameter.
    • body: (String | Object) - Entity body for certain requests.
    • form: (Object) - Form Data.
    • field: (Object) - Form fields.
    • attach: (Object) - For handle files.
    • followRedirect: (Boolean) - Default true; Follow HTTP 3xx responses as redirects.
    • followAllRedirects: (Boolean) - Default false; Follow Non-GET HTTP 3xx responses as redirects.
    • maxRedirects: (Number) - Default 10; Maximum number of redirects before aborting.
    • timeout: (Number) - Number of milliseconds to wait before aborting.
    • encoding: (String) - Encoding to be used on setEncoding of response data.
    • strictSSL: (Boolean) - Default true; Sets strictSSL flag to require that SSL certificates be valid.
    • httpSignature: (Object) - HTTP Signature.
    • proxy: (String) - HTTP Proxy.
    • secureProtocol: (Object) - Sets the secure protocol to use.
    • localAddress: (Object) - Sets localAddress, local interface to bind for network connections.
    • auth: (Object) - Accepts either an Object containing user, pass, and optionally sendImmediately.
    • aws: (Object) - Sets aws, AWS Signing Credentials.
    • hawk: (Object) - Sets hawk, HAWK Signing Credentials.
    • cookie: (String) - Creates a cookie.

    request.remove(name)

    This will remove url in collection.

    • name is the url of the request.
    request.remove('http://google.com');

    request.clean()

    This will cleanup all request in collection.

    request.clean();

    request.getCollection()

    This will return all request in collection.

    request.getCollection();

    request.unirest

    If you want to use unirest (the underlying layer of parallel-http-request) directly.
    Because Sometimes we are not always have to call request in parallel.
    Please see Unirest Documentation.

    request.unirest.get('http://google.com')
        .end(function(response){
            console.log(response.body)
        });

    Example

    Request with Method

    You can just set method request by like this

    request.add({url:'https://www.google.com', method:'get'});
    request.add({url:'https://www.google.com', method:'post'});
    request.add({url:'https://www.google.com', method:'put'});
    request.add({url:'https://www.google.com', method:'patch'});
    request.add({url:'https://www.google.com', method:'delete'});
    request.add({url:'https://www.google.com', method:'head'});
    request.add({url:'https://www.google.com', method:'options'});

    Request with Query / Body / Form

    POST with Query Parameter
    request.add({
            url:'https://jsonplaceholder.typicode.com/posts/1', 
            method:'post', 
            query: {
                search:'value'
            }
        });
    POST with Body
    request.add({
            url:'https://jsonplaceholder.typicode.com/posts/1', 
            method:'post',
            headers:{'Content-Type':'application/json'}
            body: {
                fullname:'value',
                address:'value'
            }
        });
    POST with Form Data Encoded
    request.add({
            url:'https://jsonplaceholder.typicode.com/posts/1', 
            method:'post',
            headers:{'Content-Type':'application/x-www-form-urlencoded'}
            form: {
                fullname:'value',
                address:'value'
            }
        })
        .add({
            url:'https://jsonplaceholder.typicode.com/posts/2', 
            method:'post',
            headers:{'Content-Type':'application/x-www-form-urlencoded'}
            body: JSON.stringify({
                fullname:'value',
                address:'value'
            })
        })
        .add({
            url:'https://jsonplaceholder.typicode.com/posts/3', 
            method:'post',
            headers:{'Content-Type':'application/x-www-form-urlencoded'}
            body: 'name=nijiko&pet=turtle'
        });
    POST with Body HTML / Other
    request.add({
            url:'https://jsonplaceholder.typicode.com/posts/1', 
            method:'post',
            headers:{'Content-Type':'text/html'}
            body: '<strong>Hello World!</strong>'
        });

    Request with Upload File

    request.add({
            url:'http://mockbin.com/request',
            method:'post',
            headers:{
                'Content-Type': 'multipart/form-data',
                'Content-Length': fs.statSync(path.resolve('favicon.ico')).size
            },
            attach:{
                'file':fs.createReadStream(path.resolve('favicon.ico')),
                'remote file':request.unirest.request('http://google.com/favicon.ico')
            }
        });

    Request with timeout

    request.add({
            url:'http://www.google.com',
            timeout:60000
        });

    Request with encoding

    request.add({
            url:'http://www.google.com',
            encoding:'utf-8'
        });

    Request with followRedirect

    request.add({
            url:'http://www.google.com',
            followRedirect:true
        });

    Request with maxRedirects

    request.add({
            url:'http://www.google.com',
            maxRedirects:5
        });

    Request with strictSSL

    request.add({
            url:'https://www.google.com',
            strictSSL:false
        });

    Request with proxy

    request.add({
            url:'http://www.google.com',
            proxy:'http://localproxy.com'
        });

    Request with secureProtocol

    request.add({
            url:'https://www.google.com',
            secureProtocol:'SSLv3_client_method'
        });

    Request with localAddress

    request.add({
            url:'http://www.google.com',
            localAddress:'127.0.0.1'
        });

    Request with auth

    request.add({
            url:'http://www.google.com',
            auth:{
                user: 'Nijiko',
                pass: 'insecure',
                sendImmediately: true
            }
        });

    Request with aws

    request.add({
            url:'http://www.google.com',
            aws:{
                key: 'AWS_S3_KEY',
                secret: 'AWS_S3_SECRET',
                bucket: 'BUCKET NAME'
            }
        });

    Request with hawk

    request.add({
            url:'http://www.google.com',
            hawk:{
                credentials: {
                    key: 'werxhqb98rpaxn39848xrunpaw3489ruxnpa98w4rxn',
                    algorithm: 'sha256',
                    user: 'Steve'
                }
            }
        });

    Request with cookie

    Create request with cookie. Please see documentation.

    request.add({
            url:'http://www.google.com',
            cookie:'yummy_cookie=choco; tasty_cookie=strawberry'
        });

    Request with Jar

    Creates a container to store multiple cookies, i.e. a cookie jar.

    var cookieJar = request.jar;
    cookieJar.add(request.cookie('yummy_cookie=choco; tasty_cookie=strawberry'));
    // or as argument
    cookieJar.add('key=value', '/');
     
    request.add({
            url:'http://google.com',
            jar:cookieJar
        });

    Limitation

    There is several feature which is not posible to do with multiple parallel request.

    • oauth - Sets oauth, list of oauth credentials.
    • part - Still Experimental; Similiar to request multipart.
    • then - promise function.
    • pool - Single request; for socket connection which is use for single connection.
    • forever - Keeps socket connections alive between keep-alive in requests.

    The solution about this limitation is you have to directly use unirest libary.

    Example to use Unirest directly

    Request with oAuth

    request.unirest
      .get('https://api.twitter.com/oauth/request_token')
      .oauth({
        callback: 'http://mysite.com/callback/',
        consumer_key: 'CONSUMER_KEY',
        consumer_secret: 'CONSUMER_SECRET'
      })
      .then(response => {
        let access_token = response.body
     
        return request.unirest
          .post('https://api.twitter.com/oauth/access_token')
          .oauth({
            consumer_key: 'CONSUMER_KEY',
            consumer_secret: 'CONSUMER_SECRET',
            token: access_token.oauth_token,
            verifier: token: access_token.oauth_verifier
          })
      })
      .then((response) => {
        var token = response.body
     
        return request.unirest
          .get('https://api.twitter.com/1/users/show.json')
          .oauth({
            consumer_key: 'CONSUMER_KEY',
            consumer_secret: 'CONSUMER_SECRET',
            token: token.oauth_token,
            token_secret: token.oauth_token_secret
          })
          .query({
            screen_name: token.screen_name,
            user_id: token.user_id
          })
      })
      .then((response) => {
        console.log(response.body)
      });

    Please see Unirest Documentation.

    Unit Test

    $ npm test

    Install

    npm i parallel-http-request

    DownloadsWeekly Downloads

    130

    Version

    1.1.3

    License

    MIT

    Unpacked Size

    83.7 kB

    Total Files

    13

    Last publish

    Collaborators

    • aalfiann