parallel-http-request

Simplicity to create multiple HTTP Request in Parallel for NodeJS.

Install using NPM

$ npm install parallel-http-request

Usage

Set Config

var ParallelRequest = require('parallel-http-request');
 
var config = {
    response: "simple"    // [optional] detail|simple|unirest, if empty then the response output is simple
};
 
var request = new ParallelRequest(config);
 
//or without config
var request = new ParallelRequest();

The config.response options value is :

• simple : minimalist output response.
• detail : output response very detail.
• unirest : output response with unirest format.

Make Request

request.add('https://jsonplaceholder.typicode.com/posts/1')
    .add('https://jsonplaceholder.typicode.com/posts/2')
    .send(function(response){
        console.log(response);
    });
 
// or
request.add({url:'https://jsonplaceholder.typicode.com/posts/1',method:'get'})
    .add({url:'https://jsonplaceholder.typicode.com/posts/2',method:'get'})
    .send(function(response){
        console.log(response);
    });
 
// or mixed it
request.add({
        url:'https://jsonplaceholder.typicode.com/posts/1',
        method:'get'
    })
    .add({
        url:'https://jsonplaceholder.typicode.com/posts/2',
        method:'post',
        headers:{'Content-Type':'application/json'},
        body: {
            fullname:'value',
            address:'value'
        }
    })
    .add({
        url:'https://jsonplaceholder.typicode.com/posts/3',
        method:'delete',
        headers:{'Content-Type':'application/json'},
        body: {
            fullname:'value',
            address:'value'
        }
    })
    .send(function(response){
        console.log(response);
    });

Documentation

This Parallel HTTP Request is created based on unirest-nodejs library, so request.options is almost similar with unirest.

Note:

• This library is intended to create multiple request in parallel, so not all unirest feature is worked. Please see Limitation.

request.add(options)

To make a multiple http request, you have to use .add() for each request.
request.add(string|object) this will add your request into collection.

• request.add(string) this request will use default method GET.
• request.add(object) this request will use object options.

request.send(callback)

This will execute your multiple request.
Return output is always array object.

request.send(function(response){
        console.log(response)
    });

Options object in request.add(options)

• url: (String) - Url to send the request.
• method: (String) - Default GET; HTTP Method.
• headers: (Object) - Default {}; HTTP Headers.
• query: (Object) - HTTP URI Parameter.
• body: (String | Object) - Entity body for certain requests.
• form: (Object) - Form Data.
• field: (Object) - Form fields.
• attach: (Object) - For handle files.
• followRedirect: (Boolean) - Default true; Follow HTTP 3xx responses as redirects.
• followAllRedirects: (Boolean) - Default false; Follow Non-GET HTTP 3xx responses as redirects.
• maxRedirects: (Number) - Default 10; Maximum number of redirects before aborting.
• timeout: (Number) - Number of milliseconds to wait before aborting.
• encoding: (String) - Encoding to be used on setEncoding of response data.
• strictSSL: (Boolean) - Default true; Sets strictSSL flag to require that SSL certificates be valid.
• httpSignature: (Object) - HTTP Signature.
• proxy: (String) - HTTP Proxy.
• secureProtocol: (Object) - Sets the secure protocol to use.
• localAddress: (Object) - Sets localAddress, local interface to bind for network connections.
• auth: (Object) - Accepts either an Object containing user, pass, and optionally sendImmediately.
• aws: (Object) - Sets aws, AWS Signing Credentials.
• hawk: (Object) - Sets hawk, HAWK Signing Credentials.
• cookie: (String) - Creates a cookie.

request.remove(name)

This will remove url in collection.

• name is the url of the request.

request.remove('http://google.com');

request.clean()

This will cleanup all request in collection.

request.clean();

request.getCollection()

This will return all request in collection.

request.getCollection();

request.unirest

If you want to use unirest (the underlying layer of parallel-http-request) directly.
Because Sometimes we are not always have to call request in parallel.
Please see Unirest Documentation.

request.unirest.get('http://google.com')
    .end(function(response){
        console.log(response.body)
    });

Example

Request with Method

You can just set method request by like this

request.add({url:'https://www.google.com', method:'get'});
request.add({url:'https://www.google.com', method:'post'});
request.add({url:'https://www.google.com', method:'put'});
request.add({url:'https://www.google.com', method:'patch'});
request.add({url:'https://www.google.com', method:'delete'});
request.add({url:'https://www.google.com', method:'head'});
request.add({url:'https://www.google.com', method:'options'});

Request with Query / Body / Form

POST with Query Parameter

request.add({
        url:'https://jsonplaceholder.typicode.com/posts/1', 
        method:'post', 
        query: {
            search:'value'
        }
    });

POST with Body

request.add({
        url:'https://jsonplaceholder.typicode.com/posts/1', 
        method:'post',
        headers:{'Content-Type':'application/json'}
        body: {
            fullname:'value',
            address:'value'
        }
    });

POST with Form Data Encoded

request.add({
        url:'https://jsonplaceholder.typicode.com/posts/1', 
        method:'post',
        headers:{'Content-Type':'application/x-www-form-urlencoded'}
        form: {
            fullname:'value',
            address:'value'
        }
    })
    .add({
        url:'https://jsonplaceholder.typicode.com/posts/2', 
        method:'post',
        headers:{'Content-Type':'application/x-www-form-urlencoded'}
        body: JSON.stringify({
            fullname:'value',
            address:'value'
        })
    })
    .add({
        url:'https://jsonplaceholder.typicode.com/posts/3', 
        method:'post',
        headers:{'Content-Type':'application/x-www-form-urlencoded'}
        body: 'name=nijiko&pet=turtle'
    });

POST with Body HTML / Other

request.add({
        url:'https://jsonplaceholder.typicode.com/posts/1', 
        method:'post',
        headers:{'Content-Type':'text/html'}
        body: '<strong>Hello World!</strong>'
    });

Request with Upload File

request.add({
        url:'http://mockbin.com/request',
        method:'post',
        headers:{
            'Content-Type': 'multipart/form-data',
            'Content-Length': fs.statSync(path.resolve('favicon.ico')).size
        },
        attach:{
            'file':fs.createReadStream(path.resolve('favicon.ico')),
            'remote file':request.unirest.request('http://google.com/favicon.ico')
        }
    });

Request with timeout

request.add({
        url:'http://www.google.com',
        timeout:60000
    });

Request with encoding

request.add({
        url:'http://www.google.com',
        encoding:'utf-8'
    });

Request with followRedirect

request.add({
        url:'http://www.google.com',
        followRedirect:true
    });

Request with maxRedirects

request.add({
        url:'http://www.google.com',
        maxRedirects:5
    });

Request with strictSSL

request.add({
        url:'https://www.google.com',
        strictSSL:false
    });

Request with proxy

request.add({
        url:'http://www.google.com',
        proxy:'http://localproxy.com'
    });

Request with secureProtocol

request.add({
        url:'https://www.google.com',
        secureProtocol:'SSLv3_client_method'
    });

Request with localAddress

request.add({
        url:'http://www.google.com',
        localAddress:'127.0.0.1'
    });

Request with auth

request.add({
        url:'http://www.google.com',
        auth:{
            user: 'Nijiko',
            pass: 'insecure',
            sendImmediately: true
        }
    });

Request with aws

request.add({
        url:'http://www.google.com',
        aws:{
            key: 'AWS_S3_KEY',
            secret: 'AWS_S3_SECRET',
            bucket: 'BUCKET NAME'
        }
    });

Request with hawk

request.add({
        url:'http://www.google.com',
        hawk:{
            credentials: {
                key: 'werxhqb98rpaxn39848xrunpaw3489ruxnpa98w4rxn',
                algorithm: 'sha256',
                user: 'Steve'
            }
        }
    });

Request with cookie

Create request with cookie. Please see documentation.

request.add({
        url:'http://www.google.com',
        cookie:'yummy_cookie=choco; tasty_cookie=strawberry'
    });

Request with Jar

Creates a container to store multiple cookies, i.e. a cookie jar.

var cookieJar = request.jar;
cookieJar.add(request.cookie('yummy_cookie=choco; tasty_cookie=strawberry'));
// or as argument
cookieJar.add('key=value', '/');
 
request.add({
        url:'http://google.com',
        jar:cookieJar
    });

Limitation

There is several feature which is not posible to do with multiple parallel request.

• oauth - Sets oauth, list of oauth credentials.
• part - Still Experimental; Similiar to request multipart.
• then - promise function.
• pool - Single request; for socket connection which is use for single connection.
• forever - Keeps socket connections alive between keep-alive in requests.

The solution about this limitation is you have to directly use unirest libary.

Example to use Unirest directly

Request with oAuth

request.unirest
  .get('https://api.twitter.com/oauth/request_token')
  .oauth({
    callback: 'http://mysite.com/callback/',
    consumer_key: 'CONSUMER_KEY',
    consumer_secret: 'CONSUMER_SECRET'
  })
  .then(response => {
    let access_token = response.body
 
    return request.unirest
      .post('https://api.twitter.com/oauth/access_token')
      .oauth({
        consumer_key: 'CONSUMER_KEY',
        consumer_secret: 'CONSUMER_SECRET',
        token: access_token.oauth_token,
        verifier: token: access_token.oauth_verifier
      })
  })
  .then((response) => {
    var token = response.body
 
    return request.unirest
      .get('https://api.twitter.com/1/users/show.json')
      .oauth({
        consumer_key: 'CONSUMER_KEY',
        consumer_secret: 'CONSUMER_SECRET',
        token: token.oauth_token,
        token_secret: token.oauth_token_secret
      })
      .query({
        screen_name: token.screen_name,
        user_id: token.user_id
      })
  })
  .then((response) => {
    console.log(response.body)
  });

Please see Unirest Documentation.

Unit Test

$ npm test