concurrency-request

Description

concurrency-request, it can split multiple requests simultaneously and group them together. You can use it when you need to send a large number of requests and don't care about the order in which they are completed.

concurrency-request，它可以同时将多个请求项进行拆分，并分组请求。你可以在需要发送大量请求且不在乎请求完成的先后顺序时使用它。

Versions

Version	Date	Change logs(CN)	Change logs(EN)
1.0.0	2022-09-28	初始化版本	Initialize the version
1.0.1	2022-09-29	新增了 setOnRequest，移除 setIgnoreStatusCodeError	Added 'setOnRequest', removed 'setIgnoreStatusCodeError'
1.0.9	2022-09-30	request 替换为 axios 的过度版本，此时还是使用的request，但setOnRequest的参数已经发生改变	'request' is replaced with an overdone version of axios. Request is still used, but the parameters of setOnRequest have changed
*1.0.91	2022-10-31	onGroup回调新增progress参数，用于知晓进度	Added 'progress' on 'OnGroup'

Usage

npm i concurrency-request

const ConcurrencyRequest = require("concurrency-request");
// create options
const options = new Array(100).fill({
    method: "GET",
    url: "https://www.example.com",
    headers: {},
});
// instantiation concurrency-request and run
// the default group is 20 requets
new ConcurrencyRequest(options).run().then((allResult) => {
    allResult = allResult
        .filter(([error, result]) => !error)
        .map(([error, result]) => result);
});

options参数说明：version < '1.1.0'? 请参考request(config): 请参考axios(config)

In addition, you can carry other properties that are not used by option, but are also available in the onRequest after the request is completed. As follows:

除此之外你还可以携带其他不被 option 使用的属性，携带的属性在请求结束之后的onRequest中同样能获取到。如下：

const options = new Array(100).fill({
    method: "GET",
    url: "https://www.example.com",
    headers: {},
    _row: {
        name: "Erick",
        car: "BMW",
    },
});
const onRequest = function (error, respones) {
    const data = respones.data,
        /* _row is additional value*/
        _row = respones.config._row;
    // ...
};
await new ConcurrencyRequest(options).group(10).setOnRequest(onRequest).run();

---

API

• group(count)
• setGroupSleepTime(timer)
• setOnRequest(onRequest)
• setOnGroup(onGroup)
• run()

---

group

Group options requests, groupGount is the maximum number of each group, groupGount default value is 20.

对 options 请求进行分组, groupGount 为每组的最大数量， 默认值为 20

await new ConcurrencyRequest(options).group(10).run();

---

setGroupSleepTime

Sets the latency, in milliseconds, for completing a set of requests

设置完成一组请求的延迟时间，单位毫秒

await new ConcurrencyRequest(options).setGroupSleepTime(300).run();

---

setOnRequest

When you need to know or process the request when it is completed. You can use setOnRequest:

It takes a function as an argument. This function will have the error, respones attributes used in the callback. Note that you need to return a value inside the function to be used as the return value of the request result.

当你需要在请求完成时知晓或处理的情况下。你可以使用 setOnRequest:

它接受一个函数作为参数。该函数回调时会有 error, respones参数提供。值得注意的是，你需要在该函数内部返回一个值，作为该请求的返回值使用。

// If you need to do something with the requested data
const options = new Array(10).fill({
    method: "GET",
    url: "https://www.example.com",
    headers: {},
    _row: {
        name: "Erick",
        car: "BMW",
    },
});
const onRequest = function (error, respones) {
    const { statusCode, data, config } = respones;
    return error
        ? ""
        : {
              name: config._row.name,
              car: config._row.car,
              data: data.slice(0, 10),
          };
};
await new ConcurrencyRequest(options).setOnRequest(onRequest).run();

---

setRequestErrorCallBack

If you need to know about or handle errors in the function that is passed in when an error occurs in the request, you can do this:

当请求发生错误时会触发传入的函数，若您需要知晓或者处理这些错误，你可以这样做：

const requestError = function (error) {
    // ...
};
await new ConcurrencyRequest(options)
    .setRequestErrorCallBack(requestError)
    .run();

---

setOnGroup

Sets the callback function that fires when a set of requests completes

设置当一组请求完成时触发的回调函数

const onGroup = function (groupResults, progress) {
    const { total, current } = progress;
    // ...
    groupResults.map(([error, result]) => result);
};
new ConcurrencyRequest(options).setOnGroup(onGroup).run();

---

run

The group request begins, and the method is called to return a Promise result containing all of the request return information.

开始进行分组请求，调用该方法之后会返回一个Promise结果，该结果中包含了所有的请求返回信息。

// `allResult` data structure like
// [[error, respones | onRequest-returns], ...]
new ConcurrencyRequest(options)
    .run()
    .then(allResult => {
        allResult.map(([error, result] => result))
    })
// or use async/await
const allResult = new ConcurrencyRequest(options).run();
allResult.map(([error, result] => result))