asm-pm-runner runs postman collections and export results to crs format for usage in Apica ASM. The command line tool is based on the standard Postman Newman utility. See Newman npm at.
The main enhancement to Newman is support for decryption of data files and certificates. Encryption/decryption is based on the cryptify npm module.
Apica ASM (Synthetics) can run Postman checks based on asm-pm-runner. You can contact Apica about this feature/functionality.
npm install -g @apica-io/asm-pm-runner
Version | Summary changes |
---|---|
1.4.8 | Enhanced the return message to provide customized error feedback upon assertion failure. |
1.3.7 | Customized return message and fixes for variables with 0 values |
1.3.5 | Report error/exception in the result message |
1.3.4 | Fixed custom return value with unit and variables in json report |
Usage: asm-pm-runner [options] <collection>
Options:
-e, --environment <path> Environment JSON file
-dk, --decryptKey <decryptKey> Decrypt key
-ev, --envVars [envVars...] Environment variables. Format name=value format
-v, --verbose Print collection information on stdout (default: false)
-r, --resultDir <dir> The result directory
-dd, --dataDir <dir> The data directory for certificates and test data
-l, --logLevel <logLevel> Log level in log4js (default: "info")
--sslClientCert <path> Specify the path to a client certificate (PEM)
--sslClientKey <path> Specify the path to a client certificate private key
--sslClientPassphrase <passphrase> Specify the client certificate passphrase (for protected key)
-V, --version output the version number
-h, --help display help for command
You will find some runnable examples in the installation directory under samples . See NPM documentation about where you can find the global npm directory. The package.json file contains scripts for running the samples with correct parameter settings.
$ asm-pm-runner samples/HTTP-Bin-Requests.postman_collection.json -v -ev username=foo password=bar
Example 2 - Testing certificate on https://badssl.com
This sample use encrypted certificates.
$ asm-pm-runner -l debug -dk Encrypt.4.ssl --sslClientCert file.cert.pem --sslClientKey file.key.pem --dataDir samples/ec_badssl_certs -r results/ -v samples/BadSSL.postman_collection.json
[2021-08-18T10:08:21.470] [INFO] @apica-io/asm-pm-runner - @apica-io/asm-pm-runner (1.0.2) started
[2021-08-18T10:08:21.478] [DEBUG] @apica-io/asm-pm-runner - collection=samples/BadSSL.postman_collection.json
[2021-08-18T10:08:21.574] [DEBUG] NewmanRunner - Decrypting /var/folders/g2/zb9q82pd18sd683qvmbyr0040000gn/T/31565_file.cert.pem
[2021-08-18T10:08:21.580] [DEBUG] NewmanRunner - Decrypting /var/folders/g2/zb9q82pd18sd683qvmbyr0040000gn/T/31565_file.key.pem
itemGroup BadSSL
item ClientBadSSL
request GET https://client.badssl.com/
response OK 200 OK 1891 { body: 687, header: 283, total: 970 }
timingPhases {
secure_handshake_ms: 339.4973890000001,
prepare: 73.57351800000015,
socket_wait_ms: 59.02749899999981,
dns_time_ms: 1147.0008660000003,
tcp_connect_ms: 171.81406999999945,
time_to_first_byte_ms: 152.74416699999983,
download_time_ms: 19.97996400000011,
process_time_ms: 1.6743750000005093,
total_time_ms: 1965.3118480000003
}
assertion Check status code is 200 true
Collection BadSSL
run.stats {
iterations: { total: 1, pending: 0, failed: 0 },
items: { total: 1, pending: 0, failed: 0 },
scripts: { total: 1, pending: 0, failed: 0 },
prerequests: { total: 1, pending: 0, failed: 0 },
requests: { total: 1, pending: 0, failed: 0 },
tests: { total: 1, pending: 0, failed: 0 },
assertions: { total: 1, pending: 0, failed: 0 },
testScripts: { total: 1, pending: 0, failed: 0 },
prerequestScripts: { total: 0, pending: 0, failed: 0 }
}
[2021-08-18T10:08:24.072] [DEBUG] NewmanRunner - Clean up temporary files [
'/var/folders/g2/zb9q82pd18sd683qvmbyr0040000gn/T/31565_file.cert.pem',
'/var/folders/g2/zb9q82pd18sd683qvmbyr0040000gn/T/31565_file.key.pem'
]
Result: true Requests:1, tests:1, assertions:1
[2021-08-18T10:08:24.074] [DEBUG] NewmanRunner - Results written to /Users/janostgren/work/node/asm-pm-runner/results/BadSSL_res.json
The json report is stored in the results directory. The format is complaint with Check Result Service in Apica ASM.
The json report is divided into one to many steps. A step contains requests from the postman collection.
- Default only one step with all requests is generated.
- Several steps will be generated if the collection contains folders. One step for each folder will be generated and step name corresponds to folder name.
- Only one level of steps is supported. Folders in folders will be named folder/subfolder. Example root folder/sub folder 1/sub folder 2
asm-pm-runner can return a custom value as the return value when running a collection. This functionality is often used in ASM when you want return something else than the total response time as the main result of the check run. What you need to do in the collection is use 2 special collection parameters and assign values to them in a JavaScript test.
Collection Variable | Description |
---|---|
_Apica_ReturnValue | A numeric value used as the check value (return value) |
_Apica_ReturnUnit | The unit of the customized return value |
_Apica_Message | A customized message return as the message in ASM |
pm.test("Status code is 200", function () {
pm.response.to.have.status(200);
});
let eventCount =pm.response.json().length;
pm.collectionVariables.set("_Apica_ReturnValue",eventCount)
pm.collectionVariables.set("_Apica_ReturnUnit","events")
pm.collectionVariables.set("_Apica_Message",`Event count = ${eventCount}`)
You can also return variable values in the result report. If you want suppress a variable from being included in the result report use a name starting with the "_" character. Collection and Environment variables not changed during the run of collection will be typed as input variables.
let json =pm.response.json();
pm.collectionVariables.set("eventName",json.name)
pm.collectionVariables.set("category",json?.category?.description|| "")
pm.collectionVariables.set("mediaLink",json?.mediaItem?.url|| "")
"variables": [
{
"key": "baseUrl",
"type": "url",
"usage": "input",
"value": "http://ticketmonster.apicasystem.com"
},
{
"key": "eventName",
"type": "string",
"usage": "inResponse",
"value": "Champions League"
},
{
"key": "category",
"type": "string",
"usage": "inResponse",
"value": "Sport"
},
{
"key": "mediaLink",
"type": "string",
"usage": "inResponse",
"value": "http://img.uefa.com/imgml/2016/ucl/social/og-default.jpg"
}
]
The name of result file will be {collection.name}_res.json. The environment variable APICA_JOB_ID can be used to set the basename of the report file.
{
"name": "BadSSL",
"type": "Postman.Collection",
"result_version": "1.0.2",
"producer": "@apica-io/asm-pm-runner 1.0.2",
"value": 1891,
"unit": "ms",
"returncode": 0,
"success": true,
"message": "Requests:1, tests:1, assertions:1",
"duration_ms": 1891,
"content_length": 687,
"start_timestamp_ms": 1629274104025,
"end_timestamp_ms": 1629274105916,
"metrics": {
"dns_time_ms": 1147,
"download_time_ms": 20,
"prepare": 74,
"process_time_ms": 2,
"secure_handshake_ms": 339,
"socket_wait_ms": 59,
"tcp_connect_ms": 172,
"time_to_first_byte_ms": 153,
"total_time_ms": 1965
},
"variables": [],
"steps": [
{
"name": "BadSSL",
"duration_ms": 1891,
"content_length": 687,
"start_timestamp_ms": 1629274104025,
"end_timestamp_ms": 1629274105916,
"metrics": {
"dns_time_ms": 1147,
"download_time_ms": 20,
"prepare": 74,
"process_time_ms": 2,
"secure_handshake_ms": 339,
"socket_wait_ms": 59,
"tcp_connect_ms": 172,
"time_to_first_byte_ms": 153,
"total_time_ms": 1965
},
"requests": [
{
"name": "ClientBadSSL",
"duration_ms": 1891,
"content_length": 687,
"start_timestamp_ms": 1629274104025,
"end_timestamp_ms": 1629274105916,
"metrics": {
"secure_handshake_ms": 339,
"prepare": 74,
"socket_wait_ms": 59,
"dns_time_ms": 1147,
"tcp_connect_ms": 172,
"time_to_first_byte_ms": 153,
"download_time_ms": 20,
"process_time_ms": 2,
"total_time_ms": 1965
},
"url": "https://client.badssl.com/",
"method": "GET",
"status": 200,
"status_text": "OK",
"request_headers": {
"User-Agent": "PostmanRuntime/7.28.2",
"Accept": "*/*",
"Cache-Control": "no-cache",
"Postman-Token": "c0cfe5c6-d962-4dcc-a2bf-bef71a09f62c",
"Host": "client.badssl.com",
"Accept-Encoding": "gzip, deflate, br",
"Connection": "keep-alive"
},
"headers": {
"Server": "nginx/1.10.3 (Ubuntu)",
"Date": "Wed, 18 Aug 2021 08:08:24 GMT",
"Content-Type": "text/html",
"Last-Modified": "Tue, 23 Feb 2021 21:28:41 GMT",
"Transfer-Encoding": "chunked",
"Connection": "keep-alive",
"ETag": "W/\"60357389-2af\"",
"Cache-Control": "no-store",
"Content-Encoding": "gzip"
}
}
],
"success": true,
"assertions": [
{
"source": "ClientBadSSL",
"description": "Check status code is 200",
"status": "info",
"expression": "Check status code is 200",
"value": true
}
]
}
]
}
Encryption/Decryption of certificates and environment files is supported. You use the NPM package cryptify for decrypting and encrypting the files.
You must supply a decryption key with -dk option. The file must been encrypted with the key using the command line version of cryptify. Install the package globally and use the tool.
$ npm install -g cryptify
$ cryptify
Usage: cryptify [options] [command]
Options:
-v, --version Display the current version
-l, --list List available ciphers
-h, --help Display help for the command
Commands:
encrypt [options] <file...> Encrypt files(s)
decrypt [options] <file...> Decrypt files(s)
help <command> Display help for the command
Example:
$ cryptify encrypt file.txt -p 'Secret123!'
$ cryptify decrypt file.txt -p 'Secret123!'
Password Requirements:
1. Must contain at least 8 characters
2. Must contain at least 1 special character
3. Must contain at least 1 numeric character
4. Must contain a combination of uppercase and lowercase
$ cryptify encrypt Http_bin_environment.postman_environment.json --password Encrypt.4.ssl
$ asm-pm-runner samples/HTTP-Bin-Requests.postman_collection.json -r results -l debug -v -e samples/ec_env/Http_bin_environment.postman_environment.json -dk Encrypt.4.ssl