idra
Command Line utility for integrating with IBM® DevOps Insights
To install this utility
npm install -g grunt-idra3
Table of Contents
- Concepts
- Getting Started
- Upload Build Records
- Upload Test Results
- Upload Deployment Records
- Making decisions about build quality using gate policies
Concepts
IBM® DevOps Insights is a tool that provides insights into your DevOps projects. Use it to learn how compliant your team is with DevOps and developer practices, to manage risk in your codebase, and to automatically enforce quality standards in continuous delivery projects.
Understanding of the following concepts will help in faster adoption of IBM® DevOps Insights for your projects.
Toolchain
Toolchains are part of IBM® DevOps Continuous Delivery. Toolchains provide an integrated set of tools to build, deploy and manage your apps. You can create toolchains that include IBM Cloud services, open source tools, and third party tools that make development and operations repeatable and easier to manage. It is required that the IBM DevOps Insights should be added as one of the tool to your toolchain.
Integrating your build tool with IBM DevOps Insights
This means instrumenting your build tool with this CLI or DevOps Insights Jenkins plugin to send build, test and deployment records to IBM DevOps Insights.
Build record
IBM DevOps Insights needs to know when a build is complete. This record contains application name, branch, build id, reference to git repo and other fields. This CLI and Jenkins plugin has a command to publish the build record to DevOps Insights.
Test record
IBM DevOps Insights needs to know about the tests performed during the build process. It is necessary that the tests should generate some kind of JSON or XML based file. Here are the list of supported format. The generated test file is published as a test record. Along with the test file this record contains application name, build id, type of tests and other fields. It is required that the application name and build id in the test record matches the one used in the build record for a given build. This CLI and Jenkins plugin has a command to publish test record to DevOps Insights.
Deployment records
IBM DevOps Insights needs to know about the deployments made during the deployment process. This record contains the application name, build id, environment name, status of the deployment and other fields. It is required that the application name and build id in the deployment record matches the one used in the build record for a given build. This CLI and Jenkins plugin has a command to publish a deployment to DevOps Insights.
Build Id (or Number)
This is an arbitrary length string uniquely identifying each build. This id is used to tie the build, test and deployment records together so it is important to publish those records with the same id for a given build.
Policy
A policy is set of rules specified by the user that collectively defines what is considered a good set of test records for a given application build. Typically, one rule is added for each test record published to DevOps Insights. The policy rules are evaluated by the gate.
Branch
This is the name of the branch in the source control system.
Environment
This is the name of environment to which deployments are made during the build process.
Data set
This is an arbitrary string that identifies the type of tests performed during the build process. Besides the standard names such as "unit test", "fvt". The tool allows the user to define a custom name.
Getting Started
Use this utility to integrate IBM® DevOps Insights with Travis CI, Concourse, IBM Urban Code Deploy, IBM Continuous Delivery Pipelines and other CI/CD tools. If you are using Jenkins, use this Jenkins plugin instead.
Using this utility, you can
- Publish build records, test records and deploy records to IBM® DevOps Insights
- Make go/no-go decisions about a build's quality by using a Gate policy
For samples of dashboards in IBM® DevOps Insights, see Value Proposition.
For more information about IBM DevOps Insights, see the official documentation.
Scenarios
This document covers two scenarios:
- You are integrating DevOps Insights with IBM Continuous Delivery tool.
- You are integrating DevOps Insights with other CI/CD tools (e.g., Travis CI, Concourse, IBM Urban Code Deploy, others)
Prerequisites
To use the CLI, you need to do the following. These steps should take about 10 minutes.
-
A valid IBM Cloud account. If you don't have one, register here.
-
Once you have an account, you can create an API Key. Log in to IBM Cloud. Click on Manage->Security->Platform API Key. On that screen, create an API Key.
-
An IBM Continuous Delivery toolchain. To create a toolchain:
- Go to https://console.bluemix.net/devops
- Click on Create a Toolchain button and then select Build Your Own Toolchain template.
- In the next screen, click Create.
- From the URL displayed in the browser, make note of your toolchain id. For example, if your URL is https://console.bluemix.net/devops/toolchains/05572df3-01dc-441b-911d-b2202fe33f96, the toolchain id is the long string after toolchains/
- You can always navigate to your toolchain by clicking here
- Add DevOps Insights tool to your toolchain
- Go to your toolchain
- Click on Add a Tool button
- Select DevOps Insights tool and click Create Integration button
- Your CI/CD tool must have Node.js® 4.x or greater.
Environment Variables
If you are using Travis, Concourse or any CI/CD tool other than the IBM Continuous Delivery pipeline, then set the following environment variables:
| Name | Description |
|---|---|
| TOOLCHAIN_ID | Required. The toolchain's GUID |
| IBM_CLOUD_API_KEY | Required. The Bluemix user's Api key |
| BUILD_NUMBER | Required. Any string that identifies the build. You also have the option of specifying build number using the --buildnumber parameter, in which case this environment variable is not required. |
| LOGICAL_APP_NAME | Required. The application's name. You also have the option of specifying application's name using the --logicalappname parameter, in which case this environment variable must not be set. It is an error to set both the environment variable and the parameter. |
If you are using IBM Continuous Delivery pipeline, you need to set the following environment variables:
| Name | Description |
|---|---|
| IBM_CLOUD_API_KEY | Required The Bluemix user's Api key |
| BUILD_PREFIX | Required. Any string. For example, the Git branch name. If the --buildnumber parameter is used to pass a build number then this environment variable is not required. |
| LOGICAL_APP_NAME | Required. The application's name. You also have the option of specifying application's name using the --logicalappname parameter, in which case this environment variable must not be set. It is an error to set both the environment variable and the parameter. |
| PATH | Required. In order to have required version of Node.js in your path, set the PATH variable before invoking the utility: export PATH=/opt/IBM/node-v4.2/bin:$PATH |
If you are using IBM Continuous Delivery pipeline and if you want to publish Insights data to another toolchain, then set the following environment variables.
| Name | Description |
|---|---|
| TOOLCHAIN_ID | Required. The target toolchain's GUID. If the target toolchain is owned by different user then provide the target toolchain owner's Bluemix Api key in the IBM_CLOUD_API_KEY variable. |
If you are using IBM UrbanCode Deploy, DevOps Insights relies on the previous set of variables, and this one:
| Name | Description |
|---|---|
| UCD_REQUEST_ID | This maps to the ${p:request.id} property value. |
Enabling Debug logs
If you want to enable debug logs for the DevOps Insights CLI, set the IDRA_DEBUG environment variable to true. This works in all environments.
| Name | Description |
|---|---|
| IDRA_DEBUG | When this value is true, the DevOps Insights CLI's debugging mode is enabled. By default, the value is false. |
Using the CLI
Set environment details as described above. Ensure that Node.js 4.2 or higher is avaialble in your CI/CD tool.
To install this utility, use npm install -g grunt-idra3
Then you can invoke the CLI with the idra command. You can run the CLI in any pipeline job's command script. For example, this command uploads functional verification test results to DevOps Insights:
# Upload test results
idra --publishtestresult --filelocation="results.json" --type=fvt
Upload Build Records
Use the --publishbuildrecord command (alias -b) to upload build records to DevOps Insights.
Command syntax and options
idra --publishbuildrecord --branch=<branch> --repositoryurl=<repository_url> --commitid=<commit_id> --status=<pass|fail> --logicalappname=<appname> --buildnumber=<buildid> [--joburl=<joburl>] [--buildnumber=<buildid>]| Option | Alias | Description |
|---|---|---|
--branch |
-B |
Required. The repository branch on which the build is being performed. This value is not required in IBM Continuous Delivery pipeline environment |
--repositoryurl |
-R |
Required. The url of the git repository. This value is not required in IBM Continuous Delivery pipeline environment |
--commitid |
-C |
Required. The git commit id. This value is not required in IBM Continuous Delivery pipeline environment |
--status |
-S |
Required. The build status. This value must be either pass or fail. |
--logicalappname |
-L |
Optional. The application's name. If this parameter is specified, environment variable LOGICAL_APP_NAME must not be set. |
--buildnumber |
-N |
Optional. Any string that identifies the build. This option overrides the value provided by BUILD_NUMBER or BUILD_PREFIX environment variables. |
--joburl |
-J |
Optional. The URL to the job's build logs. |
An example command
In this an example, the command uploads a build record from a pipeline job.
idra --publishbuildrecord --branch=master --repositoryurl="https://github.com/oic/dlms.git" --commitid=dff7884b9168168d91cb9e5aec78e93db0fa80d9 --status=pass --logicalappname=appname --buildnumber=release:17Upload Test Results
Use the --publishtestresult command (alias -p) to upload results from these test types to DevOps Insights:
- Unit tests
- Code coverage tests
- Functional verification tests (FVT)
- Static security scans from IBM Application Security on Cloud
- Dynamic app scans from IBM Application Security on Cloud
- SonarQube results
- Vulnerability Advisor
Command syntax and options
idra --publishtestresult --filelocation=<test_result_file_location> --type=<test_result_type> --logicalappname=<appname> --buildnumber=<buildid> [--drilldownurl=<url>] [--env=<environment_name>] [--token=<token>]| Option | Alias | Description |
|---|---|---|
--filelocation |
-F |
Required. The location of the test results that you want to upload. This can be a single file, an entire directory, or multiple files that match a wildcard expression. |
--type |
-T |
Required. The type of test results that you want to upload. The value can be code for code coverage, unittest for unit tests, fvt for FVT tests, staticsecurityscan for static security scans, dynamicsecurityscan for dynamic app scans, sonarqube for SonarQube scans, and vulnerabilityadvisor for Vulnerability Advisor scans. In addition to these types you can also use a custom quality data set tag to upload test results of different type. |
--logicalappname |
-L |
Optional. The application's name. If this parameter is specified, environment variable LOGICAL_APP_NAME must not be set. |
--buildnumber |
-N |
Optional. Any string that identifies the build. This option overrides the value provided by BUILD_NUMBER or BUILD_PREFIX environment variables. |
--drilldownurl |
-U |
Optional. A URL where more information about the test results can be found. If this URL is invalid, the option is ignored. |
--env |
-E |
Optional. The environment name to associate with the test results. This option is ignored for unit tests, code coverage tests, and static security scans. |
--token |
-K |
Optional. This is a SonarQube token. This flag is valid only if the type specified is sonarqube. It is used by the CLI to pull additional information from the SonarQube server. |
An example command
In this example, the command uploads all JSON-formatted FVT results from a directory to DevOps Insights. Additionally, a test tool is declared, a test result URL is given, and a logical staging environment is provided. Specifying an environment is especially important when you want your results to be analyzed alongside other activities that take place in that same environment.
idra --publishtestresult --filelocation="tests/fvt/*.json" --type=fvt --logicalappname=appname --buildnumber=release:17 -drilldownurl="https://example.com/testinfo" --env="staging"Supported Test Formats
| Type of Test | tag | Supported Formats |
|---|---|---|
| Unit Test | unittest |
JUnit, xUnit, Mocha (mocha JSON reporter), KarmaMocha (karma-mocha-reporter) |
| Test Case | fvt |
JUnit, xUnit, Mocha (mocha JSON reporter), KarmaMocha (karma-mocha-reporter) |
| Code Coverage | code |
Cobertura (xml), lcov, Istanbul, JaCoCo |
| SonarQube | sonarqube |
SonarQube |
| Static Security Scan | staticsecurityscan |
IBM Application Security on Cloud |
| Dynamic Security Scan | dynamicsecurityscan |
IBM Application Security on Cloud |
| Vulnerability Advisor | vulnerabilityadvisor |
JSON output of the command ibmcloud cr va |
| Custom Data Sets - Tests | Custom Tag* | JUnit, xUnit, Mocha (mocha JSON reporter), KarmaMocha (karma-mocha-reporter) |
| Custom Data Sets - Code Coverage | Custom Tag* | Cobertura (xml), lcov, Istanbul, JaCoCo |
For golang gocov users, please convert to Cobertura using gocov XML(Cobertura) export
* For custom tags create a quality data set by using the user interface. Navigate to your toolchain, click on DevOps Insights card. Navigate to quality dashboard. Click on the Kebab menu which is to the top-right of the quality dashboard. Click on Configure Quality Data Sets
Upload SonarQube Results
To upload data from SonarQube, you must provide the SonarQube server token using --token. Your SonarQube server is not required to be accessible from your CI/CD tool.
After you run a scan using SonarQube, you can upload SonarQube results by running this command.
idra --publishtestresult --filelocation=.scannerwork/report-task.txt --type=sonarqube --token=234234434
In the above command, report-task.txt is a file generated during the SonarQube scan.
If this command fails for some reason, you can enable logs by setting IDRA_DEBUG environment variable to true.
If you are using self-signed certificates for SonarQube server, you will need to set export NODE_TLS_REJECT_UNAUTHORIZED=0
Upload Deployment Records
Use the --publishdeployrecord command (alias -d) to upload deployment records to DevOps Insights.
Command syntax and options
idra --publishdeployrecord --env=<environment_name> --status=<pass|fail> --joburl=<joburl> --logicalappname=<appname> --buildnumber=<buildid> [--appurl=<appurl>] [--deployableid=<deployable_id>]| Option | Alias | Description |
|---|---|---|
--env |
-E |
Required. The environment where the pipeline job deployed the app. |
--status |
-S |
Required. The deployment status. This value must be either pass or fail. |
--joburl |
-J |
Required. The URL to the job's deployment logs. |
--logicalappname |
-L |
Optional. The application's name. If this parameter is specified, environment variable LOGICAL_APP_NAME must not be set. |
--buildnumber |
-N |
Optional. Any string that identifies the build. This option overrides the value provided by BUILD_NUMBER or BUILD_PREFIX environment variables. |
--appurl |
-A |
Optional. The URL where the deployed app is running. |
--deployableid |
-I |
Optional. The deployed app's application ID. You can get the ID by running the Cloud Foundry CLI's cf env command. The application ID is listed under VCAP_APPLICATION. |
An example command
In this an example, the command uploads a deployment record from a pipeline job. The deployment is to the "staging" environment. The deployment is successful and the app is running at a specified URL.
idra --publishdeployrecord --env="staging" --status=pass --joburl="https://app.staging.example.com" --logicalappname=appname --buildnumber=release:17Making decisions about build quality using gate policies
To use this feature, create a quality policy by using the user interface. Navigate to your toolchain, click on DevOps Insights card. Then on the left menu, click on Policies.
Use the --evaluategate command (alias -g) to request a gate decision for a DevOps Insights policy in your pipeline. If the policy is satisfied, the pipeline continues running. If not, and you use the --forcedecision flag, the pipeline stops running.
Command syntax and options
idra --evaluategate --policy=<policyname> --logicalappname=<appname> --buildnumber=<buildid> [--forcedecision] [--type=<rule_type>]| Option | Alias | Description |
|---|---|---|
--policy |
-P |
Required. The name of the policy that the gate uses to make its decision. |
--logicalappname |
-L |
Optional. The application's name. If this parameter is specified, environment variable LOGICAL_APP_NAME must not be set. |
--buildnumber |
-N |
Optional. Any string that identifies the build. This option overrides the value provided by BUILD_NUMBER or BUILD_PREFIX environment variables. |
--forcedecision |
-D |
Optional. Set the value to true to cause the process to exit with an error code, if the policy evaluation fails. The value defaults to false if this option is not specified. |
--type |
-E |
Optional. A rule type to consider. If you include this option, only rules of this type are considered in the decision-making process. The value can be code for code coverage, unittest for unit tests, fvt for FVT tests, staticsecurityscan for static security scans, dynamicsecurityscan for dynamic app scans, and sonarqube for SonarQube scans. |
An example command
idra --evaluategate --policy="somepolicy" --logicalappname=appname --buildnumber=release:17Release History
(Nothing yet)