Sonatype for Azure DevOps
The Sonatype for Azure DevOps extension integrates with the Azure DevOps pipeline to run policy evaluations in the build workspace. It adds a new step within the build, during which applications are scanned by Sonatype IQ Server to identify any open-source security, license, or quality policy violations. It can be configured to fail the build or generate a warning. This allows the build maintainers to understand the reasons for build failures and plan a remediation strategy.
The Sonatype for Azure DevOps integration is available on the Visual Studio Marketplace.
Note
This extension wraps the Nexus IQ CLI (nexus-iq-cli). Additional documentation, including the mapping of extension version to Lifecycle version, is available on the marketplace.
Main Features
Perform a Sonatype IQ Server policy evaluation on files in the build workspace.
Display scan results within Azure DevOps pipeline report.
Provide a link to a comprehensive Sonatype Lifecycle policy evaluation report indicating violation details and remediation recommendations.
Sonatype for Azure DevOps
The organization ID parameter requires at least version 1.5.0 of the plugin and IQ Server 143 or higher
Sonatype for Azure DevOps Version | IQ Server Version | Supported Environment | Java Runtime |
---|---|---|---|
2.0.0 and newer | 180 | Azure DevOps Services (hosted) Azure DevOps Server 2019 | Java 17 |
1.2.0 to 1.7.21 | 66 and higher | Azure DevOps Services (hosted) Azure DevOps Server 2019 | JDK 8, JDK 11 |
1.0.0 to 1.1.0 | 66 and higher | Azure DevOps Services (hosted) | JDK 8, JDK 11 |
Installation
Sign in to your Azure DevOps account at https://dev.azure.com.
Click on "Organization Settings" button at the bottom left corner:
From the left-hand side menu choose "Extensions":
Click "Browse Marketplace" button in the top right corner of the page:
You will be redirected to the Azure DevOps Extensions marketplace.
Enter "Sonatype IQ" in the search box to find the Sonatype for Azure DevOps.
Click on the Sonatype for Azure DevOps tile, to access the extension's home page.
Click on the "Get it free" button, to start the installation process.
Follow the on-screen steps to complete the installation
Sonatype for Azure DevOps icon will now be visible on the Organization Settings page as below:
Java Runtime Requirements
Note
The Sonatype for Azure DevOps extension requires a Java runtime to be installed on the Azure DevOps agents.
For extension versions up to 1.7.21 (included), Java 8 is the minimum required runtime version. Azure DevOps agents usually include either Java 8 or 11 by default, so no additional configuration is needed for the extension.
Starting with version 2.0.0 of the extension, Java 17 is the minimum required runtime version. If your Azure DevOps agents are set up to use Java 17 by default, no additional configuration is needed for the extension. Otherwise, you'll need to configure your pipeline to use Java 17. There are a couple of ways to do this:
Modify your pipeline's YAML file:
Add the following task to install the correct Java version:
- task: JavaToolInstaller@0 inputs: versionSpec: '17' jdkArchitectureOption: 'x64' jdkSourceOption: 'PreInstalled'
This will ensure the pipeline uses Java 17 for all subsequent tasks.
Configure through the Azure Pipelines UI:
Click on the "Java tool installer" task in your pipeline.
Set the JDK version field to 17 and the other options as shown below:
After the NexusIqPipelineTask
, you can reset the Java version to your preferred version for subsequent tasks in your pipeline using the same method described above.
Updating to a newer version of Sonatype for Azure DevOps
By default, if a new version of the extension has been released in the Azure DevOps Marketplace, it is updated in your pipeline automatically.
For some cases, when there is a change in access permissions for the extension, manual user approval will be required to install the update. Follow the steps below:
Go to "Organization settings" → "Extensions" (the same menu as the "Installation" section) → "Installed" extension tab
An "Action required" message appears.
Click on Sonatype for Azure DevOps.
Click on the "Review" button.
A menu with scope changes appears.
Click on the "Authorize" button.
Sonatype for Azure DevOps is now updated to the new version.
Configuration
1. Sonatype IQ Service Configuration
Open your project in Azure DevOps.
Go to the "Project Settings" in the bottom left corner.
Choose "Service Connections" from the menu:
Click "New Service Connection" button and choose "Sonatype IQ" from the list of connections:
Enter the URL and credentials for your Sonatype IQ server in the pop-up window and click the "OK" button:
2. YAML Pipeline Configuration
Select "Pipelines" → "Builds" from the left-hand side menu:
Create a "New build pipeline" if one does not exist:
The following message will appear:
Choose a repository where you host your project from the list above and follow the steps to create the pipeline.
To edit an existing build pipeline, click on "Edit":
An azure-pipelines.yml file will be opened.
Search for "Sonatype IQ" in the search box on the right-hand side of the page:
Click on found Sonatype IQ task extension to view a configuration screen:
Configure the Sonatype IQ task on the configuration page, as below:
Sonatype IQ service connection (configured previously)
Organization ID (optional): ID of the organization under which the application will be created if the automatic application creation is enabled and the application does not exist.
Application ID: ID of the application to evaluate against as configured in Sonatype IQ
Stage: Stage in IQ for the evaluation
Scan Targets : Targets to perform policy evaluation are listed as comma-separated glob patterns (more details below). E.g.
**/*.jar, **/*.json
Use ${Pipeline.Workspace} as the base folder for scanning: If checked, the above scan targets are evaluated against the ${Pipeline.Workspace} folder and its descendants (more details below).
Ignore IQ Server's system errors: Controls the pipeline outcome when the scan or evaluation fails to produce results for some (possibly intermittent) connection problem. Usually such a failure would result in a FAILURE of the pipeline, but ignoring system errors allows it to show as a WARNING on the pipeline. DNS and network connection failures can be ignored, but misconfiguration of Stages or Application IDs cannot be ignored.
Ignore IQ Server's Scanning errors: Controls the pipeline outcome when there are scanning errors such as malformed files. Scanning errors result in a FAILURE of the pipeline if they are not ignored. Ignoring scanning errors will lead to pipeline finishing in a WARNING state instead of a FAILURE in case of scanning errors.
Enable Debug Logging: Enables debug logging for IQ policy evaluation. This should be used only to troubleshoot any problem you may have when scanning a repository.
Java System Properties (Advanced): Command line arguments to alter the behavior of the JVM, for example:
-Djava.net.useSystemProxies=false
Click on the "Add" button, to update the YAML file with the above inputs.
Save your build by clicking the "Save" button in the right top corner of the page:
Configuration of Sonatype for Azure DevOps is now complete and ready to run policy evaluations on your builds.
Scan Targets
The Scan Targets field allows for a fine-grained selection of the files on which the policy evaluation is performed. Scan targets are listed as comma-separated glob patterns. The supported glob patterns are described in the file matching patterns reference section of the Azure DevOps documentation.
The provided glob patterns are evaluated as follows:
If Use ${Pipeline.Workspace} as the base folder for scanning is checked, they are evaluated against the files located in the pipeline workspace directory (and its descendants)
Otherwise, they are evaluated against the files located in two directories:
${System.DefaultWorkingDirectory} - the system default working directory - the local path on the agent where your source code files are downloaded. For example:
c:\agent_work\1\s
${Build.ArtifactStagingDirectory} - the build artifact staging directory - the local path on the agent where any artifacts are copied to before being pushed to their destination. For example:
c:\agent_work\1\a
The union of all the files that match the provided glob patterns is used as target for the policy evaluation.
For example, the following scan target will consider for evaluation all the JAR and POM files found anywhere under the two directories mentioned above: **/*.jar, **/pom.xml
The Azure DevOps glob patterns also support exclude patterns (all above examples are include patterns). The exclude patterns start with a '!
' character and must be specified after all other include patterns. For example, the following scan target will consider for evaluation all files except those under the '.git
' directory: **, !**/.git/**
3. Optional: Using the classic editor to create a pipeline without YAML
Create a new pipeline from the left menu, Pipelines → Build
Click on the "Use the classic editor to create a pipeline without YAML" link at the bottom.
3. After creating a pipeline in the classic editor, add a Sonatype IQ pipeline task by clicking on the "+" icon on the "Agent job" tile:
4. Configure the Sonatype IQ task.
5. Enable the "Allow scripts to access the OAuth token" checkbox under the "Agent job" tile to enable widgets correctly, with settings as below:
Evaluating Policies
You can now start running policy evaluations as a part of your build.
Note that "NexusIqPipelineTask" appears as one of the tasks in the list, while running a build:
Accessing/Viewing Results
Open NexusIqPipelineTask to view a console output with the results of the evaluation:
The console output contains a summary of the policy evaluation and a link to the detailed report.
Click on the link on the console output to view the detailed report in Sonatype IQ Server.
Select the Sonatype IQ Build Report tab on the build for a detailed report with all the components and their correspondent violations:
Select the Sonatype IQ Summary Report tab on the build to see a summary report for the policy evaluation for the scanned components:
Add dashboard widgets for Sonatype IQ
For ease of use, the following widgets for Sonatype IQ can be added to Azure DevOps dashboard.
Sonatype IQ Policy Evaluation widget: shows the policy evaluation results for the latest build.
Trends for Sonatype IQ Policy Evaluation: shows a historical trend of Sonatype IQ Policy evaluations of the last 5 builds.
How to add Sonatype IQ widgets to Azure DevOps Dashboard:
Go to the "Overview" → "Dashboards" and click the "Edit" button.
On the right-hand side, under "Add Widget" search for the "Sonatype IQ"
Select the appropriate widget and click the "Add" button at the bottom right corner of the page. Click the "Done editing" button.
Sonatype IQ widget now displays on the dashboard.
A sample screenshot for Sonatype IQ Policy Evaluation widget showing the results summary for the latest build:
A sample screenshot for Trends for Sonatype IQ Policy Evaluation widget showing the historical summary for the last 5 builds:
Running Sonatype IQ in Azure Self-hosted Agents
If you use an HTTP proxy within your infrastructure and use Azure self-host build agents, then the Azure DevOps agent options for a proxy can be specified and then will be automatically used while connecting to IQ. Refer: https://docs.microsoft.com/en-us/azure/devops/pipelines/agents/proxy?view=azure-devops&tabs=windows
In the Azure provided sample command:
./config.sh --proxyurl http://127.0.0.1:8888 --proxyusername "myuser" --proxypassword "mypass"
This would appear in the scan output as it is passed through to the IQ scan client:
... -p 127.0.0.1:8888 -U myuser:*** ...
Tips and Tricks
The Azure DevOps extension uses the Sonatype IQ CLI to perform the scan. Part of the IQ CLI scan process involves using git to determine the repository URL and commit hash. If native git is available on the agent then that is used, otherwise, jgit (java-git) is used. If jgit is used it tries to create some config files in the $HOME
directory of the current user. If it does not have sufficient permissions to do that you may see logs on 'ERROR' level. These are not critical errors to the scan and can be ignored. To address the error, ensure native git is available or for jgit set the XDG_CONFIG_HOME
environment variable to a directory that is writable by the build agent user.