Pull Request Commenting
Note
This page uses the phrase "Pull Request," but in GitLab, the terminology is "Merge Request."
Lifecycle can add comments to pull requests in repositories configured for source control if the pull request introduces a new policy violation. This is accomplished by comparing the policy evaluation report of the pull request's source branch with:
the latest policy evaluation report for the common ancestor commit between the pull request's source and target branches, if such a report exists, or
the latest policy evaluation report for the pull request's base commit, which is the head commit of the pull request's target branch at the moment the pull request was created, if such a report exists, or
the latest available policy evaluation report for the source control configured default branch.
Note
The PR comment reports Threat Level Low, Medium, and High (2-10) and excludes the Informational level (0 and 1) to keep development focus on policy violations that require most attention.
There are several ways that pull request comments can be triggered:
In direct response to a policy evaluation.
Via a polling mechanism when a pull request is created after the policy evaluation occurs.
When a new commit is detected on the pull request feature branch (pull request monitoring).
|
The diagram below illustrates a branch-on-branch scenario to explain the full selection process of selecting the commits used to find the evaluation reports. However, pull requests are often for a feature branch stemming from the default branch.
 |
Prerequisites
Topic | Requirement |
|---|---|
Platform | Azure DevOps, Bitbucket Server, GitHub, GitLab |
Platform Version | Azure DevOps server must be 2019+ GitLab must be version 11+ |
Repository Visibility | Repositories must be set to Private. Exceptions are GitHub Enterprise, which permits any visibility, and GitLab, which allows for Private or Internal repositories. |
Token | You must have a token for your SCM system, and the token permissions must be configured correctly. |
Repository URL | Your Application inside IQ Server must have a repository URL, which must be entered as HTTPS, e.g. https://github.com/samplecount/samplerepo. The trailing .git is optional. If an ssh:// URL is used, Lifecycle will attempt to convert it to HTTPS. The .git:// style URL is not supported. Also, note that the same URL may be used for multiple applications. |
Pull Request Commenting option | The Pull Request Commenting option must be enabled. |
Default Branch | Your repository must have a default branch to compare against the Feature/Development branch, |
Feature/Development Branch | Your repository must have a feature/development branch to compare against the default branch. |
PR Summary Comment
IQ Server supports two approaches to PR commenting:
Base comments on customer-initiated policy evaluations via their CI system or other automation processes. This is the preferred approach and allows the customer to specify exactly what they want to be evaluated, including build artifacts, and they want them evaluated.
In cases where the customer has not set up CI/automation yet on a given repository, IQ Server will still detect pull requests and, noticing that these CI-initiated policy evaluations are not occurring, will initiate evaluations based on the contents of the repository. This will produce the necessary policy evaluations on which to base comments.
The diagram below illustrates these approaches to PR commenting in more detail. Please note the use of IQ Server stages.
The PR summary comment contains the summary of policy violations, affected components, and a description of the policy violations that were introduced by the PR. Suggested remediation for policy violations is included when available.
The 
and 
icons that can be seen next to the affected components are used to indicate whether a component is a direct or transitive dependency of the project. For Bitbucket, these are shown as text instead of icons.
If PR line comments are created (more details in the next section), links are added to the associated components for convenience. See an example below (for GitHub):
The policy violation details are collapsed by default for GitHub and GitLab. Developers can expand them as needed. For Bitbucket the policy violation details are fully expanded, by default.
There is also a section for fixed policy violations if the PR fixes preexisting policy violations.
PR comments are generated when:
Policy evaluation reports are run against new commits for an open PR and new policy violations have been detected or existing policy violations have been fixed
Newly detected PRs satisfy the prerequisites for PR commenting
The PR comment is designed to appear on the PR within a few moments after either of these conditions are satisfied.
Sonatype for SCM will update existing PR comments with the latest available information if the feature branch policy evaluation report is reevaluated, or if a new policy evaluation is run against the head commit for the PR.
Note
The pull request (PR) comment only reports policy violations introduced or fixed by the specific PR. These policy violations are determined by comparing two policy evaluation reports as described above in the Overview section. Note that the Summary and Application Report shown in the status check section includes all policy violations present in the latest policy evaluation for a given commit.
PR Line Comments
An attempt is made to create a line comment for each change that introduces new policy violations. That is not always possible due to the fact that line comments can be only created on lines available in the pull request's file diff. Additionally, sometimes there are multiple potential locations to comment on for a particular component. In those cases, the most relevant location is chosen for each component in order to keep the number of line comments to a minimum.
Identifying changes that introduce policy violations is ecosystem-specific. Currently, the PR line commenting feature is enabled for direct dependencies in Maven, Gradle, NPM, and Go projects. The strategy used to identify the location for the line comments is similar to the one used by the Automated Pull Request feature for version bumping.
Similar to the PR summary comment, PR line comments contain the affected component, the summary, and the description of the policy violations that were introduced by the change at the line of code to which the comment is attached. A component version that would remediate all policy violations is also included, if available. The policy violation details are expanded by default. An example is shown below (for GitHub):
 |
Breaking Changes
Breaking changes information will inform the developer about the amount of work required to perform the version upgrade whenever suggested remediations for policy violations are included.
In the screenshot below the line starting with a red square and "Multiple breaking changes" was added to the PR comment, based on the breaking changes information available for the suggested remediation.
 |
Transitive Solver
Note
If you experience challenges with seeing Transitive Solver recommendations, please refer to this Knowledgebase article.
Remediations that fix policy violations for the target component and its dependencies are preferred, if available, over remediations that fix policy violations for just the target component.
In the example shown below, upgrading to the suggested remediation will fix all policy violations for 'jackson-databind' and its dependencies:
 |
Running Policy Evaluations on Different Branches
We recommend that policy evaluations against the default branch use the ‘build' stage in the Sonatype Lifecycle and that all other branches' evaluations are performed against the ‘develop’ stage. The ‘develop’ stage is intended for policy evaluation snapshots, where changes to application dependencies and any associated policy violations are not tracked linearly. Because feature/development branches are volatile and vary across the features developed, using the ‘develop’ stage helps reduce noise by not showing these evaluations on the dashboard or your reports.
Enabling policy evaluations on feature branches will also increase the amount of disk space consumed by stored application composition reports. Sonatype IQ Server can be configured to automatically purge outdated reports.
Please review the actions in the Sonatype Lifecycle policy configuration if the feature branch policy evaluation is configured to use the 'develop' stage. By default, using the 'develop' stage will not result in a 'fail' action for any of the defined policies. Status checks and the associated branch protection only report failure if the policy evaluation triggers a 'fail' action.