Starting with the 1.33 release, IQ Server provides access to Success Metrics. These are reports that demonstrate the value of IQ Server, highlighting metrics like the average number of policy violations per application. They can also show the progress your organization is making by presenting changes in metrics over time.
To access the reports, click the Success Metrics icon in the toolbar. On the Success Metrics page, you have the option to add a new Success Metrics Report as well as a list of all Success Metrics Reports which have previously been added. Upon first usage, click the Add Report button near the top-right corner to open the Add Report modal window.
The "Aggregation data" section allows you to specify whether to include the most recent evaluations in Success Metrics Reports. By default, only full calendar weeks and months are included.
The "Scope" section lets you specify which applications and organizations to include in each report. By default, all applications to which you have access are included. Alternatively, you can specify a custom set.
Success Metrics Reports are defined on a per-user basis. A report created by one user will not be visible to others.
Once a new report is configured, it will appear on the Success Metrics page where it can be selected to navigate to a Success Metrics Report page generated from the associated set of applications. That page is shown below:
Larger data sets may take considerable time to load the first time you access a report. If the option to include most recent evaluations was selected, even the subsequent loads may take considerable time.
The first section is the 12 Week Policy Violation Activity chart that represents violation discovery and remediation activity over each week. Note that all remediation activity over the week is displayed (not just counts in the end of the week). For example if a violation was waived, and then the wavier was removed within the same week, both the waiver and new violation resulting from waiver removal will be displayed.
The second section is the 12 Week Open Violation Totals chart that shows the number of unresolved violations per policy type at a given point in time (in weekly intervals).
The third section is the Averages tile that shows the average number of evaluations done per month and the average number of violations per application, including how many of those violations are critical. That information is broken down even further by policy type - security, license, quality, or other.
Next is the Mean Time to Resolution (MTTR) by Month tile. It shows the average age of resolved violations for the last year. This information is further broken down by their threat level.
Second-to-last is the Applications tile that breaks down how many of the applications included in the report have violations and of those how many are critical.
The last section is the Components tile that breaks down which components are used across the most applications and which components have the most policy violations.
If needed, Success Metrics can be turned off by a System Administrator via the System Preferences menu in the toolbar.
Success Metrics Report Legend
The Success Metrics Report returns policy violation categories based on the specific policy constraints that triggered the violation.
Security Violation - Violation for which the policy constraint was on the Security Vulnerability Severity Score
License Violation - Violation for which the policy constraint was on the License or License Threat Group
Quality Violation - Violation for which the policy constraint was on the Age or Relative Popularity of a component
Other Violation - Violation for which the policy constraint was something other than a Security, License, or Quality constraint, such as a label
It also aggregates violations based on policy threat level as seen in the following table:
|Threat Aggregation||Policy Threat Level|
Success Metrics in Action
A demo from our first release, IQ Server 1.33 in July 2017, showing the core functionality. Enhancements have been made in subsequent releases, see the 2017 Release Notes and 2018 Release Notes for details.