How the Doctor Scores

Once you've had a few Software Bills of Materials (SBOMs) evaluated by BOM Doctor, you may ask, "How does the Doctor score the health of its patients?"

That's a good question. Just like a real doctor asks a lot of questions at an annual-check up, BOM Doctor evaluates SBOMs on a number of factors.

Criteria

Grading components and component versions is complex. However, we can summarize with the following broad statements.

Fewer vulnerabilities is preferable to many vulnerabilities, and lower-risk vulnerabilities are preferable to high-risk vulnerabilities.
- Component vulnerabilities are a serious health risk to your application. But, just like you'd probably rather have a cough than a broken bone, fewer and less-severe vulnerabilities are better than many and high-risk vulnerabilities.
Non-restrictive licenses are preferable to restrictive licenses.
- Licensing is a key part of the open-source ecosystem. Restrictive licenses are important, but may not be compatible with your app's purpose or your company's business goals. In general, non-restrictive licenses are preferable because they place fewer requirements on component's user – in this case, you.
High popularity is preferable to low popularity.

It's important to understand that BOM Doctor only suggests other versions of the components already in your SBOM. Sometimes, the best way to improve your application's score is to switch components entirely, but BOM Doctor won't suggest these sorts of changes.

Diagnoses from the Crowd

Popularity is a complex topic, but BOM Doctor thinks it is important. Why?

The key is to understand what popularity means in the open-source community.

Popular components are more heavily scrutinized by your peers, and are probably undergoing active development, or at least regular maintenance.
Every component can bring in risk, regardless of its popularity, but a popular component is more likely to be patched in case of a critical vulnerability being found or introduced.

Popularity is also a phenomenal indicator for Breaking Changes.

A Breaking Change is a change to a component's inner workings that might require additional developer time to resolve.
A component that's very popular amongst your peers indicates that it probably doesn't introduce any Breaking Changes or other unintended side effects, which means that it's a good choice.

Picking the Best Version

So, how does BOM Doctor select a best version from amongst all available versions?

As you probably know by now, it's complex. But by tracking a huge number of component releases, Sonatype's Research teams have identified some key rules that make the best version clear.

Don’t choose a pre-release unless it solves a bigger problem.
- Pre-releases haven't been vetted by the community, and may contain issues or carry unintended side effects. Avoid these.
- However, if the pre-release solves a major problem or patches a super-severe vulnerability, then moving to the pre-release is a viable choice.
Don’t choose a known bad release.
- As stewards of Maven Central, Sonatype knows that there are still organizations that are ingesting versions of log4j with the infamous RCE exploit.
Choose a version with the least security risk.
Choose a version with the least legal risk.
Choose a version that is relatively popular.
Choose a newer version.
- If everything else is equal, choose the newer version. Newer versions could introduce new problems, but they're also more likely to have patched old ones.

Visualizing Urgency

Now that you understand a little about how BOM Doctor scores your application, let's put it into context. By tracking these criteria across many releases, Sonatype's Research team can broadly categorize any component's individual versions into four urgency zones.

Image Caption

Example of an urgency graph, with version number on the X axis and component score on the Y axis. The bars over the version numbers are gray, orange, light green, or dark green, and indicate four different levels of urgency. The left side of the graph is mostly gray, with the orange and light green appearing mostly to the right. Version 5.3.19 is the only bar colored dark green.

In the graph above, we're looking at the component org.springframework:spring-core between versions 1.0 and 5.3.8. This graph was built sometime in Summer of 2021. The table below summarizes the meaning of each color-coded division.

Zone	Urgency Level	Comment
Reactive Zone	High Urgency	Very likely to go bad soon. Teams using these versions need to jump immediately, regardless of how hard it is to upgrade.
Borderline Proactive Zone	Medium Urgency	Okay for now. Teams using these versions should consider jumping soon, but can wait a bit until the upgrade path is easy.
Proactive Zone	Low Urgency	Almost the best version possible. Teams using these versions can afford to wait until the upgrade path is easy.
Best Version	No Urgency	No upgrade necessary!

To summarize, BOM Doctor wants you to be on the Best Version at all times – and that's how it scores your components!

Learn More

Much of what's reviewed here is discussed in more detail in our 2021 and 2022 State of the Software Supply Chain Reports.