Upgrading the IQ Server to Version 1.45

If you have been directed here from your IQ Server's error log and this is the first time you see these upgrade instructions, we suggest you abort the server upgrade you were about to commence and revert to your previous IQ Server version.

The upgrade procedure outlined below requires preparation and should not be rushed. Read these instructions carefully and only retry the server upgrade once you have had a chance to make the necessary preparations.

Starting with version 1.45 or greater, IQ Server uses a more effective and condensed format to store policy violation data. Depending on the database size of your installation and the hardware you use, upgrading your database to this new format can take notable time (up to hours in worst case scenarios). These instructions are meant to help you plan and reduce the downtime of your IQ Server while the upgrade is carried out.

Preparing for the Upgrade

Upgrade to IQ Server 1.44 First

If your current installation is several versions behind and not already running IQ Server version 1.44 you should update to that version first. This reduces the number and duration of upgrade tasks both the server and its administrator have to consider when later upgrading to version 1.45 or greater. In particular, you should perform the update to version 1.44 during a different maintenance window than the update to version 1.45 or greater.

Version 1.44 can be obtained from these links

Ensure Sufficient Free Disk Space

Until the upgrade is fully completed, the IQ Server's database will temporarily contain policy violation data in both the old and new storage format. As a result, the database will generally grow while the upgrade is in progress. To avoid running out of disk space during the upgrade, we recommend having at least as much free space as the database currently occupies on the disk hosting it. You can determine the amount needed by looking at how large the database file is, e.g. sonatype-work/clm-server/data/ods.h2.db (see your server's config.yml and its sonatypeWork option for the path applicable to your installation).

Perform a Test Run of the Upgrade to 1.45 or greater in a Staging Environment

Given the number of unknown variables about your particular installation, it is impossible to predict the duration of the upgrade and the maintenance time you should expect. By performing a test run of the upgrade in a non-production environment, you can learn about its approximate duration and plan your maintenance for the real upgrade accordingly.

For that test run to be representative, it should be executed using a recent backup/clone of your installation and on hardware closely comparable to your production system. Especially the I/O performance of the storage holding the server's database (sonatype-work/clm-server/data) is a key factor for the upgrade duration.

The test run also helps you estimate the resulting database size and verify that your storage system has enough free space to accommodate it.

Compact the Database Before the Upgrade

A long-lived database contains some percentage of empty space to support its efficient growth. If your server's database (sonatype-work/clm-server/data/ods.h2.db) is several gigabytes large, this percentage of empty space can amount to gigabytes as well. Removing this empty space, a process called compacting, can reduce the upgrade duration for IQ Server 1.45 or greater.

To compact your server's database:

  1. Shut down your IQ Server
  2. Execute the command java -jar nexus-iq-server-1.44.0-01.jar compact-db config.yml or equivalent for your environment
  3. You may restart your IQ Server once the above command has finished

Compacting the database is an I/O heavy operation and, depending on the size of your database and the storage performance, can take several minutes or longer. To help plan the needed maintenance window, we suggest you perform a test run of this process in a staging environment on comparable hardware using a recent backup of your installation.

Over time and with continued use of IQ Server, the database will again accumulate some empty space. In order to benefit the upgrade to IQ Server 1.45 or greater, compacting the database should therefore be done not too far in advance of the upgrade, e.g. within a week prior to the 1.45 or greater upgrade. If easier for you to implement, you can perform this step right before upgrading to version 1.45 or greater. We primarily mention it as a separate preparation step to allow you to organize the upgrade over several smaller maintenance windows instead of one long maintenance if desired.

Use a Fast Disk/Storage

The upgrade to IQ Server 1.45 or greater is an I/O intensive task. The performance of the storage device holding the server's database (sonatype-work/clm-server/data) is a key factor for the upgrade duration. Depending on your environment, it can be beneficial to at least temporarily employ a faster disk for IQ Server and its upgrade.

Internal Reference Numbers

Our internal benchmarks may provide some rough guidance on expected upgrade times, though as noted above the upgrade complexity is highly dependant on your particular installation.

Database Size Upgrade Time SSD Upgrade Time HDD
3.3 GB ~ 1 minute ~ 10 minutes
7.2 GB ~ 2 minutes ~ 15 minutes
8.9 GB ~ 3 minutes ~ 40 minutes
19.0 GB ~ 5 minutes ~ 60 minutes
67.0 GB ~ 60 minutes ~ 15 hours

Performing the Upgrade

Take a Fresh Backup Before the Upgrade

This should be second nature for the diligent reader of our general upgrade instructions but is worth emphasizing: Backing up the IQ Server before the upgrade ensures you have something to fall back to in the unlikely event something goes haywire.

Update the Server's config.yml to Consent to the Upgrade

To save unsuspecting administrators from surprises, IQ Server 1.45 or greater requires a change to the configuration file to signal the implications of the upgrade are understood and the necessary preparations have been made. So for the aforementioned test runs and the actual upgrade of your production system, the following line needs to be inserted into the config.yml file used to launch IQ Server 1.45 or greater:

# Confirm you have read the upgrade instructions and come prepared
consentToUpgradeToVersion_1_45: true

If the line is missing, IQ Server will refuse to start with an error in its log, directing to these upgrade instructions.

Once the upgrade to version 1.45 or greater has completed, you can remove that line from the config.yml again.

Compact the Database After the Upgrade

Once IQ Server has completed the upgrade to 1.45 and is up and running again, its database will still roughly have the same size as before. To cash in on its more efficient storage format for policy violation data, its database needs to be compacted (again) as described earlier. This task can be done at a later time like the next available maintenance window or right after the upgrade, whatever works best with your maintenance schedule.

To compact your server's database:

  1. Shut down your IQ Server
  2. Execute the command java -jar nexus-iq-server-1.45.0-01.jar compact-db config.yml or equivalent for your environment or version.
  3. You may restart your IQ Server once the above command has finished

You might choose to skip compacting the database before the upgrade, e.g. because your test run found the upgrade was already sufficiently quick, but we strongly recommend to compact the database after the upgrade to 1.45 or greater. In our testing, we observed size reductions of 50% and more.