Upgrading Nexus Repository Using Rolling Upgrades in a High Availability Environment
As of Sonatype Nexus Repository 3.71.0, those running in a high availability (HA) environment can enable the ability to update Nexus Repository without downtime (i.e., rolling upgrade, zero-downtime upgrade). Note that 3.72.0 is the first version to which you can upgrade with the zero-downtime method.
Rolling upgrades involve first stopping and upgrading each node and then upgrading the database schema. While an individual node is stopped, the other active nodes will continue to handle traffic, resulting in zero downtime.
Limitations for Rolling Upgrades
We may occasionally have limitations on eligible version upgrade ranges. Attempting to exceed the upgrade windows will result in new nodes failing to start. Before upgrading, see our supported HA rolling upgrade paths to see if you will need to upgrade to an intermediate version before going to your final version.
Rolling upgrades result in nodes being on different versions (mixed mode) temporarily. Running in mixed mode outside of an active upgrade is not supported and can cause Nexus Repository to become unstable and behave in unexpected ways.
We highly recommend that you complete your upgrade process (nodes and database) in one limited time period to minimize the amount of time Nexus Repository nodes are on different versions.
If you attempt to upgrade without downtime without enabling rolling upgrades on one of your nodes, you will encounter an error trying to start the upgraded node when Nexus Repository detects mixed mode without rolling upgrades enabled. Enable rolling upgrades on all nodes to avoid errors.
Requests that were already in progress on a specific node when it is shut down may fail; attempt the request again so that the load balancer can send it to one of the active nodes.
New features will not be available until the final step (database schema upgrade) is complete.
While database schema migration is active, the cluster will not allow additional nodes to join; if you attempt to add nodes during schema migration, these nodes will stop with an error code and logged errors.
Upgrading the database schema is a one-way process. Once you proceed with finalizing the upgrade (updating the database schema), you will not be able to roll it back.
Prerequisites for Rolling Upgrades
To use rolling upgrades with zero downtime, you must meet the following prerequisite requirements:
You must be using Sonatype Nexus Repository Pro in an HA environment with a PostgreSQL database.
You must be on at least Sonatype Nexus Repository 3.71.0; the first version you can upgrade to with zero downtime will be 3.72.0.
You must have configured sticky sessions on your load balancer for user interface assets to avoid mixing assets from different versions of Nexus Repository. Without sticky sessions, you may see UI inconsistencies during the upgrade process due to changes made between versions. NGINX provides its own documentation on enabling session persistence .
You must be in compliance with our supported HA rolling upgrade paths; this may require you to go to an intermediate version before going to your final version.
We highly recommend you back up your data before performing an upgrade. Once you proceed with finalizing the upgrade (updating the database schema), you will not be able to roll it back.
If upgrading using the Helm chart, you must use the NXRM HA Helm chart version 71.0.0 or above.
You must explicitly enable zero-downtime upgrades on each node as described below before upgrading.
Prerequisite Step for Rolling Upgrades: Enable Zero-Downtime Upgrades
Before upgrading, you must enable zero-downtime upgrades on each node. You must enable this feature on your starting version (at least 3.71.0) before upgrading each node. After enabling the feature on each node, restart each instance one by one before proceeding to migrating to 3.72.0+ with zero downtime. For example, enable the feature using one of the methods below in 3.71.0, restart each node, then proceed to upgrading to 3.72.0+.
Helm Chart
If using the Helm chart, update your values.yaml to set zeroDowntimeEnabled to true.
OpenShift
Update your Nexus Repository custom resource definition (called “NexusRepo”) to set the NEXUS_ZERO_DOWNTIME_ENABLED environment variable to true.
Docker and Kubernetes Environments
Update your statefulset.yaml to set the NEXUS_ZERO_DOWNTIME_ENABLED environment variable to true.
Manual
Set the NEXUS_ZERO_DOWNTIME_ENABLED environment variable to true or modify the $data-dir/sonatype-work/nexus3/etc/nexus.properties in each instance to add nexus.zero.downtime.enabled = true.
Phase 1 - Upgrade Each Node
In phase 1, you will take each of your nodes offline individually, upgrade them one at a time, then restart them at which point they will be using the new Nexus Repository version.
During this time, the load balancer will continue to distribute requests to the active nodes in your cluster. Your database will continue to use the old schema, and new features will not be available until the next phase (migrate database schema) completes.
 |
Note
While upgrading nodes and for a short period afterwards, administrators may see a warning indicating that the cluster is running in mixed mode.
Running in mixed mode outside of an active upgrade is not supported and can cause Nexus Repository to become unstable and behave in unexpected ways. We highly recommend that you complete your upgrade process (nodes and database) in one limited time period to minimize the amount of time Nexus Repository nodes are on different versions.
Also, if you do not enable rolling upgrades on every node, any node without the feature enabled will fail to start when Nexus Repository identifies that other nodes are on different versions.
You can upgrade your nodes manually by replacing/upgrading each instance with your existing upgrade process. Or, follow our instructions for upgrading Nexus Repository in a Kubernetes environment.
Tip
Once all nodes are on the same version, you may want to perform validation to identify any potential issues in your workflows. Once you proceed to upgrading your database schema, you will not be able to return to your previous version.
While rolling upgrades do require that you enable HA, you could choose to use clustering to upgrade a single-node instance without downtime.
To do this, enable clustering on your original node as described in our manual high availability deployment option documentation. Then, add a second instance that is running on the new Nexus Repository version. Finally, stop your original node.
Note that there is additional overhead when using Nexus Repository in HA (clustered) mode even when the cluster only contains a single node. Carefully review the system requirements for HA deployments.
Phase 2 - Finalize Upgrade with Database Schema Migration (if Applicable)
After all nodes are on the same version and you have validated your workflows, you can finalize your upgrade by migrating your database schema if applicable; not all releases will require a database schema migration. When a database schema migration is required for a Nexus Repository upgrade, you will not have access to new features included in new releases until this phase completes successfully. In some circumstances, asynchronous tasks may also need to finish before features become available.
Note
Once you initiate the database schema migration, you will not be able to return to your previous version.
Finalize Upgrade via the User Interface
If your upgrade requires a database schema migration, administrators with the nexus:* permission will see a banner at the bottom of the screen. Note: there may be a delay of up to 10 minutes before the banner appears.
 |
Select the Finalize Upgrade button in the Upgrade Pending banner; this will open a dialogue asking if you are ready to proceed with database schema migration.
Select the Acknowledge checkbox to acknowledge that you have backed up your database; then, select Continue to proceed with database schema migration.
The banner at the bottom of the screen will show that an upgrade is in progress; when it completes successfully, it will display an Upgrade Complete banner.
If you encounter an Upgrade Failure banner, the cause will be captured in the audit log.
Reload the user interface. If the browser retrieved assets from the old version, reloading the interface may be required to use new features.
Finalize Migration with REST APIs
REST API users can finalize their upgrade via the REST APIs listed in the table below.
API | Description |
|---|---|
GET /service/rest/v1/clustered/upgrade-database-schema | Returns an object that describes the current state:
|
POST /service/rest/v1/clustered/upgrade-database-schema | When called, this starts the database migration process asynchronously. Unsuccessful response codes indicate that a precondition failed and administrators should view the error message. |
DELETE /service/rest/v1/clustered/upgrade-database-schema | If an error occurs and a retry is possible, use this endpoint to clear the error state. |
The process for finalizing your upgrade and migrating your database schema is as follows:
Use a GET request to verify that the state is MIGRATION_REQUIRED
Use a POST request to start a migration
Poll the GET endpoint; it will return one of the following statuses:
MIGRATION_SUCCEED - indicates successful database schema migration. Send a DELETE request to clear the status.
MIGRATION_FAILED_RETRYABLE - indicates that a precondition for migration failed; consult the accompanying error message to identify and remediate the cause. After remediation, use a POST to try again.
MIGRATION_FAILED - indicates that an error occurred, and we do not believe it can be rectified without Support team assistance.
After successful database schema migration, your upgrade is complete.
Supported Rolling Upgrade Paths
We may occasionally have limitations on eligible version upgrade ranges. Attempting to exceed the upgrade windows will result in new nodes failing to start. Follow this section to stay up to date on any required intermediate versions.
Starting Version | Can Upgrade with Zero Downtime to These Versions |
|---|---|
3.71.0 | 3.72.0+ |