Skip to main content

Option 4 - High Availability Deployment in Azure

Note

Only available in Sonatype Nexus Repository Pro. Interested in a free trial? Start here.

Warning

Sonatype Nexus Repository High Availability deployments should be fully deployed and tested in a development environment before attempting to deploy in production. Improper deployment in a production environment can result in critical data loss.

The reference architecture and steps below provide detailed information on setting up a highly available Nexus Repository deployment in Azure.

NXRM_Azure_HA.png

Important Terms

Before proceeding, you should know these important terms:

  • Node - A virtual or physical machine

  • Pod - A group of one or more containers with shared storage and network resources and a specification for how to run containers

  • Container - A package with the program to execute (Nexus Repository) and everything required for it to run (e.g., code, runtime, system libraries)

  • Instance - An instance of Sonatype Nexus Repository

Use Cases

HA in Azure is designed to protect against the following scenarios:

  • An Availability Zone (AZ) outage within a single region

  • Azure Virtual Machine failure

  • A Sonatype Nexus Repository instance failure

It also allows you to auto scale instances up and down.

You would use this architecture if you fit the following profiles:

  • You are a Nexus Repository Pro user looking for a highly available Nexus Repository deployment option in Azure in order to reduce downtime

  • You would like to achieve automatic failover and fault tolerance as part of your deployment goals

  • You already have an Azure Kubernetes Service (AKS) cluster set up as part of your deployment pipeline for your other in-house applications and would like to leverage the same for your Nexus Repository deployment

  • You have migrated or set up Nexus Repository with an Azure PostgreSQL database and want to fully reap the benefits of an externalized database setup

Requirements

Limitations

  1. The ability to upgrade with zero downtime (rolling upgrades) is only available from release 3.71.0 forward; 3.72.0 is the first release you can upgrade to without shutting down all active nodes. See our HA upgrade documentation for details.

  2. We do not recommend deploying HA clusters across regions due to the large number of database interactions is involved in HA.

  3. In the absence of a distributed solution for logging, Azure Disks are required to persist the log files that are needed for support zip creation.

Setting Up the Infrastructure

Note

Unless otherwise specified, all steps detailed below are still required if you are planning to use the HA/Resiliency Helm Chart (GitHub, ArtifactHub).

Step 1 - Azure AKS Cluster

The first thing you must do is create a resource group. A resource group is a container that holds related resources for an Azure solution. In this case, everything we are about to set up will be contained within this new resource group that you create.

  1. Follow Microsoft's documentation to create a resource group from the Azure portal.

  2. Put the resource group in the same region you intend to setup your all your resources.

  3. Then, you can follow Microsoft's documentationfor creating an AKS cluster.

    1. If you plan to use Azure Monitor (described in step 6), ensure you enable it when creating your AKS cluster.

    2. If you plan to use Azure Key Vault (described in step 4), you can also enable the Azure Key Vault Secret Store CSI Driver add-on following Microsoft's documentation. Ensure that you provide the AKS node(s) on which Sonatype Nexus Repository will run with appropriate permissions for accessing the Key Vault as described in Microsoft's documentation.

      1. The CSI Secrets Store Driver is a Kubernetes-specific plugin that will allow the Kubernetes node on which Sonatype Nexus Repository runs to retrieve the license and database user/password information from the key vault.

Step 2 - Azure PostgreSQL

You will now configure your database for storing Sonatype Nexus Repository configurations and component metadata. We recommend Azure's Flexible Server option.

  1. Follow Microsoft's documentation for creating an Azure database for PostgreSQL server.

  2. When setting up the database, select the resource group you created when setting up your AKS cluster.

  3. Select the same region as the resource group you created.

You can also use the Azure CLI to create a database callednexus using Microsoft's documentation as a reference. You may use an alternative name if desired. (See Microsoft's documentation for installing the Azure CLI.)

Step 3 - Ingress Controller

The next step is to configure an Ingress Controller to provide reverse proxy, configurable traffic routing, and TLS termination for Kubernetes services.

  1. Follow Microsoft's documentation for setting up an ingress controller in AKS.

  2. Optionally, configure the Ingress Controller to associate a static IP address with the Sonatype Nexus Repository pod. This way, if the Nexus Repository pod restarts, it can still be accessed through the same IP address.

Step 4 - License Management

There are a few options you can use for license management:

  1. Azure Key Vault and CSI Secrets Store Driver - You can use this option whether you're using the HA/Resiliency Helm Chart or manually deploying YAML files.

  2. Configure License in Helm Chart - This option is for those using the HA/Resiliency Helm Chart but not using Azure Key Vault.

  3. License Configuration Mapping YAML - This option is for those who are neither using the HA/Resiliency Helm Chart nor Azure Key Vault.

Option 1 - Azure Key Vault and CSI Secrets Store Driver (Optional)

Azure Key Vault is one of the options for storing secrets like your Sonatype Nexus Repository license and database credentials. In the event of a failover, Key Vault can retrieve the Nexus Repository license when the new Nexus Repository container starts. This way, your Nexus Repository always starts in Pro mode.

External Secrets Operator (Recommended)

To use an external secrets operator, locate the external secrets portion of your values.yaml (shown in the code block below) and complete the steps detailed in the Helm chart README

externalsecrets:
  enabled: false
  secretstore:
    name: nexus-secret-store
    spec:
      provider:
#        aws:
#          service: SecretsManager
#          region: us-east-1
#          auth:
#            jwt:
#              serviceAccountRef:
#                name: nexus-repository-deployment-sa # Use the same service account name as specified in serviceAccount.name
# Example for Azure
#    spec:
#      provider:
#        azurekv:
#          authType: WorkloadIdentity
#          vaultUrl: "https://xx-xxxx-xx.vault.azure.net"
#          serviceAccountRef:
#            name: nexus-repository-deployment-sa # Use the same service account name as specified in serviceAccount.name
  secrets:
    nexusSecret:
      enabled: false
      refreshInterval: 1h
      providerSecretName: nexus-secret.json
      decodingStrategy: null # For Azure set to Base64
    database:
      refreshInterval: 1h
      valueIsJson: false
      providerSecretName: dbSecretName # The name of the AWS SecretsManager/Azure KeyVault/etc. secret
      dbUserKey: username #  The name of the key in the secret that contains your database username
      dbPasswordKey: password # The name of the key in the secret that contains your database password
      dbHostKey: host # The name of the key in the secret that contains your database host
    admin:
      refreshInterval: 1h
      valueIsJson: false
      providerSecretName: adminSecretName # The name of the AWS SecretsManager/Azure KeyVault/etc. secret
      adminPasswordKey: "nexusAdminPassword" # The name of the key in the secret that contains your nexus repository admin password
    license:
      providerSecretName: nexus-repo-license.lic # The name of the AWS SecretsManager/Azure KeyVault/etc. secret that contains your Nexus Repository license
      decodingStrategy: null # Can be Base64
      refreshInterval: 1h
Secret Store CSI Driver

If you do not wish to use an external secrets operator, you can use the Secrets Store CSI driver.

  1. Follow Microsoft's documentation for creating a key vault.

  2. Select the resource group you created when setting up your AKS cluster.

  3. Select the same region as the resource group you created.

  4. Restrict it to your virtual network.

  5. Add your license file to the key vault using the Azure CLI. The command to add your license will look similar to the following:

    az keyvault secret set --name <name_for_secret> --vault-name <name_of_keyvault> --file <path_of_license_file.lic> --encoding base64

  6. Add the secrets for your database username, password, and host as well as your Nexus Repository admin password.

    1. You can do this manually in the key vault you created via the portal.

    2. Follow Microsoft's documentationfor adding secrets via the portal.

  7. If you did not enable the CSI Secrets Store driver during AKS cluster creation, you can follow Microsoft's documentation for upgrading an existing AKS cluster with Azure Key Vault provider for Secrets Store CSI Driver capability. Ensure that you provide the AKS node(s) on which Sonatype Nexus Repository will run with appropriate permissions for accessing the Key Vault as described in Microsoft's documentation.

    • The CSI Secrets Store Driver is a Kubernetes-specific plugin that will allow the Kubernetes node on which Sonatype Nexus Repository runs to retrieve the license and database user/password information from the key vault.

Option 2 - Configure License in Helm Chart

This option is only for those using the HA/Resiliency Helm Chart but not Azure Key Vault.

  1. In the values.yaml, locate the license section as shown below.

      license:
    name: nexus-repo-license.lic
    licenseSecret:
      enabled: false
      file: # Specify the license file name with --set-file license.licenseSecret.file="file_name" helm option
      fileContentsBase64: your_license_file_contents_in_base_64
      mountPath: /var/nexus-repo-license

  2. Change the license.licenseSecret.enabled to true.

  3. Do one of the following:

    1. Specify your license file in license.licenseSecret.file with --set-file license.licenseSecret.file="file_name" helm option.

    2. Put the base64 representation of your license file as the value for license.licenseSecret.fileContentsBase64.

Option 3 - Configure Licensing with License Configuration Mapping YAML

If you are not using the HA/Resiliency Helm Chart and also do not wish to use Azure Key Vault, you will need to use Kustomize when applying your YAML files and include a completed License Configuration Mapping YAML.

Step 5 - Azure Monitor (Optional, but Recommended)

Tip

If you plan to use the HA/Resiliency Helm Chart (GitHub, ArtifactHub) and Azure Monitor, you will need to enable Azure Monitor when creating your cluster; however, you do not need to do anything else in this step as the Helm chart will handle it for you.

Why Use Azure Monitor?

As a best practice, we recommend enabling Azure Monitor when creating your AKS cluster.

When running Nexus Repository on Kubernetes, it is possible for it to run on different nodes in different AZs over the course of the same day. In order to be able to access Nexus Repository's logs from nodes in all AZs, we recommend that you externalize your logs to Azure Monitor.

How to Enable Azure Monitor

Follow Microsoft's documentation to enable Azure monitor when creating your AKS cluster.

Enabling Azure Monitor when creating the AKS cluster automatically pushes logs to stdout from all containers in the Sonatype Nexus Repository pod to Azure Monitor

In addition to the main log file (i.e., nexus.log), Nexus Repository uses side car containers to log the contents of the other log files (request, audit, and task logs) to stdout so that they can be automatically pushed to Azure Monitor.

See an example of obtaining log messages from Azure Monitor

Below is an example of using a query to find out container IDs and use them to view log messages.

Open the Azure portal and navigate to the Log Analytics workspace that you specified for Azure Monitor during AKS cluster creation. Select the Logs tab and open a query editor.

Run the following query for each of the containers in the Nexus Repository pod:

ContainerInventory
| where Name contains "<container_name>" and ContainerState == "Running"
| order by TimeGenerated desc

container_name could be one of the following:

  • nxrm-app_nxrm-deployment-nexus

  • tasks-log_nxrm-deployment-nexus

  • request-log_nxrm-deployment-nexus

  • audit-log_nxrm-deployment-nexus

This will return a result such as the following:

150406514.png

You can then copy the ContainerID from this result and use it to access the logs in another query such as the following:

ContainerLog
| where TimeGenerated > ago (1h)
| where ContainerID in ('4f5165265cd79f4876b308fa95b89a0de08b93cf6abbdb47a9a2c25f7ef3736d')
| order by LogEntry desc

This would return a result such as the following with log messages shown in the LogEntry column:

150406515.png

Step 6 - Azure Files Dynamic Provisioning

Sonatype Nexus Repository requires persistent volumes to store logs and configurations to help populate support zip files. Azure Files provides a shared location for log storage. Note that the Helm chart supports both Azure Disks and Files; however, we recommend that you use Azure Files.

When you apply the StatefulSet YAML (or the values.yaml if using the HA/Resiliency Helm Chart), the volumeClaimTemplate defined therein will tell Azure to dynamically provision Azure Files shares of a specified volume size.

This step requires enabling the CSI driver for AKS, which contains built-in storage classes such as azurefile, azurefile-csi , azurefile-csi-premium, and azurefile-premium. (The provided StatefulSet YAML specifies azurefile-csi-premium, but you can adjust this as desired.)

Follow Microsoft's documentation to enable the CSI driver for AKS.

For those using the Helm chart, you will need to make updates to your values.yaml before installing the Helm chart. Details about what to update are included in the Helm chart's README file.

You can confirm that you've set up your cluster correctly by running the following command:

kubectl get storageclasses

If your cluster is set up correctly, you should get a response like the following:

NAME                    PROVISIONER            RECLAIMPOLICY    VOLUMEBINDINGMODE    ALLOWVOLUMEEXPANSION    AGE
azurefile               file.csi.azure.com     Delete           Immediate            true                    29d
azurefile-csi           file.csi.azure.com     Delete           Immediate            true                    29d
azurefile-csi-premium   file.csi.azure.com     Delete           Immediate            true                    29d
azurefile-premium       file.csi.azure.com     Delete           Immediate            true                    29d

Step 7 - Azure Blob

Azure Blob Storage provides your object (blob) storage.

Use Microsoft's documentation to set up a storage account and then begin working with blobs:

  1. Select the resource group you created when setting up your AKS cluster.

  2. Select the same region as the resource group you created.

  3. Preferably, use thepremium performanceoption; select the block blobs premium account type.

    1. While premium is preferred, standard is also supported.

  4. Select the zone-redundant storage option.

  5. Restrict it to your virtual network.

Apply YAML Files to Start Your Deployment

Tip

If you are using the HA/Resiliency Helm Chart (GitHub, ArtifactHub), skip these steps and follow the instructions on GitHub for installing the Helm Chart instead. Ensure you update your values.yaml as detailed in the Helm chart's README file.

Warning

The YAML files linked in this section are just examples and cannot be used as-is. You must fill them out with the appropriate information for your deployment to be able to use them.

Step 1 - Create a Kubernetes Namespace

All Kubernetes objects for the Sonatype Nexus Repository need to be in one namespace.

There are two ways to create a Kubernetes namespace: manually using the kubectl command-line tool or through a Namespaces YAML.

Option 1 - Manually Create Namespace

  1. Follow the Kubernetes documentation for creating a Kubernetes namespace with the kubectl command-line tool.

  2. You will use the following command to create the namespace:

kubectl create namespace <namespace>

Option 2 - Use the Namespaces YAML

  1. Fill out a Namespaces YAML with the correct information for your deployment.

  2. Apply your YAML to create a namespace.

Step 2 - Fill Out YAML Files

  1. Use the sample YAML files linked in the Sample YAML Files as a starting point.

  2. Fill out your YAML files with the appropriate information for your deployment.

Step 3 - Apply the YAML Files

You will now apply your YAML Files using one of two methods:

Option 1 - Apply YAML Files Using Kustomize

Follow the steps below to apply all of your completed YAMLs in one command:

  1. Create a Kustomization directory into which you will place your YAML files.

  2. Fill out a license configuration mapping YAML and place it in this Kustomization directory.

  3. Place all of your other completed YAML files as well as your Sonatype Nexus Repository license file in the Kustomization directory.

  4. Use the following command where <kustomization_directory> is your Kustomization directory containing all of the YAML files; this will apply all YAML files in that directory:

    kubectl apply -k <kustomization_directory>

    Your Nexus Repository license is passed into the Nexus Repository container by using Kustomize to create a configuration map that is mounted as a volume of the pod running Nexus Repository.

    See Kubernetes documentation for using Kustomize for more information.

Option 2 - Apply YAML Files Manually

Manually run each of your completed YAML files in the order below:

  1. Secrets YAML

  2. Logback Tasklogfile Override YAML

  3. Services YAML

  4. StatefulSet YAML

  5. Ingress YAML

Explanations of what each file does can be found in the Sample YAML Files reference below.

Sample YAML Files

Warning

The YAML files linked in this section are just examples and cannot be used as-is. You must fill them out with the appropriate information for your deployment to be able to use them.

* If you are using the HA/Resiliency Helm Chart (GitHub, ArtifactHub), do not use this table. Follow the instructions on GitHub for installing the Helm chart instead.

File

What it Does

Do I Need This File?*

Namespaces YAML

Creates a namespace into which to install the Kubernetes objects for Sonatype Nexus Repository

Needed only if you are using the YAML method for creating a namespace.

Secrets YAML

Creates a Kubernetes secret object backed by Azure Key Vault that contains the database credentials and license

Needed only if you are using Azure Key Vault.

Logback Tasklogfile Override YAML

Changes Nexus Repository's default behavior of sending task logs to separate files; the task files will now exist at startup rather than only as the tasks run

Yes

Services YAML

Creates a Kubernetes service through which the Nexus Repository application HTTP port (i.e., 8081) can be reached

Yes

StatefulSet YAML

Starts your Sonatype Nexus Repository pods; uses secrets for mounting volumes

Needed only if you are using Azure Key Vault.

Kustomize StatefulSet YAML

Starts your Sonatype Nexus Repository pods; uses config map for mounting volumes

Needed only if you are using Kustomize.

Ingress YAML

Exposes the Nexus Repository service externally so that you can communicate with the pods

Yes

License Configuration Mapping YAML

Creates a config map containing a base64 representation of your license file contents and associates that config map with the specified StatefulSet YAML

Needed only if you are using Kustomize.

Note

  • Always run your YAML files in the order specified above.

  • The resources created by these YAMLs are not in the default namespace.

  • The sample YAMLs are set up to disable the default blob stores and repositories on all instances.

Next Steps

You can now move on to the Post-Deployment Steps for HA.