Option 2 - On-Premises High Availability Deployment Using Kubernetes
Only available in Sonatype Nexus Repository Pro. Interested in a free trial? Start here.
Warning
Sonatype Nexus Repository High Availability deployments should be fully deployed and tested in a development environment before attempting to deploy in production. Improper deployment in a production environment can result in critical data loss.
This example architecture illustrates how to use a Kubernetes cluster and PostgreSQL database to create a highly available Nexus Repository deployment.
Use Cases
This reference architecture is designed to protect against the following scenarios:
A node/server failure within a data center
A Sonatype Nexus Repository service failure
You would use this architecture if you fit the following profiles:
You are a Sonatype Nexus Repository Pro user looking for a resilient Nexus Repository deployment option on-premises in order to reduce downtime
You would like to achieve automatic failover and fault tolerance as part of your deployment goals
You already have a Kubernetes cluster set up as part of your deployment pipeline for your other in-house applications and would like to leverage the same for your Nexus Repository deployment
You have migrated or set up Nexus Repository with an external PostgreSQL database and want to fully reap the benefits of an externalized database setup
Requirements
Each Sonatype Nexus Repository instance must meet our Sonatype Nexus Repository System Requirements.
You must also meet all System Requirements for High Availability Deployments, including using shared blob storage (see Migrating to Shared Blob Storage if necessary).
A tool such as Kops for setting up Kubernetes clusters
kubectl command-line tool
Kustomize to customize Kubernetes objects
If you require your clients to access more than one Docker Repository, you must use an external load balancer (e.g., NGINX) as a reverse proxy instead of the provided ingress for Docker YAML.
Before proceeding, you must have adjusted the max_connections.
Note
We highly recommend that on-premises containerized deployments have only one Sonatype Nexus Repository instance per machine (e.g., if you are running a cluster of three Sonatype Nexus Repository instances, you should have three virtual/bare metal machines)
Limitations
The ability to upgrade Nexus Repository with zero downtime (rolling upgrade) is only available from release 3.71.0 forward; 3.72.0 is the first release to which you can upgrade with zero downtime. See our HA upgrade documentation for details.
We do not recommend deploying HA clusters across regions due to the large number of database interactions is involved in HA.
Setting Up the Architecture
Note
Unless otherwise specified, all steps detailed below are still required if you are planning to use the HA/Resiliency Helm Chart (GitHub, ArtifactHub).
Step 1 - Kubernetes Cluster
Kubernetes works by placing containers into pods to run on nodes. You must set up a Kubernetes cluster comprising one master node and one worker node per Sonatype Nexus Repository instance that you wish to run.
See Kops documentation for an example of how to set up a Kubernetes cluster with one master and two worker nodes using Kops..
Attach separate disks (i.e., separate from the root disk) to your worker nodes.
Install the Local Persistence Volume Static Provisioner. Please refer to Local Persistence Volume Static Provisioner documentation.
Use the Local Persistence Volume Static Provisioner to automatically create persistent volumes for your chosen storage class name as documented.
Step 2 - PostgreSQL Database
Set up a PostgreSQL database and ensure the worker nodes can communicate with this database. See Installing Nexus Repository Pro with a PostgreSQL Database for more information.
To avoid single points of failure, we recommend you set up a highly available PostgreSQL cluster.
Step 3 - Licensing
There are two options for managing licensing:
Configure License in Helm Chart - This option is for those using the HA/Resiliency Helm chart
License Configuration Mapping YAML - This option is for those not using the HA/Resiliency Helm chart
Option 1 - Configure License in Helm Chart
In the values.yaml within the HA/Resiliency Helm Chart, locate the license section as shown below.
license: name: nexus-repo-license.lic licenseSecret: enabled: false file: # Specify the license file name with --set-file license.licenseSecret.file="file_name" helm option fileContentsBase64: your_license_file_contents_in_base_64 mountPath: /var/nexus-repo-license
Change the
license.licenseSecret.enabled
totrue
.Do one of the following:
Specify your license file in
license.licenseSecret.file
with--set-file license.licenseSecret.file="file_name" helm option
Put the base64 representation of your license file as the value for
license.licenseSecret.fileContentsBase64.
Option 2 - License Configuration Mapping YAML
Tip
If you plan to use the HA/Resiliency Helm Chart (GitHub, ArtifactHub),use option 1 instead.
If you are not using the HA/Resiliency Helm Chart, you will need to use Kustomize when applying your YAML files and include a completed License Configuration Mapping YAML.
Step 4 - NFS Dynamic Subdir Provisioning
NFS provides a shared location for storing logs. The NFS subdir external provisioner uses your existing NFS server to support dynamic provisioning of Kubernetes Persistent Volumes via Persistent Volume Claims.
Install the Kubernetes NFS subdir external provisioner following the instructions provided in the Kubernetes NFS subdir external provisioner GitHub.
If you are using the Helm chart, you will also need to make updates to your values.yaml before installing the Helm chart. Details about what to update are included in the Helm chart's README file.
Starting Your Deployment
Tip
If you are using the HA/Resiliency Helm Chart (GitHub, ArtifactHub), skip these steps and follow the instructions on GitHub for installing the Helm chart instead. Ensure you update your values.yaml as detailed in the Helm chart's README file.
Step 1 - Create a Kubernetes Namespace
All Kubernetes objects for Sonatype Nexus Repository need to be in one namespace.
There are two ways to create a Kubernetes namespace: manually using the kubectl command-line tool or through a Namespaces YAML.
Option 1 - Manually Create Namespace
Follow the Kubernetes documentation for creating a Kubernetes namespace with the kubectl command-line tool.
You will use the following command to create the namespace:
kubectl create namespace <namespace>
Option 2 - Use the Namespaces YAML
Fill out a Namespaces YAML with the correct information for your deployment.
Apply your YAML to create a namespace.
Step 2 - Fill Out YAML Files
Use the sample yaml files linked in the Sample YAML Files as a starting point.
Fill out your YAML files with the appropriate information for your deployment.
Step 3 - Apply the YAML Files
You will now apply your YAML Files using one of two methods:
Option 1 - Apply YAML Files Using Kustomize
Follow the steps below to apply all of your completed YAMLs in one command:
Create a Kustomization directory into which you will place your YAML files.
Fill out a License Configuration Mapping YAML and place it in this Kustomization directory.
Place all of your other completed YAML files as well as your Sonatype Nexus Repository license file in the Kustomization directory.
Use the following command where
<kustomization_directory>
is your Kustomization directory containing all of the YAML files; this will apply all YAML files in that directory:kubectl apply -k <kustomization_directory>
Your Nexus Repository license is passed into the Nexus Repository container by using Kustomize to create a configuration map that is mounted as a volume of the pod running Nexus Repository.
See Kubernetes documentation for using Kustomize for more information.
Option 2 - Apply YAML Files Manually
Manually run each of your completed YAML files in the order below:
License Configuration Mapping
StatefulSet YAML
Services YAML
Ingress YAML
Explanations of what each file does can be found in the Sample YAML Files reference below.
Sample Kubernetes YAML Files
Warning
The YAML files linked in this section are just examples and cannot be used as-is. You must fill them out with the appropriate information for your deployment to be able to use them.
* If you are using the HA/Resiliency Helm Chart (GitHub, ArtifactHub), do not use this table. Follow the instructions on GitHub for installing the Helm chart instead.
You can use the sample on-premises YAML files from our sample files GitHub repository to help set up the YAMLs you will need for your deployment. Ensure you have filled out the YAML files with appropriate information for your deployment.
File | What it Does | Do I Need This File?* |
---|---|---|
Creates a namespace into which to install the Kubernetes objects for Sonatype Nexus Repository | Needed only if you are using the YAML method for creating a namespace. | |
Creates a config map containing a base64 representation of your license file contents and associates that config map with the specified StatefulSet YAML | Needed only if you are using Kustomize. | |
Starts your Sonatype Nexus Repository pods; uses secrets for mounting volumes | Yes | |
Creates a Kubernetes service through which the Nexus Repository application HTTP port (i.e., 8081) can be reached | Yes | |
Exposes the Nexus Repository service externally so that you can communicate with the pods | Yes |
Note
Always run your YAML files in the order specified above.
The resources created by these YAMLs are not in the default namespace.
The sample YAML files are set up to disable the
default
blob stores and repositories on all instances.
Next Steps
You can now move on to the Post-Deployment Steps for HA.