Skip to main content

Repository Replication (Legacy)

Note

Only available in Sonatype Nexus Repository Pro. Interested in a free trial? Start here.

Warning

In release 3.48.0, we introduced Content Replication, which removes Replicator and makes replication easier to set up and deploy. This documentation remains available for current implementations only. We do not recommend implementing this version of replication as we will be completely removing it once the new replication design is available.

Note

Replication is not appropriate for disaster recovery. If you need a disaster recovery solution, please see our resilient deployment options and backup and restore procedures.

What is Repository Replication?

Repository replication allows you to publish artifacts to one Nexus Repository Pro instance and make them available on other instances to provide faster artifact availability across distributed teams.

With repository replication, you can manage what binaries can be replicated between two or more instances. The following diagram illustrates the replication process.

83495687.png

Replication Use Cases

Primary Use Case

Repository replication is primarily used to make content in one repository available to another repository as soon as it is published so that users on the second instance do not have to wait for Nexus Repository to proxy content from another instance.With large files (e.g., Docker images), waiting for Nexus Repository to proxy content from another instance can induce timeouts. So, rather than using a proxy repository and only replicating content on demand, we now replicate the content on creation. When a person or CI tool on the target end requests content, their local Nexus Repository instance will already have that content, so the request won't induce any timeouts to the format-specific client as Nexus Repository can immediately start serving the new content. This use case is illustrated above.

Replicate One Source to Multiple Targets

You can also use repository replication to replicate content from one source repository to multiple target repositories. You may use this setup if you have a central Nexus Repository server with multiple sites in other parts of the world and the need to push all content from the central server to those other sites for speedier local access for developers. This setup is illustrated below with setup detailed in Replication Scenarios.

91947717.png

Bi-Directional Replication

You could also set up a bi-directional replication scenario where content is replicated between two geographically separate locations. You would use this setup if you had two locations with separate Nexus Repository instances (Team A using Instance A and Team B using Instance B). If both teams consume content that the other team creates, you would need to replicate content in both directions so that Team A has quick access to the content Team B publishes and vice versa. This setup is illustrated below with setup detailed in Replication Scenarios.

116230948.jpeg

Prerequisites for Using Replication

In order to use repository replication, you must meet the following prerequisites:

  • Repository replication is only available for Nexus Repository Pro customers; you must have a valid Nexus Repository Pro license to proceed.

  • Source and target repositories must be either Raw, Docker, npm, or Maven hosted repositories.

  • Source and target repository blob stores cannot be part of a blob store group.

  • Repository replication is not compatible with the legacy High Availability Clustering (HA-C) feature.

  • For file-based blob stores, the source and target repository blob stores must be readable/writable on the machine where the Replicator tool (the replication CLI jar) runs

  • Ensure the target repository is a newly created hosted repository in one of the supported formats. As a best practice, do not use a target repository that has previously uploaded binaries. This ensures you can maximize the manageability of the storage requirement of replicated content. Note also that the target repository will be read-only to all other requests outside of replication as long as it is used as a target repository.

Prerequisite Steps for Nexus Repository Instances

  1. Download and install Nexus Repository Pro version 3.34.0 or later in two separate environments following regular installation instructions.

    1. Both Nexus Repository instances must have a hosted repository configured: one will be your source repository, and the other will be your target repository.

    2. The target repository should not be one to which you are deploying components. SeeBi-Directional Replication for details.

      Note

      Always use repository replication with two or more instances.

  2. Ensure that a valid Nexus Repository Pro license file is installed in these instances.

  3. Give a user the new Replication Administrator role in each of the target instances.

  4. If you are running the new Replicator on these same instances, install rsync (for file system blob store replication) or AWS CLI (for S3 blob store replication) in your environments. For instructions on installing AWS CLI, see theAWS documentation.

In order to run the Replicator, you will also need appropriate permissions to run the Replicator app, rsync, and/or AWS CLI as well as permission to read/write to/from the source and target blob stores.

Additional Prerequisites for Separate Replicator Instances (Optional)

  1. Ensure you are using a Unix, Linux, or Unix/Linux variant operating system; alternatively, you may use Windows 10 with a Linux subsystem. This is required for access to rsync.

  2. Ensure you have Java 8 installed.

Nexus Repository Configuration

Follow these steps to enable repository replication in each instance:

Enable the Capability

  1. Sign in to Nexus Repository as an administrator.

  2. Navigate toAdministration>System>Capabilities.

  3. Select Create capability.

  4. SelectReplicationfrom the list.

  5. Select theCreate capabilitybutton to enable it.

  6. Verify that you now have access to a new Replication page under Administration > Repository > Replication.

  7. Create and run the new Replication - Backfill blob store attributes with component metadata task

Note

Running this task initiates a required process that adds necessary metadata to any existing blobs in your Nexus Repository instance. This task may take some time to complete, and you should not attempt to create replication connections until this has completed. You can confirm completion by looking at the task status.

Replication Administrator Role

The pre-configured Replication Administrator role has all six privileges that relate to replication:

Four privileges control access to replication endpoints:

  • nx-replication-readgives the ability to view existing replication endpoints.

  • nx-replication-create gives the ability to view existing replication endpoints and to create new ones.

  • nx-replication-updategives the ability to view existing replication endpoints and to update existing ones.

  • nx-replication-deletegives the ability to view existing replication endpoints and to delete existing ones.

In addition, the following existing privileges are also required to list the available repositories:

  • nx-repository-admin-*-*-edit

  • nx-repository-view-*-*-*

On the source repository, any user with nx-replication-read, nx-replication-create, nx-replication-update, and nx-repository-view-*-*-* can configure a replication from a source repository; however, they must use a login for a user on the target repository that has the nx-repository-view-*-*-*, nx-repository-admin-*-*-edit, and nx-replication-update permissions.

  • In the target instance, assign the Replication Administrator role to a local user that can be used to authenticate in the Target Information configuration of a replication connection.

Configure a Replication Connection

At this point, you can continue to configuring a replication connection.

Troubleshooting

Replication loggers provide additional log statements that can be useful for debugging. Complete these steps on both the source and target Nexus Repository instances to see additional replication logs.

  1. Navigate toAdministration>Support>Logging.

  2. Select theCreate Loggerbutton.

  3. Enter "com.sonatype.nexus.replication" under Logger Name.

  4. Select the DEBUG Logger Level.

  5. Select Save.

You can use the log viewer to observe any debug information logged by the com.sonatype.nexus.replication logging object.

Files to investigate for troubleshooting include the following:

  • The nexus.log files on both the source and target Nexus Repository instances

    • Located in $data-dir/log/nexus.log

  • The replicator.log file from the CLI app

    • Located in a logs subdirectory next to where you are running the Replicator CLI

    • Retains up to 30 days worth of logs

    • Contains all rsync and AWS CLI logging