Skip to main content

Content Replication

Only available in Sonatype Nexus Repository Pro. Interested in a free trial? Start here.

Publish artifacts to one Nexus Repository instance and have other instances pre-fetch them through a standard proxy repository. This provides faster artifact availability across distributed teams.

This design lets you move large binaries to remote distribution instances before users request them. This reduces latency by eliminating the time your builds need to wait for the latest binaries to be available across your organization.

Use content replication to:

  • Update multiple distribution services all proxying the primary development source

  • Set up bi-directional replication of content between two development teams

  • Replicate a subset of content for a particular audience

How it Works

Manage the binaries replicated between two or more instances with content replication. The following diagram illustrates the process.

  1. New assets are published to the Nexus Repository.

  2. The replication task runs every minute to identify all new assets.

  3. The task sends a pre-fetch request for each asset to the remote instance to proxy them.

  4. Users on the target instance have access to the new artifacts as soon as the download is complete.

150406295.png

When to use Content Replication

Content replication fetches content before being requested to reduce latency. For distributions that are not sensitive to latency, the standard proxying feature may be sufficient.

Content Replication Considerations

  • Not appropriate for disaster recovery

    Content replication only copies components to the remote proxy repository. Content replication does not copy configuration or internal component metadata therefore it is not an appropriate disaster recovery solution.

    See resilient deployments for disaster recovery solutions.

  • Impact on cleanup policies

    Once a component has been replicated to a remote instance, any component downloads from the target instance do not update the last downloaded date on the source instance. This could result in a component being removed from the source but not the target.

  • Impact on remote storage

    The content replication task fetches all new content, which may increase your storage requirements on remote servers. Pre-existing assets are not automatically replicated unless requested through the proxy. Clean-up of artifacts on remote servers needs to be managed independently of the source. With a typical proxy repository, deleting the component from the source is not sent to the remote server Ensure that remotes include adequate storage and enable cleanup policies on the remote proxy.

  • Do not use with Repository Firewall

    We do not recommend using Repository Firewall on proxies configured with content replication. These proxies only contain components from your internally hosted repositories.

Prerequisites

To use content replication, you must meet the following prerequisites:

  • Available for Nexus Repository Pro customers.

  • Both the source and target instances are on the same version of Nexus Repository using a PostgreSQL database.

  • The source repository is a hosted repository while the target repository is a proxy repository targeting the host repository.

  • The source repository should not be the writable member of any group repository.

Supported Formats

Content replication requires that the source repository be a hosted repository and the target be a proxy repository. The following formats supporting both hosted and proxy repositories and are suitable for content replication:

  • APT, Docker, Conan (NEW IN 3.77.0), Helm, Maven, npm, NuGetV3, R, Raw, RubyGems, Rust Cargo (NEW IN 3.73.0)

Enable Content Replication

  1. On the target instance, configure a proxy repository pointing to a host repository on the primary server.

    For Docker repositories, use the repository URL, not the subdomain or port.

  2. Enable the Pre-emptive Pull option under the Remote Storage property.

    150406422.png

  3. (Optional) use the Asset Name Matcher to scope the content to replicate.

    See Asset Name Matcher for examples.

  4. Provide a service account to authenticate with the primary server in the authentication section.

  5. A task with the name "Replicate content: <target-repository-name>" is automatically created and configured to check for new components every minute.

    150406423.png

Content Replication Task

Content is replicated when the content replication task runs on the target instance.

When the task runs, the target instance connects to the source instance, requests a list of new content since the last successful replication, and then issues proxy requests for those components to replicate them to the target.

A list of the replicated components is stored in a database table with the following name format:

<format>_content_repository
  • Pausing Content Replication

    Disabling or removing the schedule from the task will pause content replication. When re-enabled, content replication uses the list to replicate content from the last successful timestamp.

  • Disabling Content Replication

    Remove the check from the Pre-emptive Pull configuration on the proxy repository to disable content replication. On saved, the content replication task is removed.

    Disabling replication does not remove the stored timestamp for the last successful replication. When re-enabling replication the task begins from the last stored timestamp.

  • Deleting the Content Replication task

    Nexus Repository does not automatically re-create the task. Toggling the configuration on the proxy recreates the task.

  • Upgrading Nexus Repository

    We recommend temporarily pausing content replication when upgrading Nexus Repository. Re-start replication once you've upgraded both instances to the same version.

    Re-start replication once you've upgraded both instances to the same version. Replication fails when both instances are not on the same version.

Troubleshooting

  • Logs

    Logs are available under Settings → Support → Logs. They are stored on disk in the following location:

    $data-dir/log/replication/
  • batchSize Property

    The number of replicated artifacts is controlled by the batch size property in the nexus.properties file. Adjust this number when content replication consistently takes longer than the frequency of the running task.

    $data-dir/etc/nexus.properties

nexus.replication.pull.batchSize=512