Repository Import

Only available in Nexus Repository Pro. Interested in a free trial?  Start here.


The Repository - Import external files task will take content from a directory on the server and import it into the desired repository in Nexus Repository. This will allow you to get external content into a repository from any source, even Nexus Repository 2.

Planning and Requirements

Before using this task, you should consider a few things:

  • Version Compatibility Requirements - When importing from one Sonatype Nexus Repository 3 instance to another, both the source and target instances must be using the same version. You will not be able to import files from an older or newer Sonatype Nexus Repository 3 version. However, you can import files from the latest Sonatype Nexus Repository 2 version in order to upgrade.
  • Space Requirements - Copying all components will double the assets and space on disk; hard linking does not do this but will take additional space to store all of the metadata.
  • Location Requirements - The source directory must be writable by the user running Nexus Repository 3.  The import history is stored in the source directory to enable incremental imports.
  • Timing - This task can take some time to run, and multiple imports are not done in parallel. Each task will need to complete before the next one will run.
  • Frequency - You can configure this task to run manually or on a schedule depending on your needs.
  • Performance - We do not recommend that you import directly to object storage such as S3 as doing so can result in poor performance.
  • Deployment Policy Impacts for Hosted Repositories - If your hosted repository's deployment policy is set to Disable Redeploy, this will prevent overwriting and updating existing assets during import. See the "Hosted" sub-section of Managing Repositories and Repository Groups for more information on deployment policies.
  • Hard Link Requirements 
    • To use hard links, the target repository blob store must be on the same file system as the source.
    • Hard linking is not supported for object storage blob stores.

Limitations

  • When importing from one Sonatype Nexus Repository 3 instance to another, the source and target Sonatype Nexus Repository 3 instances must be using the same version. You will not be able to import files from an older or newer Sonatype Nexus Repository 3 version. However, you can import files from the latest Sonatype Nexus Repository 2 version in order to upgrade.
  • Import is limited to just artifacts; it does not include any server configurations.
  • Metadata and checksums are generated automatically on import and will not maintain the exported attributes.
    • As metadata and checksums are generated, the Blob created and Blob updated attributes will be set to the date and time when Sonatype Nexus Repository generated the file during import.
  • Import will not include hidden files and directories (e.g., those with names that start with a period).
  • Only one import/export task can run at a time. If you attempt to run more than one at a time, the others will be queued to run after the initial one completes.

Configuration

Create a new Repository - Import external files task and you will be presented with the configuration options listed below:

Important import-specific fields are as follows:

Target repository - The repository into which to import content. This list will only be populated with Hosted repositories of the supported Formats (see format list below).

Source directory - The directory from which to import content. 

Files in this directory should be in a directory structure that supports the desired format (see format list below).

Enable Hard Links - For some formats (see Hard Link Support in table below), enabling this option will create hard links for the blobs in the source directory to be used in the target directory. A physical copy of the blobs will not be created in the target directory. This can save time if the source directory will continue to be accessible by the current instance. Ensure you have read the hard link requirements under Planning and Requirements.

Running the Task

Only one import/export task can run at a time. If you attempt to run more than one at a time, the others will be queued to run after the initial one completes.

When the task runs, it generates a .nexus/nxrm-export-import-task/ directory inside of the source directory. This .nexus/nxrm-export-import-task/ directory contains metadata that Nexus Repository uses for future imports so that it will not import content multiple times if the task is run more than once. This will allow you to keep placing content into this directory while only importing newer content if you run the task again.

As Nexus Repository processes the import, it will log every single file it imports to the task log; a new task log is created each time a task runs. If an error occurs with any particular file, Nexus Repository will simply log that fact, then continue importing the rest of the content.

Resetting Data (.dat) Files

If you need to reset the import data files, run the following command:

rm .nexus/nxrm-import-export-task

Deleting Import Task Source Cache Files Inside Nexus Repository 2 Repository Storage Directory

Should you need to reset an import, you will need to delete the cache.

The following steps provide a workaround for deleting .dat files inside a Nexus Repository 2 storage directory where the .dat files are stored in a .nexus directory instead of .nexus/nxrm-import-export-task/:

  1. Locate the source .nexus directory of the import
    1. In Nexus Repository 2, the default location would be sonatype-work/nexus/storage/<repoid>/.nexus
  2. Use a command like the following to find and delete all the import cache files:
find sonatype-work/nexus/storage/releases/.nexus -type f -name 'imported-files*.dat' -delete


Supported Formats

You can import content into repositories of any format listed below. Some formats require that content be structured in a certain manner:

FormatSinceDirectory LayoutHard Link Support Since
Raw3.23.0No specific layout required, content will simply be imported exactly as laid out in the source directory3.31.0
Maven3.23.0

Files need to be laid out in the usual maven GAV structure, /(groupId)/(artifactId)/(version)/(artifactId-version.extension)

For example:

{rootdir}/com/mycompany/awesome-artifact/1.0/awesome-artifact-1.0.jar

maven-metadata files at each level (G, A or V) will also be imported.

3.31.0
npm3.25.0No specific layout required, will simply import all tgz packages contained in the source directory3.34.0
NuGet3.25.0No specific layout required, will simply import all nuget packages contained in the source directory3.35.0
Docker3.26.0

V1

  • Image metadata must be stored in the file v1/images/{layerId}/json
  • Image content must be stored in the file v1/images/{layerId}/layer
  • Tags must be stored in the file v1/repositories/{name}/tags/{tag}
  • Image index must be stored in the file v1/repositories/{name}/images/{image}

V2

  • Blobs must be stored in path /v2/-/blobs/{digest}
  • Manifests must be stored in file /v2/{name}/manifests/{tag}

Examples:

(Note the &#58 in the following path, this is simply an ascii encoded colon ':' as windows file systems do not support this character in the filename)

{rootdir}/v2/-/blobs/sha256&#5803f4658f8b782e12230c1783426bd3bacce651ce582a4ffb6fbbfa2079428ecb

{rootdir}/v2/helloworld/manifests/latest

{rootdir}/v2/helloworld/manifests/1.0

3.35.0
PyPI3.26.0No specific layout required, will simply import all archives contained in the source directory (along with any .asc signature files alongside them)3.35.0
RubyGems3.26.0No specific layout required, will simply import all gem files contained in the source directory3.35.0
Yum3.26.0

No specific layout required, will simply import all of the rpm (or drpm) files, as well as any comps.xml files in the source directory


3.35.0