Cleanup Policies

Cleanup policies help you automatically manage repository storage by removing old, unused, or outdated components. By defining rules for what to delete and when, you can maintain repository hygiene, reduce storage costs, and ensure compliance with your organization's retention requirements.

Note that cleanup policies and the associated tasks are not configured by default. Define the policies that best suit your requirement.

Cleanup Policies Workflow

Administrators create cleanup policies. The following sequence is a high level description of using cleanup policies:

Administrators create Cleanup Policies depending on their requirements.
Administrators assign one or more Cleanup Policies to the repository configuration.
See Repository Management
Administrators schedule Cleanup Tasks to run regularly. (e.g., once a night)
```
Admin - Cleanup repositories using their associated policies
Admin - Cleanup unused asset blobs
```
Cleanup tasks "soft delete" components by flagging them for removal. Components still consume space but may be recovered when needed.
See Tasks
Run the compact blob store task on off-peak hours for each blob store to reclaim disk space. These tasks completely removed the components from the storage disk, freeing up the available space.
```
Admin - Compact blob store
```

Security Requirements

Only users with admin privileges may create cleanup policies. (i.e., nexus:*). Users with edit repository privileges may add a clean policy to the repository.

nexus:repository-admin:{format}:{repository}:edit

Users with either of the following privileges may modify cleanup tasks:

nx-tasks-update, nx-tasks-all

See Repository Admin

Creating Cleanup Policies

Cleanup Policies are located in the repository section of the settings menu and require admin privileges to create or modify. They are intended for use in one more many repositories typically associated with a specific repository format type.

Navigate to Settings → Repository → Cleanup Policies.
Select Create Cleanup Policy.
Provide a unique name for the policy
Only letters, digits, underscores(_), hyphens(-), and dots(.) are allowed and may not start with underscore or dot.
Under Format field, select a target format for the policy
The 'All formats' option may be selected for any repository type. However most formats have specific cleanup criterias. See Cleanup Criteria by Format.
Enter a brief description about the policy in the Description field (limit of 400 characters).
New in 3.95 Under Repositories field, add the Available Repositories to Applied Repositories.
Under Cleanup Criteria, select at least one criterion to remove the components. For more details on the available criterias, see Cleanup Criteria.
If you wish to retain the components by the number of versions, select the Number of Versions checkbox and enter the component age. For more details, see Retain Select Versions.
Select Save.

Strategies for Creating a Cleanup Policy

Cleanup Policies are intended to target components to be removed from either hosted or proxy repositories.
The criteria of the policy are combined to remove only the components that meet every condition specified.
Multiple cleanup policies may be applied to the same repository.
Policies can have overlapping criteria targeting the same components.

Adding A Cleanup Policy to a Repository

Cleanup policies may be assigned to both proxy and hosted repositories. Users require repository admin edit privileges to edit or add repositories to cleanup policies.

Select the repository from Repositories view in the Settings menu or create a new one
Navigate to the Cleanup section
Optionally use the search filter to limit the available cleanup policies from the list
Select the required cleanup policies from the available section
Use the right-facing arrow to move the cleanup policy to the applied section
Repeat for as many policies that are required
Select the Save button

Previewing Cleanup Policy Results

Before running a cleanup policy on a production repository, preview the results to see which components would be deleted. Previewing is not an audit as the results may be differ from the actual run. Take the following steps to preview the results:

Navigate to Settings → Repository → Cleanup Policies.
Select an existing cleanup policy or create a new policy with your desired criteria.
Under Preview Cleanup Policy Results, select the repository to preview.

Note that there is a 1-minute timeout on the preview to reduce the impact to performance.

PostgreSQL Cleanup Preview

Generate the list of components that the policy would remove as configured. This is a point-in-time audit as results may differ when testing the policy at a different time.

Navigate to Settings → Repository → Cleanup Policies.
Select an existing cleanup policy or create a new policy with your desired criteria.
Under Preview Cleanup Policy Results, select the repository to preview.

Select Generate CSV Report. The CSV file will be downloaded once the query is complete.

Filename: <cleanup policy name>-<repository name>-<timestamp>.csv
Fields:   namespace, name, version, path

Generating the cleanup preview CSV takes time depending on deployment size and configuration. The table below provides the approximate estimated generation time:

Deployed on an AWS ECS c6i.4xlarge instance with Aurora PostgreSQL db.r6g.large database
----------------------------------------------------------------------------
Components in Repository (millions) || Time to Generate CSV Report (minutes)
 1M  =   ~1min
 5M  =   ~2min
20M  =   ~8min
25M  =  ~15min
27M  =  ~25min
30M  =  ~40min

Under Nexus One UI, you can immediately see the list of components marked for cleanup.

Preview Limitations

The preview is a snapshot in time. Components matching the criteria may change if new components are published or existing components are downloaded before the cleanup runs. Preview counts are approximate. The actual number of deleted components may differ slightly due to concurrent changes.

Cleanup Tasks

Nexus Repository automatically creates a few system tasks to soft delete components identified for cleanup by the cleanup policies. These tasks are not manually created and are re-added on a restart of the service when deleted.

Cleanup service: Admin - Cleanup repositories using their associated policies
This task soft deletes components based on the repository's configured cleanup policies. These tasks may be rescheduled or manually executed. By default, this task is set to run once an hour.
Cleanup unused {format} blobs from nexus: Admin - Cleanup unused asset blobs
These tasks soft delete orphaned assets that are no longer needed after a component is removed. They are added when a new format has been added as a repository. By default, these tasks run every 30 minutes.

Hard Deleting Components with the Compact Blob Store Task

Cleanup Policies soft delete components by updating their metadata with a marker for deletion. This optimizes the cleanup tasks and allows administrators to recover components when they are accidently deleted. While Nexus Repository no longer displays soft deleted components in the user interface, they are not immediately deleted from storage and still use storage space.

Use the Admin - Compact blob store task to reclaim disk space and permanently delete the components from storage.

Sonatype Nexus Repository Cloud

The compact blob store task is managed and scheduled automatically. It runs daily and deletes blobs that have been soft-deleted for at least 3 days. After blobs are deleted, they may be retained in underlying storage for up to 45 days before permanent removal.

Create and schedule the Admin - Compact blob store task to reclaim disk space. Add this task for every blob store that requires cleanup.

Retain Soft-Deleted Blobs for a Period of Time Before Hard Deleting

When configuring the Admin - Compact blob store task you may set the Blobs Older Than property to temporarily retain deleted artifacts for a designated number of days before deleting them from storage. The default value is 0 days.

Interaction with Azure Blob Store Cleanup

The compact blob store task requests the Azure blob store to mark blobs for deletion. These are later hard-deleted during garbage collection on the Azure side. This varys on whether the soft delete feature is enabled as described in Azure's documentation.

Interaction with AWS S3

Before release 3.80.0, the Admin - Compact blob store task did not delete components from AWS S3 blob stores. As of release 3.80.0, AWS S3 blob stores uses the "Blobs Older Than" property for files in the S3 bucket. The S3 blob store configuration with the Expiration Days fields is no longer used for the S3 lifecycle policy.

Docker Cleanup Strategies

Docker's tagging, manifests, and layers are unique ways of managing components and assets that require additional configuration when designing a cleanup strategy.

See Components and Assets in Docker to learn more about the docker format.

Docker - Delete incomplete uploads
Soft-delete uploads to the temporary storage that are not complete
Admin - Cleanup repositories using their associated policies
Soft-delete old published or downloaded docker components i.e. tags, not layers or manifests
Docker - Delete unused manifests and images
Soft-delete orphaned layers and manifests no longer referenced by tags, possibly orphaned by cleanup policies
Warning
Docker manifest requests by digest do not include a reference to a tag, therefore they are cleaned up with this task.
```
docker pull image_name@DIGEST
```
Use this task with caution when cleaning up images that are not tagged.

Once the above tasks have run, the following tasks are needed to hard delete the components and reclaim space depending on your deployment.

Run the Admin - Compact blob store task for file-based blob stores

Additional Information

Clean Up Components That Have Never Been Downloaded
Use the Component Usage (Days) criterion to clean up components that have never been downloaded. This criterion removes components that haven't been downloaded in a specified number of days. The date the component was published is used when the component has never been downloaded.
Cleanup Policies Does Not Remove Components from Replicated Repositories
Content Replication does not replicate the deletion of components on remote repositories. Cleanup policies only remove components from the specific instance on which it is run. The remote repository requires its own setup of cleanup policies.
SQL-Based Cleanup Performance
Nexus Repository instances using PostgreSQL databases use SQL-based cleanup by default.
See the metrics at Cleanup Performance Data.
Determining the Space Your Repositories Are Using
Use the following Support article to determine the space your repositories are using.
Investigating Blob Store and Repository Size and Space Usage

Replace the following tasks with Cleanup Policies

Maven - Delete unused SNAPSHOT
Repository - Delete unused components

Cleanup Criteria

Cleanup Criteria by Format Table

The table below lists the available cleanup criteria and the formats to which they apply:

Format	Component Age	Component Usage	Release Type	Retain Select Versions³	Asset Name Matcher
All Formats
APT
Bower²
CocoaPods
Conan
Conda
Docker¹
GitLFS
Go
Helm
Maven
npm
NuGet
p2
Pub
PyPI
R
Raw
RubyGems
Swift
Terraform
Yum

¹ - Cleanup only evaluates tagged manifests for Docker.

² - Bower functionality is for proxy repositories only.

³ - Retain Select Versions for Docker and Maven are available on classic UI. For other formats, switch to Nexus One UI.

Component Age (Days)

This criteria sets how long to keep content based on component age.

Proxy repositories: based on when the component was first downloaded
Hosted repositories: based on when the component was uploaded or updated

Component Usage (Days)

This criteria sets how long to keep content based on when a component was last downloaded. The published or updated dates are used when the component has never been downloaded.

Release Type

Use to set the cleanup policy to either PRELEASES or RELEASES. Prereleases are different by format:

Maven
Versions contain the -SNAPSHOT phrase
npm
Uses semantic versioning where a version is a prerelease when it contains the dash "-" character
Yum
The non-case-sensitive "release" property in the RPM header contains one of the following:
```
alpha, beta, rc, pre, prerelease, snapshot
```

Retain Select Versions

Use the Number of Versions criteria to keep only the most recent versions of each component in a repository. This is useful for repositories where components are published frequently and older versions consume unnecessary storage. This feature requires Nexus Repository Pro and a PostgreSQL database.

The following list describes how Nexus Repository normalizes and sorts component versions for each supported format:

NEW IN 3.70

Maven
The version number is used, available for the release type Releases.
Docker
The age of the manifest is used.

NEW IN 3.95

Note

The Retain Select Versions field is only visible in Nexus One UI for the below formats.

npm
npm versions follow semantic versioning. Pre-release versions have lower precedence than release versions. Retaining select versions is available for both Releases and Pre-Releases/Sanpshots.
APT
APT (Debian) versions follow Debian version comparison rules. Retaining select versions is available for both Releases and Pre-Releases/Sanpshots.
PyPI
Python package versions follow PEP 440 versioning. Retaining select versions is available for both Releases and Pre-Releases/Sanpshots.
Go
Go module versions follow semantic versioning. Pseudo-versions are sorted by timestamp. Retaining select versions is available for both Releases and Pre-Releases/Sanpshots.
Helm
Helm chart versions follow semantic versioning. Retaining select versions is available for both Releases and Pre-Releases/Sanpshots.
NuGet
NuGet package versions follow NuGet versioning rules. Versions are sorted numerically for major, minor, and patch segments. Pre-release versions have lower precedence than release versions. Retaining select versions is available for both Releases and Pre-Releases/Sanpshots.
Pub
Dart pub package versions follow semantic versioning. Retaining select versions is available for both Releases and Pre-Releases/Sanpshots.
RubyGems
RubyGems versions follow RubyGems versioning rules. Retaining select versions is available for both Releases and Pre-Releases/Sanpshots.
Swift
Swift package versions follow semantic versioning. Retaining select versions is available for both Releases and Pre-Releases/Sanpshots.
Terraform
Terraform provider and module versions follow semantic versioning. Retaining select versions is available for both Releases and Pre-Releases/Sanpshots.
YUM
YUM (RPM) versions follow RPM version comparison rules. Retaining select versions is available for both Releases and Pre-Releases/Sanpshots.

Asset Name Matcher

Rules are based on the component name, namespace, or path in the repository. Supported regular expression patterns differ between the legacy OrientDB and the newer PostgreSQL and H2 environments.

When migrating to PostgreSQL or H2, legacy cleanup policies may result in more assets being removed than expected.

PostgreSQL and H2 Expressions

In H2 or PostgreSQL environments, the Asset Name Matcher uses Java regular expressions.

Not compatible with OrientDB Lucene regular expressions
Java regular expressions may match any part of the component path

When migrating to PostgreSQL, revise cleanup policies to include the leading slash in asset matcher names. Failure to do so may result in assets not being matched and cleaned up as expected.

OrientDB Expressions

Expressions in OrientDB uses the Elastic Search regular expression query syntax, from Apache Lucene.

Not compatible with Perl (PCRE) or Java util.regex.Pattern regular expressions
Expressions must match the entire name when wildcards are not used.
Asset names do not require a leading slash and use a limited set of operators
Asset matchers in OrientDB are different than the asset request path value used when evaluating content selector or routing rule expressions

Comparison between OrientDB and PostgreSQL

This example contains the following assets for consideration:

Pattern
 antlr.*

Repository
 /antlr/antlr/2.7.2/antlr-2.7.2.jar
 /org/antlr/antlr-master/3.1.3/antlr-master-3.1.3.pom

OrientDB - the first component is matched while the second is not
H2 or PostgreSQL - both components are matched

Expression Examples

The following examples demonstrate a specific asset name matcher against a repository and the remaining components after using the matcher in a cleanup policy.

Components in a version range

/hello/-/hello-0.0.[1-2].tgz

Repository
 /hello/-/hello-0.0.1.tgz
 /hello/-/hello-0.0.2.tgz
 /hello/-/hello-0.0.3.tgz

Remaining
 /hello/-/hello-0.0.3.tgz

Components with a specific path

/(org|com)/.*

Repository
 /org/example/test.jar
 /com/example/test.jar
 /test/example/test.jar

Remaining
 /test/example/test.jar

Cleanup components matching a specific team

/org/sonatype/^(team2)/.*

Repository
 /org/sonatype/team1/ui/5.0/ui-5.0.jar
 /org/sonatype/team2/format/1.0/format-1.0.jar
 /org/sonatype/team3/database/10.0/database-10.0.jar

Remaining
 /org/sonatype/team2/format/1.0/format-1.0.jar

A specific component

/pool/main/z/zsh/zsh-common_5.4.2-3ubuntu3_all.deb

Repository
 /pool/main/libc/libcap2/libcap2_2.25-1.2_amd64.deb
 /pool/main/z/zsh/zsh_5.4.2-3ubuntu3_amd64.deb
 /pool/main/z/zsh/zsh-common_5.4.2-3ubuntu3_all.deb

Remaining
 /pool/main/libc/libcap2/libcap2_2.25-1.2_amd64.deb
 /pool/main/z/zsh/zsh_5.4.2-3ubuntu3_amd64.deb

Cleanup components that DO NOT MATCH this expression

For the expression com/sonatype

^(?!.*?com/sonatype/).*$

Repository
/org/apache/commons/lang3/StringUtils.java
/com/google/guava/collect/ImmutableList.java
/java/util/ArrayList.java
/com/sonatype/nexus/repository/Repository.java
/org/eclipse/aether/impl/internal/com/sonatype/Logging.java

Remaining
/com/sonatype/nexus/repository/Repository.java
/org/eclipse/aether/impl/internal/com/sonatype/Logging.java

Cleanup components that DO NOT MATCH more than one expression

^(?!.*?(?:/org/sonatype/|/com/sonatype/)).*$

Repository
/org/apache/maven/project/MavenProject.java
/com/google/inject/AbstractModule.java
/my/own/sonatype/unrelated/file.txt
/home/user/.m2/repository/com/sonatype/nexus/nexus-client-core/2.7/nexus-client-core-2.7.jar
/org/sonatype/suffix/file.xml
/another/path/prefix/com/sonatype/suffix/data.json

Remaining
/home/user/.m2/repository/com/sonatype/nexus/nexus-client-core/2.7/nexus-client-core-2.7.jar
/org/sonatype/suffix/file.xml
/another/path/prefix/com/sonatype/suffix/data.json