Staging Concepts

Another way to think about staging is the lifecycle of the artifacts your organization produces. In this context, an artifact is one of the tangible by-products produced during software development. Artifacts commonly go through a set of lifecycle phases, or ‘stages’, where they are promoted from one stage to the next as they pass automated quality checks. A core DevSecOps principle is to fail fast. If a release candidate fails checks, they are quickly dropped to not take up time and space in the workflow. Anything left around too long will be removed by regular cleanup policies.

Production environments need to be protected from artifacts that have not passed the required quality checks. The risk to organizations is too great. Stages make it easier to control access to artifacts through the pipeline. Here are the stages commonly found in use.

The Sonatype Nexus Repository best practice in staging is to funnel access through group repositories aligned to lifecycle stages. This gives administrators control over the mix of repositories, and in turn artifacts, available in a given stage. This mix can later be edited without having to change the access endpoints used by the pipeline. Even with one repository in a group, this step will make the management of environments easier in the long term.

Deploying to a repository, tagging of release candidates, and the promotion of artifacts should happen as a part of an automated CI/CD pipeline. Any dependence on a manual workflow may lead to errors and does not scale well. This is why the staging and tagging functions are done through the platform plugins and API calls. The workflow can be simulated through Sonatype Nexus Repository's built-in swagger interface to test out the calls mentioned below.

Planning for cleanup is a Nexus administrator’s top priority. The number of artifacts moving into the repository will need to be matched by the artifacts being cleaned. This is critical to keeping server costs manageable and avoiding running out of space. Staging environments are an effective means to clean up the repository.

Aligning cleanup policies with the artifact’s lifecycle makes more sense when selecting by age and last downloaded. Review the Cleanup Policies for suggestions on configuring this effectively for different stages and formats.

When designing the initial layout of repositories in NXRM3 there are 2 common models used.

Generally, the best practice is to standardize on the format-based model however there are very good reasons to leverage the team-based model. This is especially true when the number of teams will not grow over time and their pipelines will not overlap. For some organizations, they may wish to leverage both models where most development teams share the same pipeline and a limited few keep their own. With either model, the deployment of the staging environments will be a similar process. They should take careful planning for repository naming to make managing them easier in the long run.

Staging is intended to fit into an organization’s existing workflow and should not require a complex setup. To start, add group repositories with a corresponding hosted repository for each stage. Here is a simple example.

In this example, there are three stages however this could also be done with just two; dev and prod.

The staging API uses a simple POST to move artifacts from a source repository to a destination repository.

POST service/rest/v1/staging/move/{repository}

This post uses the same structure to search for components as the Search API. Below is the endpoint with a simple search for an artifact in the [maven-dev-hosted] repository which we will move to the [maven-uat-hosted] repository.

POST service/rest/v1/staging/move/maven-uat-hosted?repository=maven-dev-hosted&group=org.osgi&name=org.osgi.core&version=4.3.1

The move command can be made against any number of artifacts that match the search criteria. It can be further simplified using a tag that is associated with the artifacts of the release candidate.

POST service/rest/v1/staging/move/maven-uat-hosted?repository=maven-dev-hosted&tag=maven-build-100

The only pieces required for tagging are creating your tag “name” as a unique identifier and assigning artifacts to the tag. We recommend using a name that is meaningful and unique but for performance reasons, not too long. Aligning identifiers with ids used in the release process is ideal. In this example, the “attributes” are optional and can contain any JSON data that would be meaningful to associate with the release candidate.

curl -u admin:admin123 -X POST -H 'Content-Type: application/json' \
'http://localhost:8081/service/rest/v1/tags' \
-d '{"name": "org.osgi.core_4.3.1","attributes": {"built-by": "jenkins"}}'

Associating artifacts to a tag is similar to the search API, where any artifacts identified with the search will be tagged. It is usually better to associate the tag when uploading the artifacts to the repository. This can be done using API or with any of the CI tooling used to upload.

POST /service/rest/v1/tags/associate/{tagName}