Configuring Blob Stores

You can configure new blob stores by navigating to Administration  Repository  Blob Stores in Nexus Repository. You will need nx-all or nx-blobstore privileges to access this portion of Nexus Repository.

The following fields appear in the blob store listing:

  • Name  - The blob store's name as displayed in repository administration.

  • Type  - The type of the blob store backend. See the System Requirements for a full list of supported file systems. The following options are available:

    • Azure Cloud Storage  PRO - Stores blobs in Azure cloud storage. 

    • File - Store blobs in file system-based storage.

    • Group  PRO - Combines multiple blob stores into one. 

    • S3  - Store blobs in AWS S3 cloud storage.

  • State  - The state of the blob store.
    • Started indicates it is running as expected.
    • Failed indicates that there is a configuration issue and, as a consequence, the blob store failed to initialise.
  • Blob Count - The number of blobs currently stored in a blob store.
  • Total Size - The blob store's total size in bytes.
  • Available Space  - A blob store's remaining storage capacity.

Click on a specific row to see the absolute Path to the file system storage (for file system blob stores) and the Soft Quota of the selected blob store.

We recommend that your blob store location be outside of the $data-dir directory and read/write accessible by the node.

Creating a New Blob Store

  1. To create a new blob store, select the Create blob store button.
  2. Select the Type and provide a Name for your blob store.
  3. For file system blob stores, in the Path field, provide the absolute path to the desired file system location. It must be fully accessible by the operating system user account that is running Nexus Repository.

Once you have created a blob store, you will not be able to modify it. You will also not be able to delete any blob store that a repository or repository group uses.

Blobs deleted in a repository are only marked for deletion (i.e., a soft delete). You can use the Compact blob store task to permanently delete these soft-deleted blobs and therefore free up the used storage space.

The following sections cover specific information for each supported file system or blob store type.

NFS v4 File System

The recommended settings for an NFS v4 server in /etc/exports are as follows:

/srv/blobstores 10.0.0.0/8(rw,no_subtree_check)

The recommended settings for mounting the NFS filesystem on the node:

defaults,noatime,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2,vers=4.1
Versions of NFS older than v4 are not supported.

AWS Elastic File System (EFS)

EFS acts as a limitless NFS v4 server and is an option if Nexus Repository is running in AWS. Mount the EFS volume with the same settings as NFS v4.

EFS performance will be lower than a dedicated NFS server. 

AWS Simple Storage Service (S3)

You can configure blob stores to use AWS Simple Storage Service.

Blobs are stored in an S3 bucket by using AWS REST APIs over HTTP. Object PUTs use multi-part uploads of approximately 5MB chunks. Since HTTP traffic to S3 blobstores introduces more I/O latency across the network, we recommend using S3 only if Nexus Repository is running on EC2 instances within AWS.

Nexus Repository automatically creates an S3 bucket when you create a blob store if one does not already exist; however, you may also create the bucket yourself. Note the following:

  • Nexus Repository will automatically apply a lifecycle rule to expire deleted content.
  • The bucket can use server-side encryption with KMS key management transparently or S3 managed encryption. 
  • If running on EC2 instances, Nexus Repository can use the IAM role assigned to those instances when accessing S3 buckets.

The AWS user for accessing the S3 Blobstore bucket needs to be granted permission for these actions on the S3 bucket resource:

  • s3:PutObject
  • s3:GetObject
  • s3:DeleteObject
  • s3:ListBucket
  • s3:GetLifecycleConfiguration
  • s3:PutLifecycleConfiguration
  • s3:PutObjectTagging
  • s3:GetObjectTagging
  • s3:DeleteObjectTagging
  • s3:GetBucketAcl ( used for problem diagnosis )  

In addition, for Nexus Repository to create and delete buckets automatically, an IAM policy associated with the Nexus Repository user must include the following additonal permissions:

  • s3:DeleteBucket
  • s3:CreateBucket

Below is a sample minimal bucket policy where <user-arn> is the ARN of the AWS user and <s3-bucket-name> the S3 bucket name:

{
    "Version": "2012-10-17",
    "Id": "NexusS3BlobStorePolicy",
    "Statement": [
        {
            "Sid": "NexusS3BlobStoreAccess",
            "Effect": "Allow",
            "Principal": {
                "AWS": "<user-arn>"
            },
            "Action": [
                "s3:PutObject",
                "s3:GetObject",
                "s3:DeleteObject",
                "s3:ListBucket",
                "s3:GetLifecycleConfiguration",
                "s3:PutLifecycleConfiguration",
                "s3:PutObjectTagging",
                "s3:GetObjectTagging",
                "s3:DeleteObjectTagging",
                "s3:GetBucketAcl"
            ],
            "Resource": [
                "arn:aws:s3:::<s3-bucket-name>",
                "arn:aws:s3:::<s3-bucket-name>/*"
            ]
        }
    ]
}


Here is a sample IAM policy to allow creation and deletion of buckets:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:DeleteBucket",
                "s3:CreateBucket"
            ],
            "Resource": [
                "arn:aws:s3:::*"
            ]
        }
    ]
}

For optimum performance:

  • Run Nexus Repository on AWS on EC2 instances.
  • Ensure that the S3 connection is using the region in which Nexus Repository is run.

Note that S3 has a hard limit of 10,000 parts for multi-part uploads. The chunk size when uploading to S3 can be adjusted by setting the property nexus.s3.multipartupload.chunksize in the nexus.properties file. The unit is bytes and the default is 5242880 (5MB). This can be tuned for optimal performance on your network and to reduce the number of parts uploaded to S3 so as not to receive errors on large uploads.

Access Denied Errors

Access denied errors are an indication that the user configured to connect to the bucket has insufficient permissions. AWS does not provide information specific to which permission is required to perform the failed action. For this reason, Nexus Repository will generically report access denied errors.

Azure Blob Store

PRO

NEW IN 3.31.0

You must create the Azure storage account in Azure before using Nexus Repository to create an Azure blob store. Below are the recommended storage account settings:

  • Location: the location hosting Nexus Repository
  • Performance: Standard or Premium
  • Account kind: StorageV2
  • Replication: Any

Nexus Repository will automatically create an Azure container when a blob store is created if one does not already exist.

The Azure storage container name must be a valid DNS name that follows the rules that Microsoft states in its documentation

There are three methods of gaining access to the Azure storage account from Nexus Repository:

  1. Use a secret access key supplied by the Azure storage account.
  2. If you're running Nexus Repository on an Azure VM, you can use System Managed Identity access.
  3. Use environment variables.

System Managed Identity allows Azure to manage the access via roles assigned to the VM in which you are running Nexus Repository. See the Microsoft documentation for details.

To properly use the System Managed Identity, the Azure VM will need the following roles assigned to the Azure storage container:

  • Storage Account Contributor
  • Storage Blob Data Contributor

Nexus Repository does not validate that the proper roles are assigned before storing the configuration. If the aforementioned roles are not properly granted to the VM, you will need to delete the blob store and then add it again after the roles have been set up in the Azure storage instance.

For optimum performance, you'll want to take the following steps:

  • Run Nexus Repository on Azure on virtual machines
  • Ensure that the Azure connection is using the location where Nexus Repository is being run

The chunk size when uploading to Azure can be adjusted by setting the property nexus.azure.blocksize in the nexus.properties file (e.g., nexus.azure.blocksize=1000000). By default, this is set to 5242880 bytes (5MB). You can tune this for optimal performance on your network.

Promoting a Blob Store to a Group

To promote a blob store to a group, select the Promote to group button. This launches the promotion process to add your blob store to a group for more flexibility. Follow the on-screen prompts to create the blob store group, which will contain the previously concrete blob store and to which you can add other blob stores.

You cannot undo promoting a blob store to a group.

What is a Fill Policy?

When configuring a blob store group, you will be asked to select a fill policy (i.e., a write policy). A fill policy is the method that the blob store group uses to choose a member for writing blobs. You can change the fill policy at any time.

Available fill policy choices include the following:

  • Round Robin - Incoming writes alternate between all blob stores in the group. If writing to one blob store fails, then it attempts again with the next blob store in the group. This is useful when you have a number of blob stores available and you want each of them to receive a roughly equal number of writes. This does not balance based upon any other metric.
  • Write to First - All incoming writes are given to the first writeable blob store (skipping blob stores in a read-only state). If you need to direct all blobs to a specific blob store (e.g., you have a blob store for a new empty disk), then this fill policy will ensure that the new blob store gets all the writes.

Removing a Blob Store from a Group

To remove a blob store from a group, you will need to use the Admin - Remove a member from a blob store group task to ensure that repositories still have access to their blobs. Groups allow you to add members dynamically, but removing a blob store requires a task to ensure that repositories still have access to their blobs.

Moving a Blob Store

The following steps can be used to move a blob store to a new location. You can also use these steps to change a blob store's storage (e.g., moving a file-based blob store to S3).

  1. Create a new blob store with the storage path set to the new location.
  2. Promote the existing to a group.
  3. When asked in the form, set the new group's  Fill Policy to Write to First.
  4. Add the new blob store that you created in step 1 to the newly promoted group blob    store underneath the original blob store.
  5. Schedule and run an Admin - Remove a member from blob store group task via  Administration → System → Tasks to remove the original blob store from the group.

The original blob store's contents will be moved over to the new blob store before the original blob store is removed. 

Splitting a Blob Store

  1. Determine the repositories you wish to move. (Also see this Support article Investigating Blob Store and Repository Size and Space Usage.)
  2. Create new blob store(s) (See Configuring Blob Stores).
  3. Use the Admin - Change Repository Blob Store task to move content.

Adding a Soft Quota

 A soft quota will never block read or write operations on the blob store

When you set a soft quota, Nexus Repository monitors a blob store and raises an alert when it exceeds your configured constraint. You can configure a soft quota by following these steps:

  1. Navigate to Administration → Blob Stores in the Nexus Repository menu.
  2. Select the blob store for which you would like to configure the soft quota.
  3. In the blob store form, check the Enable Soft Quota box.
  4. Select the Type of Quota from the following choices available in the drop-down menu:
    1. Space Used - Issues an alert when a blob store exceeds your quota limit for total bytes used.
    2. Space Remaining - Issues an alert when the space available for new blobs drops below your quota limit.
  5. Under Quota Limit in MD, enter the number of MB at which you would like to receive an alert for the quota type you selected.
  6. Select Save.

When a soft quota is violated, that information will be available in the following locations:

  • WARN level message in the logs.
  • Under Administration → Support → Status, all soft quotas' statuses are aggregated in the BlobStores status.
  • A REST API endpoint.