npm Registry

Available in Nexus Repository OSS and Nexus Repository Pro

Introduction

The command line tool npm is a package management solution for Javascript-based development. It is used to create and use node packaged modules and is built into the Javascript platform Node.js.

NXRM supports the npm registry format for proxy repositories. This allows you to take advantage of the packages in the npm registry and other public registries without incurring repeated downloads of packages, since they will be proxied in the repository manager.

In addition, Nexus Repository Manager supports running your own private registry, also known as a hosted repository, using the npm format. You can share internally developed, proprietary packages within your organization via these private registries allowing you to collaborate efficiently across development teams with a central package exchange and storage location.

To share a package or tool with npm, you create a npm package and store it in the npm registry hosted by the repository manager. Similarly, you can use packages others have created and made available in their npm repositories by proxying them or downloading the packages and installing them in your own private registry for third party packages.

To simplify configuration Nexus Repository Manager supports aggregation of npm registries into a single group. This allows you to expose all the external packages from the npm registry and other public registries as well as the private registries as one registry, which greatly simplifies client configuration.

Proxying npm Registries

To reduce duplicate downloads and improve download speeds for your developers and CI servers, you should proxy the registry served at https://registry.npmjs.org. By default npm accesses this registry directly. You can also proxy any other registries you require.

To proxy an external npm registry, you simply create a new npm (proxy) as documented in Repository Management.

Minimal configuration steps are:

  • Define Name
  • Define URL for Remote storage e.g. https://registry.npmjs.org
  • Select Blob store for Storage

Private npm Registries

A private npm registry can be used to upload your own packages as well as third-party packages. You can create a private npm registry by setting up a hosted repository with the npm format in the repository manager. It is good practice to create two separate hosted repositories for these purposes.

To create a hosted repository with npm format, simply create a new npm (hosted) as documented in Repository Management.

Minimal configuration steps are:

  • Define Name
  • Select Blob store for Storage

The npm registry information is immediately updated as packages are deployed or deleted from the repository.

Grouping npm Registries

A repository group is the recommended way to expose all your npm registries repositories from the repository manager to your users, with miminal client side configuration. A repository group allows you to expose the aggregated content of multiple proxy and hosted repositories with one URL to npm and other tools. This is possible for npm repositories by creating a new npm (group) as documented in Repository Management.  

Minimal configuration steps are:

  • Define Name
  • Select Blob store for Storage
  • Add npm repositories to the Members list in the desired order

A typical, useful example would be to group a npm hosted repository with internal software packages, another npm hosted repository with third-party packages and the proxy repository that proxies the npm registry.

Using the URL of the repository group as your npm repository URL in your client tool will give you access to the packages in all three repositories with one URL. Any new packages added as well as any new repositories added to the group will automatically be available.

NEW IN 3.28

Using a Pro version, you can push images to the group repository as documented in Publishing npm packages.

Browsing npm Registries and Searching Modules

You can browse npm repositories in the user interface inspecting the components and assets and their details as documented in Browsing Repositories and Repository Groups.

Searching for npm modules can be performed in the user interface as described in Searching for Components. This search will find all npm modules images that are currently stored in the repository manager, either because they have been pushed to a hosted repository or they have been proxied from an upstream repository and cached in the repository manager.

Configuring npm

Once you have set up your hosted and proxy repositories for npm packages, and created a repository group to merge them, you can access them with the npm tool on the command line as one registry.

You can configure the registry used by npm in your .npmrc file located in your user's home directory with the npm config command and the public URL of your repository group available in the repository list by clicking the copy button in the URL column of either Administration → Repository → Repositories or user's Browse page.

npm config set registry http://localhost:8081/repository/npm-all/

The command inserts the configuration in the .npmrc file in your users home directory.

Registry configuration in .npmrc

registry = http://localhost:8081/repository/npm-all/

With this configuration any npm commands will use the new registry from the repository manager. The command line output will reference the URLs in --verbose mode or with info logging for the downloads of the requested packages:

$ npm --loglevel info install grunt
...
npm http fetch GET http://localhost:8081/repository/npmjs-org/grunt/-/grunt-0.4.5.tgz
npm http fetch 200 http://localhost:8081/repository/npmjs-org/grunt/-/grunt-0.4.5.tgz
...
npm http fetch GET http://localhost:8081/repository/npm-all/underscore/-/underscore-1.7.0.tgz
npm http fetch 200 http://localhost:8081/repository/npm-all/underscore/-/underscore-1.7.0.tgz
...

npm Security

If enabled, any anonymous user has read access to the repositories and repository groups. If anonymous access, as documented in Anonymous Access, is disabled or write access is required for publishing a package, the user needs to authenticate to the repository manager. There are two methods to authenticate npm with your repository manager. Only one should be used at a time.

Authentication Using Realm and Login

This authentication method requires the npm Bearer Token Realm. Simply add the realm to the active realms in the Realms feature of the Security menu from the Administration menu to activate it as documented in Realms.

Once the realm is activated, a npm CLI user can establish the authentication to a repository with the npm adduser  ( npm login is an equivalent alias ) command.

npm adduser --registry=http://localhost:8081/repository/npm-internal/

Provide your repository manager username and password as well as an email address when prompted. Upon successful completion, a line for authentication of this combination is automatically added to your .npmrc configuration file for the specific repository.

Despite its name, the npm adduser command does not actually create a user account inside NXRM. It merely associates a token with an existing user account and allows the CLI to store that token for re-use.

Authentication Using Basic Auth

In some instances you cannot use the realm and login method, for example if you have a username which includes capital letters (disallowed by npm login). In these you can still use npm by configuring it to use basic auth with your repository manager. This authentication method involves editing the .npmrc configuration file adding an encoded username and password as well as configuring authentication to always occur. It is considered the less flexible of the methods supported.

The _auth variable has to be generated by base64-encoding the string of username:password. You can create this encoded string with the command line call to openssl e.g. for the admin user using password admin123:

echo -n 'admin:admin123' | openssl base64

Other tools for the encoding are uuencode or, for Windows users, certutil. To use certutil on Windows you need to put the credentials to be encoded into a file:

admin:admin123

Ensure your file does not have extra whitespace or a trailing line separator as either of these will negatively impact the resultant output.

Then run:

c:\certutil /encode in.txt out.txt 

After this the base64 encoded credentials can be found in between the begin and end certificate lines in the output file:

-----BEGIN CERTIFICATE-----
YWRtaW46YWRtaW4xMjM=
-----END CERTIFICATE-----

Whatever tool you use to generate the encoded username and password string can be tested by encoding the string admin:admin123 , which should result in YWRtaW46YWRtaW4xMjM= . Another example is jane:testpassword123 which should result in amFuZTp0ZXN0cGFzc3dvcmQxMjM=.

Once you have encoded credentials they can be added to the .npmrc file, along with your author email and enabled authentication (below your already entered registry configuration). For example, for admin/admin123:

email=you@example.com
always-auth=true
_auth=YWRtaW46YWRtaW4xMjM= 

With one (not both) of these authentication methods in place, you are ready to publish.

Publishing npm Packages

Publishing your own packages to a npm hosted repository allows you to share packages across your organization or with external partners. With authentication configured you can publish your packages with the npm publish command.

The npm publish command uses a registry configuration value to know where to publish your package. There are several ways to change the registry value to point at your hosted npm repository.

Since the .npmrc file usually contains a registry value intended only for getting new packages, a simple way to override this value is to provide a registry to the publish command:

npm publish --registry http://localhost:8081/repository/npm-internal/

Alternately, you can edit your package.json file and add a publishConfig section:

"publishConfig" : {
  "registry" : "http://localhost:8081/repository/npm-internal/"
}, 

Detailed information about package creation can be found on the npm website.

If your package requires the use of npm scope, the repository manager supports this functionality. Packages published to the repository manager with a defined scope are reflected with the scope value populating the repository group field in Browse and Search. Details on scoping are available on the npm website also.

Once a package is published to the private registry in the repository manager, any other developers or build servers that access the repository manager via the repository group have instant access to it.

NEW IN 3.28

Using a Pro version, you can push images to the group repository. In that case, you have to select a Writable Repository in the configuration menu.

The procedure of pushing images  is otherwise the same as for a hosted repository.

Deprecating npm Packages

Once your packages have been pushed to an npm hosted repository, you can mark them as deprecated. This is useful when a newer version of the package is available and you want to warn people that the old package has reached end of life, or you want to avoid usage and warn your users for some other reason.

The npm deprecate command uses a registry configuration value to inform where the package lives. To deprecate an existing package, use a command like the following:

npm deprecate --registry http://localhost:8081/repository/npm-internal/ testproject1@0.0.1 "This package is deprecated" 

If you change your mind, you can reverse this action using the same command. To un-deprecate a package, pass an empty string to the deprecate command:

npm deprecate --registry http://localhost:8081/repository/npm-internal/ testproject1@0.0.1 ""

The message text is persisted in the deprecated attribute of the packageJson section for the asset and can be viewed in the user interface.

npm audit

The npm audit command submits a description of the dependencies configured in your project to the registry configured in your .npmrc and asks for a report of policy violations. The report returned includes instructions on how to act on this information.

Detailed information about npm audit can be found on the npm website . 

Using Nexus IQ Server you can easily configure verification of your npm project not only against security vulnerabilities but also against your own policy enforcement, like component age or using a certain kind of license. All policy violations will be aggregated in the npm report. You can find more information about policy management and policy configuration.


Example

Add a policy with type License in IQ Server:

In the npm report you will get warning of this violation:

npm audit requirements

  • Requires an IQ server ( Lifecycle or Firewall ) and configuration
  • If you want to use  npm audit without APP_ID, you need to use IQ Server release 89 or higher.

If configuration is incomplete or incorrect, you will receive this message: 

Setup

Audit information is locally cached for a period of 12 hours. Invalidating Cache at the repository level will additionally clear the audit cache.

An IQ server is required to use this feature with NXRM. You have the option to evaluate npm packages in the context of the repository or you can specify an application ID for each project.

Link your IQ server in NXRM following this guide.

Also you need to configure  your npm project within NXRM in order to  use the npm audit command. Details on the configuration  of npm can be found above in Configuring npm  section.

npm audit is supported by only proxy and group repositories.

There are three different methods to evaluate your npm projects, described below. Each method is ordered in preference but may depend on your usecase.

Methods

  1. Adding an app_id identifier within a project local npmrc file
  2. Introduce a script into your project to inject an app_id
  3. No app_id declared (scans against root organization policies)


npm v6.14.4 limits requests to 30 seconds. You may have to add the  --timeout flag to the npm audit command. For example, npm audit --timeout=300000 which will limit a request to 5 minutes.

Specifying an application id within a project local npmrc file

  NEW IN 3.29

Creating a project local .npmrc file and adding the following lines:

headers[]="app_id:<application_id>"

<application_id> should be replaced with the application name in your IQ server you would like the audit to be conducted under.

This approach unlocks the full functionality of  npm audit  by scanning against application defined policies.

Running npm audit will produce a report listing the application policies that your build will violate, for example:

With application id in package-lock.json / npm-shrinkwrap.json

This method also unlocks the full functionality of npm audit you will need to provide the APP_ID parameter in the package-lock.json file. The value of this parameter should be one of the application ids from the IQ Server. You can inject the APP_ID into your package-lock.json or npm-shrinkwrap.json by adding a script to your package.json file. For example, using a script called add_id.sh:

app_id.sh

node -p "fs = require('fs');fp = './' + (fs.existsSync('package-lock.json') ? 'package-lock.json' : 'npm-shrinkwrap.json');fs.writeFileSync(fp, JSON.stringify({...require(fp), APP_ID: '$npm_package_APP_ID'}, null, 2));"

package.json

{
  "name": "my-super-project",
  "version": "0.0.1",
  "description": "Test Project",
  "main": "index.js",
  "dependencies": {
    "debug": "^2.2.0",
    "qs": "^2.3.3"
  },
  "optionalDependencies": {
    "lodash": "2.4.2"
  },
 "APP_ID": "<APP_ID>",
 "scripts": {
    "postshrinkwrap": "bash app_id.sh"
  }
}

The result of npm install should be package-lock.json / npm-shrinkwrap.json, for example:

{
"name":"my-super-project",
"version":"0.0.1",
"lockfileVersion":1,
"requires":true,
"dependencies":{
"debug":{
"version":"2.6.9",
"resolved":"https://registry.npmjs.org/debug/-/debug-2.6.9.tgz",
"integrity":"sha512-bC7ElrdJaJnPbAP+1EotYvqZsb3ecl5wi6Bfi6BJTUcNowp6cvspg0jXznRTKDjm/E7AdgFBVeAPVMNcKGsHMA==",
"requires":{
"ms":"2.0.0"
}
},
"lodash":{
"version":"2.4.2",
"resolved":"https://registry.npmjs.org/lodash/-/lodash-2.4.2.tgz",
"integrity":"sha1-+t2DS5aDBz2hebPq5tnA0VBT9z4=",
"optional":true
},
"ms":{
"version":"2.0.0",
"resolved":"https://registry.npmjs.org/ms/-/ms-2.0.0.tgz",
"integrity":"sha1-VgiurfwAvmwpAd9fmGF4jeDVl8g="
},
"qs":{
"version":"2.4.2",
"resolved":"https://registry.npmjs.org/qs/-/qs-2.4.2.tgz",
"integrity":"sha1-9854jld33wtQENp/fE5zujJHD1o="
}
},
"APP_ID":"<APP_ID>"
}

If you don't get APP_ID in your package-lock.json, try npm cache clean --force then  npm clean install  command.

Running npm audit will produce a report listing the policies that your build will violate:

Without specifying application id in package-lock.json / npm-shrinkwrap.json

If you do npm audit without specifying an APP_ID in your package-lock.json or npm-shrinkwrap.json, NXRM will download and cache the packages before evaluating them. This may cause a short delay the first time a project is evaluated.

IQ Server release 89 is required.

Using this method the report  will be not  as detailed  as using  APP_ID , but it will contain all  policy violations under the root organization as shown here:

npm audit fix

If you have configured an APP_ID then you can use npm audit fix to automatically remediate any vulnerable dependencies.

Removing non-cataloged package versions for proxied repositories

NEW IN 3.29.0

Enforcing an IQ server policy to block non-cataloged components can lead to build errors and can be difficult for developers to troubleshoot. The npm package metadata will contain all available versions however retrieval of a non-cataloged tarball will fail when Firewall is enabled. Often this happens when a project uses the latest tag for a dependency and that dependency was recently updated and IQ server has not yet cataloged the new version. Manual intervention can be taken to pin versions but requires handling both direct and transitive dependencies. As of 3.29, you can configure Repository Manager to remove non-catalogued versions from npm package metadata. With this option enabled npm will only use new versions that are known to Nexus Intelligence . Once the component is know it will begin showing up in the proxied metadata. Two settings are needed to enable this behavior. First, the proxy repository must have the "IQ: Audit and Quarantine" capability enabled. Second, enable the new "Remove Non-cataloged Versions" option in the repository settings page.

Remove Quarantined Versions: Policy-Compliant Component Selection for npm

NEW IN 3.35.0

When a user requests an npm package without explicitly specifying a version (e.g., npm install package@1.1), the npm client relies on the package metadata from the npm registry to select a version that satisfies the version constraints. If the selected version has policy violation and is quarantined by Nexus Firewall, it will cause a build failure that requires a manual fix of the root cause.

By enabling this option, Nexus Firewall will remove quarantined versions from the npm package metadata to prevent you from selecting a version with policy violations.

The Remove Quarantined Versions option will eventually replace the Remove Non-cataloged Versions option since, using properly configured policy, it is able to do that on top of additional functionality.

Two settings are needed to enable this behavior:

  1. Enable the IQ: Audit and Quarantine capability on the proxy repository.
  2. Check the Remove Quarantined Versions box in the Nexus Repository settings page.