npm Registry

Available in Nexus Repository OSS and Nexus Repository Pro

Introduction

The command line tool npm is a package management solution for Javascript-based development. It is used to create and use node packaged modules and is built into the Javascript platform Node.js.

NXRM supports the npm registry format for proxy repositories. This allows you to take advantage of the packages in the npm registry and other public registries without incurring repeated downloads of packages, since they will be proxied in the repository manager.

In addition, Nexus Repository Manager supports running your own private registry, also known as a hosted repository, using the npm format. You can share internally developed, proprietary packages within your organization via these private registries allowing you to collaborate efficiently across development teams with a central package exchange and storage location.

To share a package or tool with npm, you create a npm package and store it in the npm registry hosted by the repository manager. Similarly, you can use packages others have created and made available in their npm repositories by proxying them or downloading the packages and installing them in your own private registry for third party packages.

To simplify configuration Nexus Repository Manager supports aggregation of npm registries into a single group. This allows you to expose all the external packages from the npm registry and other public registries as well as the private registries as one registry, which greatly simplifies client configuration.

Proxying npm Registries

To reduce duplicate downloads and improve download speeds for your developers and CI servers, you should proxy the registry served at https://registry.npmjs.org. By default npm accesses this registry directly. You can also proxy any other registries you require.

To proxy an external npm registry, you simply create a new npm (proxy) as documented in Repository Management.

Minimal configuration steps are:

  • Define Name
  • Define URL for Remote storage e.g. https://registry.npmjs.org
  • Select Blob store for Storage

Private npm Registries

A private npm registry can be used to upload your own packages as well as third-party packages. You can create a private npm registry by setting up a hosted repository with the npm format in the repository manager. It is good practice to create two separate hosted repositories for these purposes.

To create a hosted repository with npm format, simply create a new npm (hosted) as documented in Repository Management.

Minimal configuration steps are:

  • Define Name
  • Select Blob store for Storage

The npm registry information is immediately updated as packages are deployed or deleted from the repository.

Grouping npm Registries

A repository group is the recommended way to expose all your npm registries repositories from the repository manager to your users, with miminal client side configuration. A repository group allows you to expose the aggregated content of multiple proxy and hosted repositories with one URL to npm and other tools. This is possible for npm repositories by creating a new npm (group) as documented in Repository Management.  

Minimal configuration steps are:

  • Define Name
  • Select Blob store for Storage
  • Add npm repositories to the Members list in the desired order

A typical, useful example would be to group a npm hosted repository with internal software packages, another npm hosted repository with third-party packages and the proxy repository that proxies the npm registry.

Using the URL of the repository group as your npm repository URL in your client tool will give you access to the packages in all three repositories with one URL. Any new packages added as well as any new repositories added to the group will automatically be available.

Browsing npm Registries and Searching Modules

You can browse npm repositories in the user interface inspecting the components and assets and their details as documented in Browsing Repositories and Repository Groups.

Searching for npm modules can be performed in the user interface as described in Searching for Components. This search will find all npm modules images that are currently stored in the repository manager, either because they have been pushed to a hosted repository or they have been proxied from an upstream repository and cached in the repository manager.

Configuring npm

Once you have set up your hosted and proxy repositories for npm packages, and created a repository group to merge them, you can access them with the npm tool on the command line as one registry.

You can configure the registry used by npm in your .npmrc file located in your user’s home directory with the npm config command and the public URL of your repository group available in the repository list by clicking the Copy button in the URL column.

npm config set registry http://localhost:8081/repository/npm-all/

The command inserts the configuration in the .npmrc file in your users home directory.

Registry configuration in .npmrc

registry = http://localhost:8081/repository/npm-all/

With this configuration any npm commands will use the new registry from the repository manager. The command line output will reference the URLs in --verbose mode or with info logging for the downloads of the requested packages:

$ npm --loglevel info install grunt
...
npm http fetch GET http://localhost:8081/repository/npmjs-org/grunt/-/grunt-0.4.5.tgz
npm http fetch 200 http://localhost:8081/repository/npmjs-org/grunt/-/grunt-0.4.5.tgz
...
npm http fetch GET http://localhost:8081/repository/npm-all/underscore/-/underscore-1.7.0.tgz
npm http fetch 200 http://localhost:8081/repository/npm-all/underscore/-/underscore-1.7.0.tgz
...

npm Security

If enabled, any anonymous user has read access to the repositories and repository groups. If anonymous access, as documented in Anonymous Access, is disabled or write access is required for publishing a package, the user needs to authenticate to the repository manager. There are two methods to authenticate npm with your repository manager. Only one should be used at a time.

Authentication Using Realm and Login

This authentication method requires the npm Bearer Token Realm. Simply add the realm to the active realms in the Realms feature of the Security menu from the Administration menu to activate it as documented in Realms.

Once the realm is activated, a npm CLI user can establish the authentication to a repository with the npm adduser ( npm login is an equivalent alias ) command.

npm adduser --registry=http://localhost:8081/repository/npm-internal/

Provide your repository manager username and password as well as an email address when prompted. Upon successful completion, a line for authentication of this combination is automatically added to your .npmrc configuration file for the specific repository.

Despite its name, the npm adduser command does not actually create a user account inside NXRM. It merely associates a token with an existing user account and allows the CLI to store that token for re-use.

Authentication Using Basic Auth

In some instances you cannot use the realm and login method, for example if you have a username which includes capital letters (disallowed by npm login). In these you can still use npm by configuring it to use basic auth with your repository manager. This authentication method involves editing the .npmrc configuration file adding an encoded username and password as well as configuring authentication to always occur. It is considered the less flexible of the methods supported.

The _auth variable has to be generated by base64-encoding the string of username:password. You can create this encoded string with the command line call to openssl e.g. for the admin user using password admin123:

echo -n 'admin:admin123' | openssl base64

Other tools for the encoding are uuencode or, for Windows users, certutil. To use certutil on Windows you need to put the credentials to be encoded into a file:

admin:admin123

Ensure your file does not have extra whitespace or a trailing line separator as either of these will negatively impact the resultant output.

Then run:

c:\certutil /encode in.txt out.txt 

After this the base64 encoded credentials can be found in between the begin and end certificate lines in the output file:

-----BEGIN CERTIFICATE-----
YWRtaW46YWRtaW4xMjM=
-----END CERTIFICATE-----

Whatever tool you use to generate the encoded username and password string can be tested by encoding the string admin:admin123 , which should result in YWRtaW46YWRtaW4xMjM= . Another example is jane:testpassword123 which should result in amFuZTp0ZXN0cGFzc3dvcmQxMjM=.

Once you have encoded credentials they can be added to the .npmrc file, along with your author email and enabled authentication (below your already entered registry configuration). For example, for admin/admin123:

email=you@example.com
always-auth=true
_auth=YWRtaW46YWRtaW4xMjM= 

With one (not both) of these authentication methods in place, you are ready to publish.

Publishing npm Packages

Publishing your own packages to a npm hosted repository allows you to share packages across your organization or with external partners. With authentication configured you can publish your packages with the npm publish command.

The npm publish command uses a registry configuration value to know where to publish your package. There are several ways to change the registry value to point at your hosted npm repository.

Since the .npmrc file usually contains a registry value intended only for getting new packages, a simple way to override this value is to provide a registry to the publish command:

npm publish --registry http://localhost:8081/repository/npm-internal/

Alternately, you can edit your package.json file and add a publishConfig section:

"publishConfig" : {
  "registry" : "http://localhost:8081/repository/npm-internal/"
}, 

Detailed information about package creation can be found on the npm website.

If your package requires the use of npm scope, the repository manager supports this functionality. Packages published to the repository manager with a defined scope are reflected with the scope value populating the repository group field in Browse and Search. Details on scoping are available on the npm website also.

Once a package is published to the private registry in the repository manager, any other developers or build servers that access the repository manager via the repository group have instant access to it.

Deprecating npm Packages

Once your packages have been pushed to an npm hosted repository, you can mark them as deprecated. This is useful when a newer version of the package is available and you want to warn people that the old package has reached end of life, or you want to avoid usage and warn your users for some other reason.

The npm deprecate command uses a registry configuration value to inform where the package lives. To deprecate an existing package, use a command like the following:

npm deprecate --registry http://localhost:8081/repository/npm-internal/ testproject1@0.0.1 "This package is deprecated" 

If you change your mind, you can reverse this action using the same command. To un-deprecate a package, pass an empty string to the deprecate command:

npm deprecate --registry http://localhost:8081/repository/npm-internal/ testproject1@0.0.1 ""

The message text is persisted in the deprecated attribute of the packageJson section for the asset and can be viewed in the user interface.

npm audit

The npm audit command submits a description of the dependencies configured in your project to the registry configured in your .npmrc and asks for a report of policy violations. The report returned includes instructions on how to act on this information.

Detailed information about npm audit can be found on the npm website. 

Using Nexus IQ Server you can easily configure verification of your npm project not only against security vulnerabilities but also against your own policy enforcement, like component age or using a certain kind of license. All policy violations will be aggregated in the npm report. Detailed information about policy management can be found here. Also here you can find information about policy configuration.


Example

Add a policy with type License in IQ Server:

In the npm report you will get warning of this violation:

npm audit requirements

  • Requires an IQ server (Lifecycle or Firewall) and configuration
  • If you want to use npm audit without APP_ID, you need to use IQ Server release 89 or higher.

If configuration is incomplete or incorrect, you will receive this message: 

Setup

An IQ server is required to use this feature with NXRM. You have the option to evaluate npm packages in the context of the repository or you can specify an application ID for each project.

Link your IQ server in NXRM following this guide.

Also you need to configure your npm project within NXRM in order to use the npm audit command. Details on the configuration of npm can be found above in Configuring npm section.

npm audit is supported by only proxy and group repositories.

There are two different methods to evaluate your npm projects, described below. The first method introduces a script in your project and evaluates packages under application id, it could a bit faster than second, which pulls packages out of NXRM and evaluates them under the root organization.

npm v6.14.4 limits requests to 30 seconds. You may have to add the --timeout flag to the npm audit command. For example, npm audit --timeout=300000 which will limit a request to 5 minutes.

Without specifying application id in package-lock.json / npm-shrinkwrap.json

If you do npm audit without specifying APP_ID in your package-lock.json or npm-shrinkwrap.json, NXRM will download and cache the packages before evaluating them. This may cause a short delay the first time a project is evaluated.

IQ Server release 89 is required.

Using this method the report will be not as detailed as using APP_ID, but it will contain all policy violations under the root organization as shown here:

With application id in package-lock.json / npm-shrinkwrap.json

To unlock the full functionality of npm audit you will need to provide the APP_ID parameter in the package-lock.json file. The value of this parameter should be one of the application ids from the IQ Server. You can inject the APP_ID into your package-lock.json or npm-shrinkwrap.json by adding a script to your package.json file.  For example, using a script called add_id.sh:

app_id.sh

node -p "fs = require('fs');fp = './' + (fs.existsSync('package-lock.json') ? 'package-lock.json' : 'npm-shrinkwrap.json');fs.writeFileSync(fp, JSON.stringify({...require(fp), APP_ID: '$npm_package_APP_ID'}));"

package.json

{
  "name": "my-super-project",
  "version": "0.0.1",
  "description": "Test Project",
  "main": "index.js",
  "dependencies": {
    "debug": "^2.2.0",
    "qs": "^2.3.3"
  },
  "optionalDependencies": {
    "lodash": "2.4.2"
  },
 "APP_ID": "<APP_ID>",
 "scripts": {
    "postshrinkwrap": "bash app_id.sh"
  }
}

The result of npm install should be package-lock.json / npm-shrinkwrap.json, for example:

{
"name":"my-super-project",
"version":"0.0.1",
"lockfileVersion":1,
"requires":true,
"dependencies":{
"debug":{
"version":"2.6.9",
"resolved":"https://registry.npmjs.org/debug/-/debug-2.6.9.tgz",
"integrity":"sha512-bC7ElrdJaJnPbAP+1EotYvqZsb3ecl5wi6Bfi6BJTUcNowp6cvspg0jXznRTKDjm/E7AdgFBVeAPVMNcKGsHMA==",
"requires":{
"ms":"2.0.0"
}
},
"lodash":{
"version":"2.4.2",
"resolved":"https://registry.npmjs.org/lodash/-/lodash-2.4.2.tgz",
"integrity":"sha1-+t2DS5aDBz2hebPq5tnA0VBT9z4=",
"optional":true
},
"ms":{
"version":"2.0.0",
"resolved":"https://registry.npmjs.org/ms/-/ms-2.0.0.tgz",
"integrity":"sha1-VgiurfwAvmwpAd9fmGF4jeDVl8g="
},
"qs":{
"version":"2.4.2",
"resolved":"https://registry.npmjs.org/qs/-/qs-2.4.2.tgz",
"integrity":"sha1-9854jld33wtQENp/fE5zujJHD1o="
}
},
"APP_ID":"<APP_ID>"
}

If you don't get APP_ID in your package-lock.json, try npm cache clean --force then npm-clean-install commands.

Running npm audit will produce a report listing the policies that your build will violate:

npm audit fix

If you have configured an APP_ID then you can use npm audit fix to automatically remediate any vulnerable dependencies.