npm Application Analysis

Evaluation: Advanced Binary Fingerprinting (ABF)

When a scan includes javascript files, those are considered the primary identification source. Any manifest files in the scan (e.g. package-lock.json) are used to refine the results identified by the javascript files. Manifest entries that do not correspond to any of the scanned javascript files are ignored. If you want all dependencies declared in manifest files to be in the report you should only scan those files and avoid including any javascript files in the report.

Scanning Javascript

Delete the node_modules before scanning to remove any cached projects that will not be included in the final application
For npm, scanning the package-lock.json is required. No results are returned if you only scan the package.json
Both package-lock.json and package.json are required to build the dependency hierarchy tree
For Node projects, use the --production flag to exclude dev dependencies in the package.json
```
npm install --production
```
Yarn
```
yarn install --prod
```
Scan after the npm install command has run and the package-lock.json file is created
Include the application source code to search for embedded javascript files
- Any declared packages are matched to discovered JavaScript target files
- The scan will match any packages present in the scan context path

Supported Configuration

Javascript scanning supports identifying packages from the npm ecosystem.
Results include Security, License, and Identity data.
Use the following scan targets where applicable:
- *.js
- *.ts
- *.tgz
- package.json (requires package-lock.json)
- package-lock.json
- npm-shrinkwrap.json
- yarn.lock
- pnpm-lock.yaml

A-Name files vs npm components

When open-source javascript is used in an application, only part of the cataloged package is integrated. The analysis attempts to match discovered javascript files with npm packages declared in the project manifest.

On a successful match, the component is listed as an npm component.
Javascript files not matched to a declared component are identified as an a-name component.

A-Name, or the Authoritative Name, is the individual javascript files associated with the open-source package they were first cataloged. A 'best guess' matching is made that may not be the component version referenced in your application manifest.

Observed licenses are not supported in a-name matching. License information is taken from the open-source package.
A-name may match other ecosystems or package managers other than npm.
When the file is not identified or the authoritative source is not accurately determined, the component is identified as an unknown javascript file (see below).

When to scan

When including open-source javascript components in an application, the dependencies are often obfuscated during the packaging process and end up modified in the final deployed artifacts. When this happens, the IQ scanners may be unable to identify the open-source components using binary fingerprinting. The following options should be considered to get an accurate analysis of the open-source components being used:

Scan the project with the CI or CLI scanners after the npm 'install' command has run but before any obfuscation process happens.
As mentioned above, delete the node_modules before scanning to remove any cached projects that will not be included in the final application
The copy-modules-webpack-plugin can be used to copy the dependencies into a target directory first to run the analysis.
The project could be scanned directly from source control if other methods are not possible.

Direct and Transitive Dependencies

Javascript analysis includes support for reporting the hierarchy of dependencies in the scan report. The manifest files must be included in the scan for Lifecycle to determine the project's direct dependencies.

Version Recommendations for Transitive Dependencies

The Transitive Solver capability (component versions that fix policy violations for the target component AND its dependencies) is currently available for the Maven ecosystem only.

node_modules

When the node_modules directory is included in the scan context path everything found will be included in the scan results. This will confuse when older dependencies and developer (dev) dependencies have not been cleaned up from this cache. Deleting this directory and running 'npm install --production' will include only the dependencies that are to be in production.

Unknown javascript files

Proprietary javascript files are not known by Sonatype Data Services. Including them in the scan report would generate a large amount of noise without a meaningful way of identifying them using the proprietary component matcher. Therefore by default, they are not displayed in the scan report view. You can view them manually by adding the flag '?unknownjs' to the end of the report URL to show all unknown javascript files.

Evaluation: Manifest and lock files

A manifest analysis is run by specifically scanning only the project-lock and manifest files for the following javascript package managers: npm, yarn, and pnpm. The scanner will default to an ABF scan if any .js files are included in the scan context path.

Lock files: manifest files alone do not include the transitive dependencies and sometimes the specific direct versions that will be used in the final application. This is why we highly recommend including the lock files in the analysis for the best results. It is further important to note that when scanning a lock file via the Evaluate a File option in the Lifecycle UI, a dependency tree will not be produced unless the lock file is put in an archive (e.g. zip file) alongside its corresponding package.json file and that archive is scanned instead.
Direct and Transitive Dependencies: To include the dependency information i.e. Direct vs Transitive, the package.json will need to include an auto-generated lock file along with the manifest files.

If only a lock file is scanned, then a package.json file in the same directory can be used to help determine direct dependencies and development dependencies to be excluded.

This is true if the package.json file:

Exists
Is not excluded
Has a "dependencies" section

If all of these conditions are met, then only the packages specified in the "dependencies" section will be considered for analysis as production dependencies. If the "dependencies" section is empty, then no dependencies will be included.

devDependencies: By default, components under the 'devDependencies' and 'optional' scopes or have the metadata 'dev: true' are excluded from the scan. Here are two ways to include them:

Binary scan should be done including these components in the node_modules directory.
To enable/disable scanning optional and devDependencies for manifest and lock files you can configure the scan-npm-dev-and-opt-dependencies feature using the Feature Configuration REST API.

npm

NPM packages can be identified by scanning the following named lock files:

Filename	Generated By
package-lock.json npm-shrinkwrap.json	npm package manager v6 npm package manager v7 preference is given to npm-shrinkwrap.json file when both files are present in the scan

Filename

Generated By

package-lock.json

npm-shrinkwrap.json

npm package manager v6

npm package manager v7

preference is given to npm-shrinkwrap.json file when both files are present in the scan

Name and version fields from dependencies (or packages for npm v7) objects will be evaluated. For example:

name: ansi-regex
- version: 3.0.0
name: wordwrap
- version: 0.0.3

{ "requires": true, "lockfileVersion": 1, "dependencies": { "ansi-regex": { "version": "3.0.0", "resolved": "https://registry.npmjs.org/ansi-regex/-/ansi-regex-3.0.0.tgz", "integrity": "sha1-7QMXwyIGT3lGbAKWa922Bas32Zg=" }, "wordwrap": { "version": "0.0.3", "resolved": "https://registry.npmjs.org/wordwrap/-/wordwrap-0.0.3.tgz", "integrity": "sha1-o9XabNXAvAAI03I0u68b7WMFkQc=" } }
}

yarn

Filename	Generated By
yarn.lock	Yarn package manager v1 Yarn package manager v2

Filename

Generated By

yarn.lock

Yarn package manager v1

Yarn package manager v2

Name and version fields will be evaluated. For example:

name: @dangl/angular-material-shared
version: 2.0.0

@dangl/angular-material-shared@2.0.0: 
  version "2.0.0" 
@progress/kendo-theme-material@0.3.2: 
  version "0.3.2" 
@angular@0.0.1: 
  version "0.0.1"

pnpm

Filename	Generated By
pnpm-lock.yaml	pnpm package manager

Name and version fields from packages and dependencies objects will be evaluated. For example:

name: @angular-devkit/schematics
version: 8.3.26

lockfileVersion: 5.3 

specifiers: 
  '@angular-devkit/schematics': 8.3.26 

dependencies: 
  '@angular-devkit/schematics': 8.3.26 

packages: 
  /@angular-devkit/schematics/8.3.26: 
    resolution: {integrity: sha512-IoZbXVFGLvVi5d0ozfssWDXuzot0/pMSKbQPzWIG8K7nCo7nNMVYpsMHrEVYUikA9EQEL5LqMCGohH36/zVPcA==} 
    engines: {node: '>= 10.9.0', npm: '>= 6.2.0'} 
    dependencies: 
      '@angular-devkit/core': 8.3.26 
      rxjs: 6.4.0 
      dev: false

Steps to analyze using the Sonatype IQ CLI

Run a scan

Invoke a Nexus IQ CLI scan of a directory or subdirectories containing yarn.lock, pnpm-lock.yaml, package-lock.json, or npm-shrinkwrap.json.

Example pnpm-lock.yaml file

lockfileVersion: 5.3 

specifiers: 
  '@angular-devkit/schematics': 8.3.26 
  '@angular/common': 8.2.14 
  '@angular/compiler': 8.2.14 
  '@angular/core': 8.2.14 
  '@angular/router': 8.2.14 
  '@ng-bootstrap/ng-bootstrap': 5.3.0 
    bootstrap: 4.5.0 
    jquery: 1.9.1 
    rxjs: ^6.4.0 
    zone.js: ~0.9.1 

dependencies: 
  '@angular-devkit/schematics': 8.3.26 
  '@angular/common': 8.2.14_@angular+core@8.2.14+rxjs@6.4.0 
  '@angular/compiler': 8.2.14 
  '@angular/core': 8.2.14_rxjs@6.4.0+zone.js@0.9.1 
  '@angular/router': 8.2.14_ed906ac8447aa5d4a7a8ac33fedcf709 
  '@ng-bootstrap/ng-bootstrap': 5.3.0_ed906ac8447aa5d4a7a8ac33fedcf709 
    bootstrap: 4.5.0_jquery@1.9.1 
    jquery: 1.9.1 
    rxjs: 6.4.0 
    zone.js: 0.9.1 

packages: 
  /@angular-devkit/core/8.3.26: 
    resolution: {integrity: sha512-b1ng9091o33s55/cwQYh1kboiJtj8y8z8xQWATDI9kRmNIQkWYVwVa/MzgPRJ4bzbEGG3zIUHCsp52A6vuGr2A==} 
    engines: {node: '>= 10.9.0', npm: '>= 6.2.0'} 
    dependencies: 
      ajv: 6.10.2 
      fast-json-stable-stringify: 2.0.0 
      magic-string: 0.25.3 
      rxjs: 6.4.0 
      source-map: 0.7.3 
      dev: false

Steps to analyze using the Jenkins plugin

By default, the Jenkins plugin will not evaluate the yarn.lock, pnpm-lock.yaml, package-lock.json or npm-shrinkwrap.json files. A custom Scan Target is needed.

Example Pipeline Script with Scan Patterns

nexusPolicyEvaluation iqApplication: 'SampApp' , iqScanPatterns: [[scanPattern: '**/npm-shrinkwrap.json' ], [scanPattern: '**/package-lock.json'], [scanPattern: '**/yarn.lock'], [scanPattern: '**/pnpm-lock.yaml']], iqStage: 'build'

Steps to analyze using the Bamboo plugin

By default, the Bamboo plugin will not evaluate the yarn.lock, pnpm-lock.yaml, package-lock.json or npm-shrinkwrap.json files. A custom Scan Target is needed.