Skip to main content

npm Application Analysis

Evaluation: Advance Binary Fingerprinting (ABF)

When a scan includes javascript files, those are considered the primary identification source. Any manifest files in the scan (e.g. package-lock.json) are used to refine the results identified by the javascript files. Manifest entries that do not correspond to any of the scanned javascript files are ignored. If you want all dependencies declared in manifest files to be in the report you should only scan those files and avoid including any javascript files in the report.

Scanning Javascript

  • Delete the node_modules before scanning to remove any cached projects that will not be included in the final application

  • For npm, scanning the package-lock.json is required. No results are returned if you only scan the package.json

  • Both package-lock.json and package.json are required to build the dependency hierarchy tree

  • For Node projects, run npm install --production to exclude dev dependencies in the package.json

  • Scan after the npm install command has run and the package-lock.json file is created

  • Include the application source code to search for embedded javascript files

    • Any declared packages are matched to discovered JavaScript target files

    • The scan will match any packages present in the scan context path

Supported Configuration

  • Javascript scanning supports identifying packages from the npm ecosystem.

  • Results include Security, License, and Identity data.

  • Use the following scan targets where applicable:

    • *.js

    • *.ts

    • *.tgz

    • package.json (requires package-lock.json)

    • package-lock.json

    • npm-shrinkwrap.json

    • yarn.lock

    • pnpm-lock.yaml

A-Name files vs npm components

Only part of a complete npm package is deployed in an application. A scan matches the discovered javascript files with the packages declared in the project manifest. When they are successfully matched, the component will be listed in the scan report as an npm component. Javascript files not matched to a declared component may be identified as an a-name component. A-Name, or Authoritative Name, is the process in which individual javascript files are associated with the package from npm where it was first identified. As these files often do not include version information, this process can often be a 'best guess' matching that may not be the component/version that was used in your application. If the file is not identified or the authoritative source cannot be accurately determined, it will be considered an unknown javascript file (see below).

When to scan

When including open-source javascript components in an application, the dependencies are often obfuscated during the packaging process and end up modified in the final deployed artifacts. When this happens, the IQ scanners may be unable to identify the open-source components using binary fingerprinting. The following options should be considered to get an accurate analysis of the open-source components being used:

  • Scan the project with the CI or CLI scanners after the npm 'install' command has run but before any obfuscation process happens.

  • As mentioned above, delete the node_modules before scanning to remove any cached projects that will not be included in the final application

  • The copy-modules-webpack-plugin can be used to copy the dependencies into a target directory first to run the analysis.

  • The project could be scanned directly from source control if other methods are not possible.

Direct and Transitive Dependencies

Javascript analysis includes support for reporting the hierarchy of dependencies in the scan report. The manifest files must be included in the scan for Lifecycle to determine the project's direct dependencies.

node_modules

When the node_modules directory is included in the scan context path everything found will be included in the scan results. This will cause confusion when older dependencies and developer (dev) dependencies have not been cleaned up from this cache. Deleting this directory and running 'npm install --production' will include only the dependencies that are to be in production.

Unknown javascript files

Proprietary javascript files are not known by Sonatype Data Services. Including them in the scan report would generate a large amount of noise without a meaningful way of identifying them using the proprietary component matcher. Therefore by default, they are not displayed in the scan report view. You can view them manually by adding the flag '?unknownjs' to the end of the report URL to show all unknown javascript files.

Evaluation: Manifest and lock files

A manifest analysis is run by specifically scanning only the project-lock and manifest files for the following javascript package managers: npm, yarn, and pnpm. The scanner will default to an ABF scan if any .js files are included in the scan context path.

  • Lock files: manifest files alone do not include the transitive dependencies and sometimes the specific direct versions that will be used in the final application. This is why we highly recommend including the lock files in the analysis for the best results. It is further important to note that when scanning a lock file via the Evaluate a File option in the Lifecycle UI, a dependency tree will not be produced unless the lock file is put in an archive (e.g. zip file) alongside its corresponding package.json file and that archive is scanned instead.

  • Direct and Transitive Dependencies: In order to include the dependency information i.e. Direct vs Transitive (refer Reviewing a Report), the package.json will need to include an auto-generated lock file along with the manifest files.

If only a lock file is scanned, then a package.json file in the same directory can be used to help determine direct dependencies and development dependencies to be excluded.

This is true if the package.json file:

  • Exists

  • Is not excluded

  • Has a "dependencies" section

If all of these conditions are met, then only the packages specified in the "dependencies" section will be considered for analysis as production dependencies. If the "dependencies" section is empty, then no dependencies will be included.

devDependencies: By default, components under the 'devDependencies' and 'optional' scopes or have the metadata 'dev: true' are excluded from the scan. Here are two ways to include them:

  1. Binary scan should be done including these components in the node_modules directory.

  2. RELEASE 153 to enable/disable scanning optional and devDependencies for manifest and lock files you can configure the scan-npm-dev-and-opt-dependencies feature using the Feature Configuration REST API.

npm

NPM packages can be identified by scanning the following named lock files:

Filename

Generated By

package-lock.json

npm-shrinkwrap.json

npm package manager v6

npm package manager v7

preference is given to npm-shrinkwrap.json file when both files are present in the scan

Name and version fields from dependencies (or packages for npm v7) objects will be evaluated. For example:

  • name: ansi-regex

    • version: 3.0.0

  • name: wordwrap

    • version: 0.0.3

{ "requires": true, "lockfileVersion": 1, "dependencies": { "ansi-regex": { "version": "3.0.0", "resolved": "https://registry.npmjs.org/ansi-regex/-/ansi-regex-3.0.0.tgz", "integrity": "sha1-7QMXwyIGT3lGbAKWa922Bas32Zg=" }, "wordwrap": { "version": "0.0.3", "resolved": "https://registry.npmjs.org/wordwrap/-/wordwrap-0.0.3.tgz", "integrity": "sha1-o9XabNXAvAAI03I0u68b7WMFkQc=" } }
}

yarn

Filename

Generated By

yarn.lock

Yarn package manager v1

Yarn package manager v2

Name and version fields will be evaluated. For example:

  • name: @dangl/angular-material-shared

  • version: 2.0.0

@dangl/angular-material-shared@2.0.0: 
  version "2.0.0" 
@progress/kendo-theme-material@0.3.2: 
  version "0.3.2" 
@angular@0.0.1: 
  version "0.0.1"

pnpm

Filename

Generated By

pnpm-lock.yaml

pnpm package manager

Name and version fields from packages and dependencies objects will be evaluated. For example:

  • name: @angular-devkit/schematics

  • version: 8.3.26

lockfileVersion: 5.3 

specifiers: 
  '@angular-devkit/schematics': 8.3.26 

dependencies: 
  '@angular-devkit/schematics': 8.3.26 

packages: 
  /@angular-devkit/schematics/8.3.26: 
    resolution: {integrity: sha512-IoZbXVFGLvVi5d0ozfssWDXuzot0/pMSKbQPzWIG8K7nCo7nNMVYpsMHrEVYUikA9EQEL5LqMCGohH36/zVPcA==} 
    engines: {node: '>= 10.9.0', npm: '>= 6.2.0'} 
    dependencies: 
      '@angular-devkit/core': 8.3.26 
      rxjs: 6.4.0 
      dev: false

Steps to analyze using the Sonatype IQ CLI

Run a scan

Invoke a Nexus IQ CLI scan of a directory or subdirectories containing yarn.lock, pnpm-lock.yaml, package-lock.json, or npm-shrinkwrap.json. Instructions on how to do this can be found here: Sonatype IQ CLI.

Example pnpm-lock.yaml file

lockfileVersion: 5.3 

specifiers: 
  '@angular-devkit/schematics': 8.3.26 
  '@angular/common': 8.2.14 
  '@angular/compiler': 8.2.14 
  '@angular/core': 8.2.14 
  '@angular/router': 8.2.14 
  '@ng-bootstrap/ng-bootstrap': 5.3.0 
    bootstrap: 4.5.0 
    jquery: 1.9.1 
    rxjs: ^6.4.0 
    zone.js: ~0.9.1 

dependencies: 
  '@angular-devkit/schematics': 8.3.26 
  '@angular/common': 8.2.14_@angular+core@8.2.14+rxjs@6.4.0 
  '@angular/compiler': 8.2.14 
  '@angular/core': 8.2.14_rxjs@6.4.0+zone.js@0.9.1 
  '@angular/router': 8.2.14_ed906ac8447aa5d4a7a8ac33fedcf709 
  '@ng-bootstrap/ng-bootstrap': 5.3.0_ed906ac8447aa5d4a7a8ac33fedcf709 
    bootstrap: 4.5.0_jquery@1.9.1 
    jquery: 1.9.1 
    rxjs: 6.4.0 
    zone.js: 0.9.1 

packages: 
  /@angular-devkit/core/8.3.26: 
    resolution: {integrity: sha512-b1ng9091o33s55/cwQYh1kboiJtj8y8z8xQWATDI9kRmNIQkWYVwVa/MzgPRJ4bzbEGG3zIUHCsp52A6vuGr2A==} 
    engines: {node: '>= 10.9.0', npm: '>= 6.2.0'} 
    dependencies: 
      ajv: 6.10.2 
      fast-json-stable-stringify: 2.0.0 
      magic-string: 0.25.3 
      rxjs: 6.4.0 
      source-map: 0.7.3 
      dev: false

Steps to analyze using the Jenkins plugin

By default, the Jenkins plugin will not evaluate the yarn.lock, pnpm-lock.yaml, package-lock.json or npm-shrinkwrap.json files. A custom Scan Target is needed.

Example Pipeline Script with Scan Patterns

nexusPolicyEvaluation iqApplication: 'SampApp' , iqScanPatterns: [[scanPattern: '**/npm-shrinkwrap.json' ], [scanPattern: '**/package-lock.json'], [scanPattern: '**/yarn.lock'], [scanPattern: '**/pnpm-lock.yaml']], iqStage: 'build'

To find more information on configuring Jenkins, please go to Sonatype Platform Plugin for Jenkins.

Steps to analyze using the Bamboo plugin

By default, the Bamboo plugin will not evaluate the yarn.lock, pnpm-lock.yaml, package-lock.json or npm-shrinkwrap.json files. A custom Scan Target is needed.

87198517.png

To find more information on configuring the Bamboo plugin, please go to Sonatype for Bamboo Data Center.