npm Application Analysis

Evaluation: Advance Binary Fingerprinting (ABF)

When a scan includes javascript files, those are considered the primary identification source.  Any manifest files in the scan (e.g. package-lock.json) are used to refine the results identified by the javascript files. Manifest entries that do not correspond to any of the scanned javascript files are ignored.  If you want all dependencies declared in manifest files to be in the report you should only scan those files and avoid including any javascript files in the report.

Supported Configuration

  • Javascript scanning supports identifying packages from the npm ecosystem. 
  • Results include Security, License, and Identity data.
  • Use the following scan targets where applicable:
    • *.js
    • *.ts
    • *.tgz
    • package.json (requires package-lock.json)
    • package-lock.json
    • npm-shrinkwrap.json
    • yarn.lock
    • pnpm-lock.yaml

Scanning Javascript

  • For npm, scanning the package-lock.json is required. No results are returned if you only scan the package.json
  • Scan after the npm install command has run and the package-lock.json file is created 
  • Consider deleting the node_modules before scanning to remove any cached projects that will not be included in the final application
  • Both package-lock.json and package.json are required to build the dependency hierarchy tree
  • Include the application source code to search for embedded javascript files
    • Any declared packages are matched to discovered JavaScript target files 
    • The scan will match any packages present in the scan context path

A-Name files vs npm components

Only part of a complete npm package is deployed in an application. A scan matches the discovered javascript files with the packages declared in the project manifest. When they are successfully matched, the component will be listed in the scan report as an npm component. Javascript files not matched to a declared component may be identified as an a-name component. A-Name, or Authoritative Name, is the process in which individual javascript files are associated with the package from npm where it was first identified. As these files often do not include version information, this process can often be a 'best guess' matching that may not be the component/version that was used in your application. If the file is not identified or the authoritative source cannot be accurately determined, it will be considered an unknown javascript file (see below).

When to scan

When including open-source javascript components in an application, the dependencies are often obfuscated during the packaging process and end up modified in the final deployed artifacts. When this happens, the IQ scanners may be unable to identify the open-source components using binary fingerprinting. The following options should be considered to get an accurate analysis of the open-source components being used:

    • Scan the project with the CI or CLI scanners after the npm 'install' command has run but before any obfuscation process happens. 
    • For Node projects, run npm install --production to exclude dev dependencies in the package.json.
    • As mentioned above, consider deleting the node_modules before scanning to remove any cached projects that will not be included in the final application
    • The AuditJS tool can be used to automate a Lifecycle scan that respects production and development scopes.
    • The copy-modules-webpack-plugin can be used to copy the dependencies into a target directory first to run the analysis.
    • The project could be scanned directly from source control if other methods are not possible.

Direct and Transitive Dependencies

Javascript analysis includes support for reporting the hierarchy of dependencies in the scan report. The manifest files must be included in the scan for Lifecycle to determine the project's direct dependencies. The node_modules directory is used to determine the hierarchy of the transitive dependencies.

node_modules

When the node_modules directory is included in the scan context path everything found will be included in the scan results. This will cause confusion when older dependencies and developer (dev) dependencies have not been cleaned up from this cache. Deleting this directory and running 'npm install --production' will include only the dependencies that are to be in production.

Unknown javascript files

Proprietary javascript files are not known by Sonatype Data Services. Including them in the scan report would generate a large amount of noise without a meaningful way of identifying them using the proprietary component configuration. Therefore by default, they are not displayed in the scan report view. You can view them manually by adding the flag '?unknownjs' to the end of the report URL to show all unknown javascript files.

Evaluation: Manifest and lock files

A manifest analysis is run by specifically scanning only the project-lock and manifest files for the following javascript package managers: npm, yarn, and pnpm. The scanner will default to an ABF scan if any .js files are included in the scan context path.

  • Lock files: manifest files alone do not include the transitive dependencies and sometimes the specific direct versions that will be used in the final application. This is why we highly recommend including the lock files in the analysis for the best results. It is further important to note that when scanning a lock file via the Evaluate a File option in the Lifecycle UI, a dependency tree will not be produced unless the lock file is put in an archive (e.g. zip file) alongside its corresponding package.json file and that archive is scanned instead.

  • Direct and Transitive Dependencies: In order to include dependency type information (i.e. Direct vs Transitive), the package.json will need to include an auto-generated lock file along with the manifest files.

NEW IN RELEASE 137 If only a lock file is scanned, then a package.json file in the same directory can be used to help determine direct dependencies and development dependencies to be excluded.

This is true if the package.json file:

    • Exists
    • Is not excluded
    • Has a "dependencies" section

If all of these conditions are met, then only the packages specified in the "dependencies" section will be considered for analysis as production dependencies. If the "dependencies" section is empty, then no dependencies will be included.

  • devDependencies: By default, components under the 'devDependencies' and 'optional' scopes or have the metadata 'dev: true' are excluded from the scan. Here are two ways to include them:
    1. binary scan should be done including these components in the node_modules directory. The AuditJS tool may be optionally configured to include devDependencies.
    2. NEW IN RELEASE 153 to enable/disable scanning optional and devDependencies for manifest and lock files you can configure the scan-npm-dev-and-opt-dependencies feature using the Feature Configuration REST API - v2.

npm

NPM packages can be identified by scanning the following named lock files:

FilenameGenerated By

package-lock.json

npm-shrinkwrap.json

npm package manager v6

npm package manager v7 NEW IN RELEASE 123

if both files are present in the scan, the preference will be given to npm-shrinkwrap.json file.

Name and version fields from dependencies (or packages for npm v7) objects will be evaluated. For example:

  • name: ansi-regex
    • version: 3.0.0
  • name: wordwrap
    • version: 0.0.3

{ "requires": true, "lockfileVersion": 1, "dependencies": { "ansi-regex": { "version": "3.0.0", "resolved": "https://registry.npmjs.org/ansi-regex/-/ansi-regex-3.0.0.tgz", "integrity": "sha1-7QMXwyIGT3lGbAKWa922Bas32Zg=" }, "wordwrap": { "version": "0.0.3", "resolved": "https://registry.npmjs.org/wordwrap/-/wordwrap-0.0.3.tgz", "integrity": "sha1-o9XabNXAvAAI03I0u68b7WMFkQc=" } }
}

yarn

FilenameGenerated By
yarn.lock

Yarn package manager v1. 

Yarn package manager v2. NEW IN RELEASE 123

Name and version fields will be evaluated. For example:

  • name: @dangl/angular-material-shared
  • version: 2.0.0

@dangl/angular-material-shared@2.0.0: version "2.0.0" @progress/kendo-theme-material@0.3.2: version "0.3.2" @angular@0.0.1: version "0.0.1"

pnpm

FilenameGenerated By
pnpm-lock.yamlpnpm package manager

Name and version fields from packages and dependencies objects will be evaluated. For example:

  • name: @angular-devkit/schematics
  • version: 8.3.26

lockfileVersion: 5.3 specifiers: '@angular-devkit/schematics': 8.3.26 dependencies: '@angular-devkit/schematics': 8.3.26 packages: /@angular-devkit/schematics/8.3.26: resolution: {integrity: sha512-IoZbXVFGLvVi5d0ozfssWDXuzot0/pMSKbQPzWIG8K7nCo7nNMVYpsMHrEVYUikA9EQEL5LqMCGohH36/zVPcA==} engines: {node: '>= 10.9.0', npm: '>= 6.2.0'} dependencies: '@angular-devkit/core': 8.3.26 rxjs: 6.4.0 dev: false

Steps to analyze using the Sonatype IQ CLI

Run a scan

Invoke a Nexus IQ CLI scan of a directory or subdirectories containing yarn.lock, pnpm-lock.yaml, package-lock.json, or npm-shrinkwrap.json. Instructions on how to do this can be found here: Sonatype IQ CLI.

Example pnpm-lock.yaml file

lockfileVersion: 5.3 specifiers: '@angular-devkit/schematics': 8.3.26 '@angular/common': 8.2.14 '@angular/compiler': 8.2.14 '@angular/core': 8.2.14 '@angular/router': 8.2.14 '@ng-bootstrap/ng-bootstrap': 5.3.0 bootstrap: 4.5.0 jquery: 1.9.1 rxjs: ^6.4.0 zone.js: ~0.9.1 dependencies: '@angular-devkit/schematics': 8.3.26 '@angular/common': 8.2.14_@angular+core@8.2.14+rxjs@6.4.0 '@angular/compiler': 8.2.14 '@angular/core': 8.2.14_rxjs@6.4.0+zone.js@0.9.1 '@angular/router': 8.2.14_ed906ac8447aa5d4a7a8ac33fedcf709 '@ng-bootstrap/ng-bootstrap': 5.3.0_ed906ac8447aa5d4a7a8ac33fedcf709 bootstrap: 4.5.0_jquery@1.9.1 jquery: 1.9.1 rxjs: 6.4.0 zone.js: 0.9.1 packages: /@angular-devkit/core/8.3.26: resolution: {integrity: sha512-b1ng9091o33s55/cwQYh1kboiJtj8y8z8xQWATDI9kRmNIQkWYVwVa/MzgPRJ4bzbEGG3zIUHCsp52A6vuGr2A==} engines: {node: '>= 10.9.0', npm: '>= 6.2.0'} dependencies: ajv: 6.10.2 fast-json-stable-stringify: 2.0.0 magic-string: 0.25.3 rxjs: 6.4.0 source-map: 0.7.3 dev: false ...

Steps to analyze using the Jenkins plugin

By default, the Jenkins plugin will not evaluate the yarn.lock, pnpm-lock.yaml, package-lock.json or npm-shrinkwrap.json files. A custom Scan Target is needed.

nexusPolicyEvaluation iqApplication: 'SampApp' , iqScanPatterns: [[scanPattern: '**/npm-shrinkwrap.json' ], [scanPattern: '**/package-lock.json'], [scanPattern: '**/yarn.lock'], [scanPattern: '**/pnpm-lock.yaml']], iqStage: 'build'

To find more information on configuring Jenkins, please go to the Sonatype Platform Plugin for Jenkins.

Steps to analyze using the Bamboo plugin

By default, the Bamboo plugin will not evaluate the yarn.lock, pnpm-lock.yaml, package-lock.json or npm-shrinkwrap.json files. A custom Scan Target is needed.

To find more information on configuring the Bamboo plugin, please go to Lifecycle for Bamboo.