npm Application Analysis

Evaluation: Advance Binary Fingerprinting (ABF)

Javascript scanning supports files and packages from NPM including (.js/.ts/.tgz) files.  For the best results, the scan should happen after the npm install command has run and the required open source dependencies are present in the scan context path.  Any lock and manifest files (listed below) included in the analysis will be used to determine the dependency hierarchy and assist in matching the declared packages with the discovered javascript files. Full support of Security, License, Identity data is available for components found in the npm repository. 

• npm components verse a-name files: When npm packages are extracted to the node_modules directory, typically only part of the component is used. An ABF scan matches these 'files' to the 'components' that are declared in the manifest and lock files.  Without the manifest context, javascript files are identified by IQ Intelligence as the best guess to their authoritative package in npm. This technique of matching javascript is called Authoritative Name matching and referenced as a-name within the product.  Review the Scanning Javascript Guide for details.
  
  
• When to scan javascript components: When including open-source javascript components in an application, the dependencies are often obfuscated during the packaging process and end up modified in the final deployed artifacts.  When this happens, the IQ scanners may be unable to identify the open-source components using binary fingerprinting.  To get an accurate analysis of the open-source components being used, the following options should be considered.
  • Scan the project with the CI or CLI scanners after the npm 'install' command has run but before any obfuscation process happens.  Consider deleting the node_modules first to remove any cached projects that will not be included in the final application.
  • For Node projects, run npm install --production to exclude dev dependencies in the package.json.
  • AuditJS can be used to automate a Lifecycle scan that respects production and development scopes.
  • The copy-modules-webpack-plugin can be used to copy the dependencies into a target directory first to run the analysis.
  • The project could be scanned directly from source control if other methods are not possible.
    
    
• Direct and Transitive Dependencies: Javascript analysis includes support for reporting the hierarchy of dependencies in the scan report.  The manifest files must be included in the scan for Lifecycle to determine the project's direct dependencies.  The node_modules directory is used to determine the hierarchy of the transitive dependencies.
  
  
• node_modules: when the node_modules directory is included in the scan context path everything found will be included in the scan results.  This will cause confusion when older dependencies and developer (dev) dependencies have not been cleaned up from this cache.  Deleting this directory and running 'npm install --production' will include only the dependencies that are to be in production.
  
  
• Unknown javascript files: Proprietary javascript files are not known by Sonatype Data Services.  Including them in the scan report would generate a large amount of noise without a meaningful way of identifying them using the proprietary component configuration. Therefore by default, they are not displayed in the scan report view. You can view them manually by adding the flag '?unknownjs' to the end of the report URL to show all unknown javascript files.  The complete list is also available in the raw component data API.

Evaluation: Manifest and lock files

A manifest analysis is run by specifically scanning only the project-lock and manifest files for the following javascript package managers: npm, yarn, and pnpm.   The scanner will default to an ABF scan if any .js files are included in the scan context path.

• Lock files: manifest files alone do not include the transitive dependencies and sometimes the specific direct versions that will be used in the final application.  This is why we highly recommend including the lock files in the analysis for the best results.
  
  
• devDependencies: Components under the 'devDependencies' and 'optional' scopes or have the metadata 'dev: true' are excluded from the scan results.  To include them, a binary scan should be done including these components in the node_modules directory.  The AuditJS tool may be optionally configured to include development dependencies.
  
  
• Direct and Transitive Dependencies:  NEW IN RELEASE 123 In order to include dependency type information (i.e. Direct vs Transitive), the package.json will need to include an auto-generated lock file along with the manifest files.

NEW IN RELEASE 137 Note that if only a lock file is scanned, then a package.json file in the same directory can be used to help determine direct dependencies and development dependencies to be excluded.

This is true if the package.json file:

• Exists
• Is not excluded
• Has a "dependencies" section

Also note that if all of these conditions are met, then only the packages specified in the "dependencies" section will be considered for analysis as production dependencies.

Consequently if the "dependencies" section is empty, then no dependencies will be included.

npm

NPM packages can be identified by scanning the following named lock files:

Name and version fields from dependencies (or packages for npm v7) objects will be evaluated. For example:

• name: ansi-regex
  • version: 3.0.0
• name: wordwrap
  • version: 0.0.3
    
    

{
  "requires": true,
  "lockfileVersion": 1,
  "dependencies": {
    "ansi-regex": {
      "version": "3.0.0",
      "resolved": "https://registry.npmjs.org/ansi-regex/-/ansi-regex-3.0.0.tgz",
      "integrity": "sha1-7QMXwyIGT3lGbAKWa922Bas32Zg="
    },
    "wordwrap": {
      "version": "0.0.3",
      "resolved": "https://registry.npmjs.org/wordwrap/-/wordwrap-0.0.3.tgz",
      "integrity": "sha1-o9XabNXAvAAI03I0u68b7WMFkQc="
    }
  }
}

yarn

Name and version fields will be evaluated. For example:

• name: @dangl/angular-material-shared
• version: 2.0.0
  
  

@dangl/angular-material-shared@2.0.0:
  version "2.0.0"

@progress/kendo-theme-material@0.3.2:
  version "0.3.2"

@angular@0.0.1:
  version "0.0.1"

pnpm

Name and version fields from packages and dependencies objects will be evaluated. For example:

• name: @angular-devkit/schematics
• version: 8.3.26
  
  

lockfileVersion: 5.3

specifiers:
  '@angular-devkit/schematics': 8.3.26

dependencies:
  '@angular-devkit/schematics': 8.3.26

packages:

  /@angular-devkit/schematics/8.3.26:
    resolution: {integrity: sha512-IoZbXVFGLvVi5d0ozfssWDXuzot0/pMSKbQPzWIG8K7nCo7nNMVYpsMHrEVYUikA9EQEL5LqMCGohH36/zVPcA==}
    engines: {node: '>= 10.9.0', npm: '>= 6.2.0'}
    dependencies:
      '@angular-devkit/core': 8.3.26
      rxjs: 6.4.0
    dev: false

Steps to analyze using the Nexus IQ CLI

Run a scan

Invoke a Nexus IQ CLI scan of a directory or subdirectories containing yarn.lock, pnpm-lock.yaml, package-lock.json, or npm-shrinkwrap.json.  Instructions on how to do this can be found here:  Nexus IQ CLI.

Example pnpm-lock.yaml file

lockfileVersion: 5.3

specifiers:
  '@angular-devkit/schematics': 8.3.26
  '@angular/common': 8.2.14
  '@angular/compiler': 8.2.14
  '@angular/core': 8.2.14
  '@angular/router': 8.2.14
  '@ng-bootstrap/ng-bootstrap': 5.3.0
  bootstrap: 4.5.0
  jquery: 1.9.1
  rxjs: ^6.4.0
  zone.js: ~0.9.1

dependencies:
  '@angular-devkit/schematics': 8.3.26
  '@angular/common': 8.2.14_@angular+core@8.2.14+rxjs@6.4.0
  '@angular/compiler': 8.2.14
  '@angular/core': 8.2.14_rxjs@6.4.0+zone.js@0.9.1
  '@angular/router': 8.2.14_ed906ac8447aa5d4a7a8ac33fedcf709
  '@ng-bootstrap/ng-bootstrap': 5.3.0_ed906ac8447aa5d4a7a8ac33fedcf709
  bootstrap: 4.5.0_jquery@1.9.1
  jquery: 1.9.1
  rxjs: 6.4.0
  zone.js: 0.9.1

packages:

  /@angular-devkit/core/8.3.26:
    resolution: {integrity: sha512-b1ng9091o33s55/cwQYh1kboiJtj8y8z8xQWATDI9kRmNIQkWYVwVa/MzgPRJ4bzbEGG3zIUHCsp52A6vuGr2A==}
    engines: {node: '>= 10.9.0', npm: '>= 6.2.0'}
    dependencies:
      ajv: 6.10.2
      fast-json-stable-stringify: 2.0.0
      magic-string: 0.25.3
      rxjs: 6.4.0
      source-map: 0.7.3
    dev: false

...

Steps to analyze using the Jenkins plugin

By default, the Jenkins plugin will not evaluate the yarn.lock, pnpm-lock.yaml, package-lock.json or npm-shrinkwrap.json files. A custom Scan Target is needed.

nexusPolicyEvaluation iqApplication:  'SampApp' , iqScanPatterns: [[scanPattern:  '**/npm-shrinkwrap.json' ], [scanPattern:  '**/package-lock.json'], [scanPattern:  '**/yarn.lock'], [scanPattern:  '**/pnpm-lock.yaml']], iqStage:  'build'

To find more information on how to configure Jenkins please go to the Nexus Platform Plugin for Jenkins.

Steps to analyze using the Bamboo plugin

By default, the Bamboo plugin will not evaluate the yarn.lock, pnpm-lock.yaml, package-lock.json or npm-shrinkwrap.json files. A custom Scan Target is needed.

To find more information on how to configure the Bamboo plugin please go to Nexus IQ for Bamboo.