Skip to main content

Preparing to Run npm Application Analysis

JavaScript projects rely on build tools and dependency managers that can significantly alter source files before deployment. These transformations improve performance and security, but they also make it harder for analysis tools to identify components accurately. Following the right scanning approach ensures that dependency data remains complete, current, and aligned with your final application.

This topic outlines how to prepare your project for scanning, when to run scans during the build process, and how to include the information needed for precise results.

When to Scan

JavaScript dependencies are often transformed during build steps such as bundling, minification, or obfuscation. These modifications can remove identifying information from files.

To achieve the most accurate results, follow these best practices:

  • Delete the existing node_modules before scanning to remove any cached projects that will not be included in the final application

  • Run the scan after installing dependencies (npm install, yarn install, or pnpm install)

  • Run the scan before bundling, minification, or obfuscation

  • The copy-modules-webpack-plugin can be used to copy the dependencies into a target directory first to run the analysis

  • You can scan the project directly from source control if other methods are not possible

  • For the most precise results, you should include a package.json, package-lock.json, and the new node_modules folder that is created when running the install command described in the sections below

If scans are performed after obfuscation, IQ may not be able to identify components using ABF.

Scanning JavaScript Files

The following guidance explains how to set up your environment for reliable results when scanning JavaScript files.

Remove Unused Dependencies

Delete the node_modules directory before running a fresh installation to avoid scanning unused or stale packages.

node_modules Considerations

If node_modules is included in a scan, all files inside of it will be evaluated. This can mean that stale, unused, or dev dependencies could be included, introducing noise into your analysis. This is why you should delete the existing node_modules file; a new one will be created when you run the install command explained below.

For Node projects, use the --production flag to exclude dev dependencies in the package.json:

npm install --production

For Yarn:

yarn install --prod

Lock Files Required

  • npm requires package-lock.json for accurate results. Scan after the npm or yarn install command has run and the package-lock.json file is created.

  • Scanning only package.json does not produce results under ABF.

  • Both package.json and package-lock.json are needed to generate a dependency hierarchy.

Include Source Code When Needed

  • To match embedded or bundled JavaScript, include the application’s own JavaScript files in the scan context path.

  • Any declared packages are matched to discovered JavaScript target files.

  • The scan will match any packages present in the scan context path.

Evaluating Manifests Without JavaScript Files

A manifest analysis is run by specifically scanning only the project-lock and manifest files for the following javascript package managers: npm, yarn, and pnpm.

If any *.js file is detected, IQ automatically switches to ABF.

Lock Files and Dependency Information

Manifest files alone do not include transitive dependencies. Sometimes, they do not include the specific direct versions that will be used in the final application. Therefore, we highly recommend including the lock files in your analysis for the best results.

To include dependency information (i.e., Direct vs. Transitive), the package.json will need to include an auto-generated lock file along with the manifest files.

If only a lock file is scanned, then a package.json file in the same directory can be used to help determine direct dependencies and development dependencies to be excluded. This package.json must contain a dependencies section.

Only the packages specified in the dependencies section will be considered production dependencies. If the dependencies section is empty, then no dependencies will be included.

Evaluate a File Feature Note

To produce a dependency tree using the Evaluate a File option from the Lifecycle user interface, put the lock file in an archive along with its corresponding package.json file (e.g., ZIP them). Then, scan this archive instead.

Handling devDependencies and optionalDependencies

By default, dev and optional dependencies are excluded from manifest-based analysis.

To include them, do one of the following:

  • Scan node_modules directly, or

  • Enable the scan-npm-dev-and-opt-dependencies feature using the Feature Configuration REST API.

    • When this feature is enabled, analysis only looks at devDependencies and optionalDependencies defined in the package.json file for the direct dependencies.

    • Transitive dependencies brought in from the package-lock.json will not be included in the analysis.

Evaluating npm Packages by Lock File

npm packages can be identified by scanning the following named lock files:

Filename

Generated by

package-lock.json

npm-shrinkwrap.json

npm package manager v6

npm package manager v7

Preference is given to a npm-shrinkwrap.json file when both files are present in a scan

name and version fields from dependencies (or packages for npm v7) will be evaluated.

Example

  • name: ansi-regex

    • version: 3.0.0

  • name: wordwrap

    • version: 0.0.3

{ "requires": true, "lockfileVersion": 1, "dependencies": { "ansi-regex": { "version": "3.0.0", "resolved": "https://registry.npmjs.org/ansi-regex/-/ansi-regex-3.0.0.tgz", "integrity": "sha1-7QMXwyIGT3lGbAKWa922Bas32Zg=" }, "wordwrap": { "version": "0.0.3", "resolved": "https://registry.npmjs.org/wordwrap/-/wordwrap-0.0.3.tgz", "integrity": "sha1-o9XabNXAvAAI03I0u68b7WMFkQc=" } }
}

Evaluating Yarn Packages by Lock File

Yarn packages can be identified by scanning the following named lock files:

Filename

Generated by

yarn.lock

yarn.lock

yarn.lock

yarn.lock

Yarn package manager v1

Yarn package manager v2

Yarn package manager v3

Yarn package manager v4

name and version fields will be evaluated.

Example

  • name:@dangl/angular-material-shared

  • version: 2.0.0c

@dangl/[email protected]: 
  version "2.0.0" 
@progress/[email protected]: 
  version "0.3.2" 
@[email protected]: 
  version "0.0.1"

Yarn Workspace Monorepos

For a Yarn workspace monorepo, include the following files in the scan target:

  • The root yarn.lock.

  • The root package.json.

  • Each workspace member’s package.json.

IQ can analyze a Yarn workspace monorepo in manifest mode without a node_modules directory. The analysis produces one consolidated component inventory and policy report for the monorepo. It does not create a separate Lifecycle application or report for each workspace member.

For Yarn Berry projects, IQ identifies supported direct and transitive dependencies from yarn.lock. Dependencies declared using an npm: alias are reported using the target package identity instead of the alias name.

The following table describes how manifest analysis handles common Yarn dependency references:

Dependency Reference

Manifest Analysis Behavior

npm:

IQ evaluates the external dependency using the target package identity.

workspace:, link:, file:, portal:, exec:, or a local path beginning with ./, ../, or /

IQ treats the reference as local rather than as an external component.

patch:, git:, github:, https:, or an unrecognized protocol

IQ may not report the dependency if it cannot construct a valid package URL from the available coordinates.

For Yarn Plug’n’Play projects, manifest analysis uses the applicable package.json files and yarn.lock. IQ does not parse .pnp.cjs.

Caution

After upgrading to a version that includes this improvement, Yarn Berry workspace scans may identify additional components and policy violations. Review newly reported violations and existing waivers.

The behavior described in Handling devDependencies and optionalDependencies also applies to dependencies declared in workspace member manifests.

Evaluating pnpm Packages by Lock File

pnpm packages can be identified by scanning the following named lock files:

Filename

Generated by

pnpm-lock.yaml

pnpm package manager

name and version fields from packages and dependencies will be evaluated.

Example

  • name: @angular-devkit/schematics

  • version: 8.3.26

lockfileVersion: 5.3 

specifiers: 
  '@angular-devkit/schematics': 8.3.26 

dependencies: 
  '@angular-devkit/schematics': 8.3.26 

packages: 
  /@angular-devkit/schematics/8.3.26: 
    resolution: {integrity: sha512-IoZbXVFGLvVi5d0ozfssWDXuzot0/pMSKbQPzWIG8K7nCo7nNMVYpsMHrEVYUikA9EQEL5LqMCGohH36/zVPcA==} 
    engines: {node: '>= 10.9.0', npm: '>= 6.2.0'} 
    dependencies: 
      '@angular-devkit/core': 8.3.26 
      rxjs: 6.4.0 
      dev: false