Skip to content

[rushstack] Use pnpm catalogs for external dependency versions - #6146

Draft
Bharat Middha (bmiddha) wants to merge 6 commits into
bmiddha-catalog-aware-rush-addfrom
bmiddha-migrate-rushstack-to-pnpm-catalogs
Draft

Bharat Middha (bmiddha) wants to merge 6 commits into
bmiddha-catalog-aware-rush-addfrom
bmiddha-migrate-rushstack-to-pnpm-catalogs

Conversation

@bmiddha

Copy link
Copy Markdown
Member

Summary

This is layer 6 (the top) of the pnpm 12 / catalogs stack, based on #6141. It moves this repo's external dependency versions into a single pnpm catalog. common/config/rush/pnpm-config.json gains globalCatalogs.default with 165 entries sorted by name, and 657 dependency specs in 171 Rush projects now use "catalog:". Neither lockfile changes any resolved version; see the churn numbers below.

Merge prerequisites (do not merge before these)

  1. A published Rush release containing fix(rush): synchronize native pnpm injections from the installed layout #6133, the Rush fix from [rush] Move the rushstack repo to pnpm 12.13.0 #6136, [rush] Use the common pnpm-config.json fallback for every pnpm catalog consumer #6138, [rush] Resolve PNPM catalog references in implicitly preferred versions #6139, [rush] Make version consistency checks aware of PNPM catalogs #6140, [rush] Make "rush add" and "rush upgrade-interactive" aware of PNPM catalogs #6141 and [rush] Copy the temp lockfile when pnpm changes it during rush update #6119. Then bump rushVersion in rush.json. This PR deliberately does not bump rushVersion; the currently pinned 5.181.0 cannot install this repo with pnpm 12 or with catalog: specs, so CI on this PR is expected to fail until then.
  2. The internal package feed must serve pnpm 12.13.0. It currently serves up to 12.10.1.

Details

Design

  • One shared catalog. Neither subspace (default, build-tests-subspace) has its own pnpm-config.json, so both use common/config/rush/pnpm-config.json and share globalCatalogs.default (effective-config behavior from [rush] Use the common pnpm-config.json fallback for every pnpm catalog consumer #6138).
  • What gets cataloged. For each external package (not a Rush project) in the dependencies, devDependencies or optionalDependencies of a Rush project, the dominant version (the one that is consistent under ensureConsistentVersions) goes in the catalog, and those specs become catalog:.
  • Config not changed. catalogPrune / cleanupUnusedCatalogs are not enabled: each subspace workspace only sees its own projects, so pruning a shared catalog would be unsafe. catalogMode is not set. preferredVersions and allowedAlternativeVersions in common-versions.json are unchanged and consistent with the catalog (typescript ~5.8.2, eslint ~9.37.0, minimatch 10.2.3).
  • Change files. 79 none change files with empty comments. pnpm pack replaces catalog: with the catalog version, so published manifests are unchanged (verified below).

What stays literal, and why

default subspace: 650 specs cataloged (197 dependencies, 453 devDependencies) across 166 projects. 207 external specs stay literal:

Count Reason
98 prod/optional deps of the 23 projects in the injected cross-subspace closure (see below)
76 versions listed in allowedAlternativeVersions (the dominant version is in the catalog)
17 npm: aliases
7 dist-tags (@types/node / @types/jest ts2.9/ts3.9/ts4.9)
5 @types/vscode: vsce reads devDependencies['@types/vscode'] with semver to check it against engines.vscode and rejects catalog:
3 @pnpm/logger: both versions in use are alternatives
1 @pnpm/lockfile.types: its only version is an alternative

build-tests-subspace: 7 cataloged (devDependencies in 5 projects). 7 stay literal because they differ from the shared catalog: 5× eslint ~9.25.1 (catalog has ~9.37.0), webpack ~5.105.0 (catalog has ~5.105.2), and typescript ~4.9.5 (an alternative). Named catalogs could absorb these later.

Never touched:

  • all peerDependencies
  • workspace: specs, and Rush projects consumed by version as decoupled local dependencies (@rushstack/heft, the eslint configs, decoupled-local-node-rig, and similar). These are still bumped by repo-toolbox's BumpDecoupledLocalDependencies, which only touches those literal specs.
  • common/autoinstallers/**, and any package.json that is not a Rush project (test fixtures and the like)

Deviation: injected cross-subspace closure stays literal for prod deps

build-tests-subspace projects inject default-subspace projects (workspace:* + dependenciesMeta.injected). The subspace pnpmfile shim rewrites those deps to file:../../<folder>. pnpm then installs the injected packages from their own manifests, and it cannot resolve catalog: there (ERR_PNPM_SPEC_NOT_SUPPORTED_BY_ANY_RESOLVER "@types/tapable@catalog:").

The transitive closure of injected projects is 35 projects (rush-lib, rush-sdk, heft, node-core-library, api-extractor, the eslint packages, local-node-rig, and others). Only 23 of them had cataloged prod deps.

  • In those projects, dependencies and optionalDependencies keep literal versions (98 specs). That leaves 42 packages used only there out of the catalog.
  • devDependencies of closure projects are still catalog:, because injected packages don't install devDependencies.
  • The globalCatalogs comment in pnpm-config.json documents this.

Follow-up option for a lower layer: have SubspaceGlobalPnpmfileShim.readPackage resolve catalog: / catalog:<name> for injected workspace packages, using the source subspace's effective catalogs. After that ships, these 98 specs can be cataloged as well.

Caveat: running catalog-aware rush add (#6141) in a closure project for a production dependency writes catalog: and breaks the build-tests-subspace install, until that shim fix lands.

Lockfile churn

Measured by comparing pnpm-lock.yaml packages, snapshots and importers before and after this change.

Lockfile packages snapshots importer version changes importer specifier changes
default 3031 → 3031 (0 added / 0 removed / 0 changed) 3157 → 3157 (0 / 0 / 0) 0 646
build-tests-subspace 867 → 867 (0 / 0 / 0) 886 → 886 (0 / 0 / 0) 0 7
  • Only specifiers changed. Each lockfile gains a catalogs.default block containing only the entries that subspace uses (build-tests-subspace uses 5: @types/node, @types/webpack-env, html-webpack-plugin, tslint, typescript). Importer specifiers change from literal ranges to catalog:.
  • Specifier count gap. The default lockfile shows 646 importer specifier changes for 650 package.json specs because the lockfile lists a package that appears in both dependencies and devDependencies only once.
  • repo-state.json. It gains pnpmCatalogsHash.
  • Tarball URLs. Rotating ms-feed tarball URLs were stripped from both lockfiles; none remain.

How it was tested

All commands were run with a locally built Rush containing layers 1–5 (node apps/rush/lib-commonjs/start-dev.js …), pnpm 12.13.0 and Node 22.23.2. The validation ran before rebasing onto the new tip of #6141. The rebase was patch-identical (git range-diff), and the lower layer only changed rush-lib's PnpmSyncUtilities. change --verify was re-run after the rebase.

Command Result
rush update (both subspaces), strip ms-feed tarball URLs, rush update again ✅ Passed. Lockfiles and repo-state.json stable, no ms-feed URLs
CI=true rush install ✅ Passed. default: frozen lockfile, "Lockfile is up to date"; build-tests-subspace: --no-frozen-lockfile --prefer-frozen-lockfile (injected deps). No changes to lockfiles or repo-state.json
rush check, rush check --subspace default, rush check --subspace build-tests-subspace ✅ "Found no mis-matching dependencies!" (all three)
rush build --production (full repo) ✅ Passed, exit 0. 200 operations (137 success, 46 from cache, 17 no-op), 0 warnings, 0 failures
rush change --verify (against origin/main and against the #6141 tip) ✅ Passed
rush-pnpm pack for node-core-library, heft, rush, heft-node-rig, api-documenter, heft-sass-plugin, heft-webpack5-plugin, lockfile-explorer ✅ Packed package.json has no catalog:; every catalog: ref resolves to its catalog entry; every external dependency spec is identical to the pre-migration package.json
VS Code extensions (package-vsix task runs vsce package --no-dependencies during the build) ✅ All three VSIXes packaged during the production build. @types/vscode is literal, so the engines.vscode check passes. catalog: in the VSIX's devDependencies is accepted, as workspace:* already was
Deploy / package-extractor ✅ Unaffected: it only reads dependency names. The package-extractor-test-* builds passed in the full build
rush add -p @types/cors --dev -s in apps/api-documenter (smoke test of #6141, reverted) ✅ Wrote "@types/cors": "catalog:" ("Assigning "catalog:" … because it is "~2.8.12" in the PNPM catalog")
repo-toolbox BumpDecoupledLocalDependencies (code review) ✅ Unaffected: it only rewrites specs for decoupled local Rush projects, which stay literal

Impacted documentation

None. This changes this repo's configuration only. The rules for which specs stay literal are documented in the globalCatalogs comment in common/config/rush/pnpm-config.json.

@bmiddha
Bharat Middha (bmiddha) added this pull request to stack #6134 October 14, 2026 21:29
- Bump pnpmVersion from 10.27.0 to 12.13.0. pnpm 12.10.1 and older drop
  files re-included by .npmignore negations, which breaks injected
  dependencies and rush-pnpm pack (fixed upstream in pnpm 12.11.0).
- Replace the globalNeverBuiltDependencies setting, which pnpm 11 removed,
  with globalAllowBuilds. Explicitly allow the native-binary packages that
  pnpm 10 previously built by default.
- Regenerate both subspace lockfiles and repo-state.json with pnpm 12.
  The ms-feed tarball URLs that pnpm records are stripped because the
  feed rotates its ms-feed-N hosts and pnpm 12 rejects mismatched
  tarball URLs.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Add a "default" catalog to common/config/rush/pnpm-config.json with the
ECV-consistent version of each external dependency, and reference it with
"catalog:" from project dependencies, devDependencies, and
optionalDependencies.

Specifiers that stay literal: peer dependencies, Rush project and decoupled
local dependencies, allowedAlternativeVersions, npm: aliases and dist-tags,
@types/vscode (validated as semver by vsce), versions that differ from the
catalog in build-tests-subspace, and production dependencies of projects that
build-tests-subspace installs as injected file: dependencies (PNPM cannot
resolve "catalog:" in those manifests).

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@bmiddha
Bharat Middha (bmiddha) force-pushed the bmiddha-migrate-rushstack-to-pnpm-catalogs branch from 51d9b41 to d3f36bb Compare October 15, 2026 01:52
…hs in snapshots

The loader output snapshots embedded the pnpm virtual store folder name
of css-loader. With the pnpm 12 lockfile, that name gains the resolved
optional peers of webpack (@swc/core, esbuild, uglify-js) and grows to
103 characters, which pnpm shortens differently on Windows (60) than on
other platforms (120). Replace the folder name with a placeholder before
snapshotting so the snapshots are platform- and lockfile-independent.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
SuccessfulMutationFixture copies the repo's own PNPM installation and
checks its version by running node_modules/pnpm/bin/pnpm.cjs. PNPM 11 and
newer ship a native executable and replace that script with
bin/pnpm.mjs, so with this repo on PNPM 12.13.0 the native install and
update tests in rush-daemon and rush-cli-client failed with "Cannot find
module .../bin/pnpm.cjs". Use bin/pnpm.cjs when it exists, otherwise
bin/pnpm.mjs.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The package feed's registry metadata only provides the sha1 `shasum`
for each tarball, not the `integrity` field. PNPM 10 kept the sha512
hashes that were already in the lockfiles, but regenerating the
lockfiles with PNPM 12 re-resolved every package and replaced 3526
sha512 hashes with sha1 hashes (2788 in the default subspace and 738 in
build-tests-subspace).

Restore the sha512 hash from the PNPM 10 lockfile for each of those
packages. The package keys (name, version, and peers) are unchanged, so
the tarballs are identical. Every package now has the same integrity
hash as in the PNPM 10 lockfile, except 6 that PNPM 12 upgraded from
sha1 to sha512. repo-state.json is regenerated.

"rush update" keeps the restored hashes, and "rush install" in CI mode
with an empty PNPM store downloads and verifies every tarball without
changing the lockfiles.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Needs triage

Development

Successfully merging this pull request may close these issues.

1 participant