Skip to content

feat(nodejs): prepare a monorepo's shared project root once - #941

Open
bnusunny wants to merge 1 commit into
aws:developfrom
bnusunny:feat/nodejs-install-shared-dependencies
Open

bnusunny wants to merge 1 commit into
aws:developfrom
bnusunny:feat/nodejs-install-shared-dependencies

Conversation

@bnusunny

@bnusunny bnusunny commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

Issue #, if available: N/A — completes the monorepo build-performance work of #931, #933/#935 and #938. No linked issue.

Description of changes

Goal: let a caller building several functions from one npm workspaces monorepo install the shared dependency tree once, instead of once per function.

In a workspaces monorepo every member's npm install reconciles the same root node_modules. Build eight functions and npm does that reconciliation eight times, each one largely redundant. #931/#935/#938 made each of those installs correct and cheaper; none of them could remove the repetition, because a workflow only ever sees one function.

LambdaBuilder.prepare_shared_project(project_root, experimental_flags) is the new entry point. A caller runs it once, then calls build() per function with download_dependencies=False; the build-in-source link steps pick the dependencies up from where npm hoisted them.

The hook is named for the contract, not for npm's implementation of it. It promises one shared preparation per project root; the nodejs workflows satisfy that by installing the hoisted dependency tree. A Cargo workspace shares one target directory, where the same phase would compile every member and install nothing, so install_shared_dependencies would have been a misnomer the moment a second workflow adopted it. The exception names follow the same reasoning.

npm invocations median wall clock
8 functions, per-function install 32 6.31 s
8 functions, installed once 11 1.56 s
20 functions, per-function install 80 16.50 s
20 functions, installed once 23 3.08 s

Two generated fixture sets, npm 11.19.0, Linux; medians of repeated runs. Against develop before any of this work (24 invocations / 21.66 s and 60 / 57.18 s), a 20-function monorepo goes from 57 s to 3 s.

The shared install reuses get_install_action, so lockfile handling cannot drift from the per-function path. One difference: no --omit=dev. At a workspace root --omit=dev removes the root's own dev dependencies, where build tools like esbuild live, on every npm version measured. Production filtering of the artifacts does not depend on it — the dependency-closure link ships only production packages, and esbuild bundles only what is imported.

Opting out of the per-function install changed one thing in the link step. A build with download_dependencies=False and a dependencies_dir takes its dependencies from the copy, so the artifacts link must not also run; otherwise it leaves a symlink the copy either keeps (pointing the artifacts at the source tree) or recurses through, writing the cache into the customer's source tree.

Both halves of the feature answer to one flag. prepare_shared_project takes the same experimental_flags the per-function build() calls take and refuses without experimentalNodejsMonorepo, before it spends a subprocess asking npm its version. Without that refusal the two halves could be enabled separately: a caller that installs at the root but omits the flag from its builds gets project_root = None in the link step, which falls back to linking the function's own node_modules — empty in a hoisted monorepo — and ships artifacts with no dependencies and no error.

A workflow with no shared-project concept raises SharedProjectNotSupportedError. A missing flag, an npm without --install-links, and a failed install all raise SharedProjectPreparationError, because the caller's response to each is the same: fall back to per-function installs.

Description of how you validated changes

The end-to-end property is that one shared install plus N dependency-free builds produces the same artifacts as N independent builds. The integration tests assert each function's artifacts carry its own dependency and the shared workspace package but not its sibling's, that the locked version won, and that node -e "require('./included.js')" succeeds from each artifacts directory — the same assertions the per-function path already passes, now reached without a per-function install.

Covered for both nodejs workflows: the plain nodejs_npm path and nodejs_npm_esbuild, where the bundler resolves from the source tree rather than linking.

Unit tests cover the new API surface — the not-supported error when a workflow has no installer, the wrapped failure, and that prepare_shared_project is dispatched to the selected workflow class with the flags it was given — plus the install command itself keeping dev dependencies when omit_dev=False, for both the install and update actions.

The flag gate is tested on both sides of the boundary. Three unit cases pin the refusal for None, [] and a list carrying some other flag, and one asserts the refusal happens before can_use_install_links, so an unflagged caller pays for no subprocess. An integration test drives the real LambdaBuilder and asserts both halves of the contract: the error names the flag, and no node_modules appears at the root. Removing the guard fails all four unit cases.

The link-step guard has its own tests: a build taking dependencies from a dependencies_dir must not emit the artifacts link, which is the case that would otherwise write into the source tree.

  • 897 unit tests pass; ruff check and black --check clean.
  • nodejs + esbuild integration: 316 passed, npm 11.19.0, node 24.21.0, Linux.
  • Benchmarked on two independent fixture sets; npm invocation counts matched exactly between runs.

Checklist

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@bnusunny
bnusunny requested a review from a team as a code owner October 15, 2026 18:50
@bnusunny
bnusunny force-pushed the feat/nodejs-install-shared-dependencies branch from 347956d to 3927340 Compare October 15, 2026 19:24
licjun
licjun previously approved these changes Oct 19, 2026
In an npm workspaces monorepo every member's install reconciles the same root
node_modules, so building N functions does that reconciliation N times. This adds
LambdaBuilder.prepare_shared_project(project_root, experimental_flags): a caller
runs it once per workspace root, then builds each function with
download_dependencies=False.

The hook is named for the contract, not for npm's implementation of it. A Cargo
workspace shares one target directory, and the same phase there compiles every
member rather than installing anything, so 'install' would have been wrong for the
second workflow to adopt it. SharedProjectNotSupportedError and
SharedProjectPreparationError follow the same reasoning.

Gated on experimentalNodejsMonorepo, which prepare_shared_project refuses without:
the per-function build gates its artifacts link on that same flag, so enabling one
half without the other would ship artifacts carrying no dependencies at all.
@bnusunny bnusunny changed the title feat(nodejs): install a monorepo's shared dependencies once per workspace root feat(nodejs): prepare a monorepo's shared project root once Oct 19, 2026
@bnusunny
bnusunny force-pushed the feat/nodejs-install-shared-dependencies branch from 3927340 to d737d28 Compare October 19, 2026 19:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants