Skip to content

Fix python -m venv end-to-end on relenv (gh-334 + gh-161) - #349

Open
dwoz wants to merge 1 commit into
mainfrom
fix-python-m-venv-on-windows
Open

dwoz wants to merge 1 commit into
mainfrom
fix-python-m-venv-on-windows

Conversation

@dwoz

@dwoz dwoz commented Nov 20, 2026

Copy link
Copy Markdown
Contributor

Closes #334, closes #161.

Two paired bugs on python -m venv against a relenv onedir. They compose but are independent, with fixes in different files; both are addressed here with regression tests that run on Windows CI.

gh-334 (commit 5bcd3d3)

python -m venv <target> fails on Windows 3.13+ with:

[WinError 2] The system cannot find the file specified
Unable to copy 'Lib\venv\scripts\nt\venvlauncher.exe' to '<venv>\Scripts\python.exe'

CPython 3.13 rewrote venv.EnvBuilder (upstream gh-99834) to look for the launchers exclusively at Lib/venv/scripts/nt/venv{w,}launcher.exe, dropping the Scripts/-adjacent fallback that 3.10-3.12 used. Earlier relenv builds staged the launchers only in Scripts/ and the finalize archive globs stripped the entire Lib/venv/scripts/ subtree, so even once the launchers were in the right place the child venv would be missing its activation scripts. Fix in relenv/build/windows.py:

  • after shutil.copytree(Lib), also copy both launchers into Lib/venv/scripts/nt/
  • add /Lib/venv/scripts/* to the finalize tarball globs so activate, activate.bat, Activate.ps1, activate.fish, deactivate.bat survive archival

gh-161 (commit e15b415)

python -m venv succeeds (on a 3.10–3.12 onedir, or on 3.13+ after the gh-334 fix), but the resulting child venv has no relenv runtime wiring: no RelenvImporter on sys.meta_path, no OpenSSL FIPS provider ordering, no sysconfig wrapping, no cargo config. On FIPS-enabled hosts this surfaces as hashlib failures (the original report); off FIPS it still leaves the child quietly running without the hooks relenv installs.

Root cause: relenv's base-interpreter runtime is wired up by a relenv.pth dropped into the base's site-packages/. python -m venv creates a child with a fresh empty site-packages/ and no such .pth, so bootstrap() never runs in the child.

Fix in relenv/runtime.py: new wrap_venv() that monkey-patches venv.EnvBuilder.post_setup from inside bootstrap() to drop a relenv.pth into every child venv it creates, baking the absolute path of this installation's runtime.py into the file so the child can import it without having a relenv package in its own site-packages. Idempotency guard via EnvBuilder._relenv_wrapped (same pattern used elsewhere in the module). Exceptions during post_setup are swallowed and logged under RELENV_DEBUG so a quirky EnvBuilder subclass can never break the user's python -m venv. Follows the project's existing design decision (commit 9346e9e, Mar 2023) to use .pth instead of sitecustomize.

Covers stdlib venv only. virtualenv, uv venv, poetry env use their own venv-creation paths and would need parallel hooks — tracked as a follow-up.

Regression tests (same commit as the gh-161 fix)

Two new tests in tests/test_verify_build.py:

  • test_venv_inherits_relenv_runtime (all platforms): creates a stdlib venv from the relenv onedir, asserts RelenvImporter is on the child's sys.meta_path. Sanity-checks the base interp has it first so a broken .pth fails loud rather than being papered over by the venv check.
  • test_python_m_venv_windows (Windows-only, pairs with Cannot create venv on Windows due to venvlauncher.exe and venvwlauncher.exe not being in expected folder #334): asserts python -m venv succeeds, the resulting venv has both launcher executables (renamed to python.exe/pythonw.exe by EnvBuilder) and the full set of activation scripts (activate, activate.bat, Activate.ps1, deactivate.bat), and the child's sys.base_prefix points back at the onedir.

Test plan

Closes #334, closes #161.

Two paired bugs on ``python -m venv`` against a relenv onedir.  They
compose but are independent, with fixes in different files; both are
addressed here with regression tests that run on Windows / macOS /
Linux CI via ``verify-build-action.yml``.

## gh-334 (relenv/build/windows.py)

``python -m venv <target>`` fails on Windows 3.13+ with::

    [WinError 2] The system cannot find the file specified
    Unable to copy 'Lib\venv\scripts\nt\venvlauncher.exe'
        to '<venv>\Scripts\python.exe'

CPython 3.13 rewrote ``venv.EnvBuilder`` (upstream gh-99834) to look
for the launchers exclusively at
``Lib/venv/scripts/nt/venv{w,}launcher.exe``, dropping the
``Scripts/``-adjacent fallback 3.10-3.12 used.  Earlier relenv builds
staged the launchers only in ``Scripts/`` and the ``finalize`` archive
globs stripped the entire ``Lib/venv/scripts/`` subtree, so even once
the launchers were in the right place the child venv would be missing
its activation scripts.  Two changes in ``build/windows.py``:

* after ``shutil.copytree(Lib)`` also copy both launchers into
  ``Lib/venv/scripts/nt/``
* add ``/Lib/venv/scripts/*`` to the ``finalize`` tarball globs so
  ``activate``, ``activate.bat``, ``Activate.ps1``, ``activate.fish``,
  ``deactivate.bat`` survive archival

## gh-161 (relenv/runtime.py)

``python -m venv`` succeeds (on 3.10-3.12, or on 3.13+ after the
gh-334 fix) but the child venv has no relenv runtime wiring: no
``RelenvImporter`` on ``sys.meta_path``, no OpenSSL FIPS provider
ordering, no ``sysconfig`` wrapping, no cargo config.  On FIPS-enabled
hosts this surfaces as hashlib failures (the original report); off
FIPS it still leaves the child quietly running without the hooks relenv
installs.

Root cause: relenv's base-interpreter runtime is wired up by a
``relenv.pth`` dropped into the base's ``site-packages/``.
``python -m venv`` creates a child with an empty ``site-packages/`` and
no such ``.pth``, so ``bootstrap()`` never runs in the child.

Fix via new ``wrap_venv()`` in ``relenv/runtime.py``, called from the
existing chain of wrappers in ``bootstrap()``: monkey-patch
``venv.EnvBuilder.post_setup`` to drop a ``relenv.pth`` into every
child venv it creates, baking the absolute path of this installation's
``runtime.py`` into the file so the child can import it without having
a ``relenv`` package in its own ``site-packages``.  Follows the
project's existing design decision (commit ``9346e9e``, Mar 2023) to
use ``.pth`` instead of ``sitecustomize``.

Four defensive details:

* Idempotency guard via ``EnvBuilder._relenv_wrapped`` -- same pattern
  used elsewhere in the module to survive ``importlib.reload(site)``.
* ``ImportError`` from ``import venv`` is swallowed -- some stripped
  stdlib builds and the nox-spawned unit-test env under
  setup-python + virtualenv 21.15 land without ``venv``, where there's
  nothing to wrap.  Matches ``wrapsitecustomize``'s posture.
* Any exception inside the wrapped ``post_setup`` is caught and logged
  under ``RELENV_DEBUG`` so a quirky ``EnvBuilder`` subclass can never
  break a user's ``python -m venv``.
* Skip the ``.pth`` drop when the venv is pip's isolated build-env
  (path contains ``pip-build-env``, pip's stable public prefix at
  ``pip/_internal/utils/temp_dir.py::tempdir_kinds.BUILD_ENV``).
  Without this, pip's wheel-build envs re-bootstrap the full relenv
  runtime -- hijacking ``sysconfig.get_config_var("BINDIR")`` to point
  at the base onedir and breaking wheel scheme computation, surfaced
  by ``test_pip_install_salt_pip_dir`` on macOS when salt builds from
  sdist.  Mirrors the same ``"pip-build-env"`` marker that
  ``wrapsitecustomize`` already uses to skip its ``sys.path`` munging.

Covers stdlib ``venv`` only.  ``virtualenv``, ``uv venv``, ``poetry
env`` use independent venv-creation paths and would need parallel
hooks -- tracked as a follow-up on the issue.

## Regression tests (tests/test_verify_build.py)

* ``test_venv_inherits_relenv_runtime`` (all platforms): creates a
  stdlib venv from the relenv onedir, asserts ``RelenvImporter`` is on
  the child's ``sys.meta_path``.  Sanity-checks the base interp first so
  a broken ``.pth`` fails loud rather than being papered over by the
  venv check.
* ``test_python_m_venv_windows`` (Windows-only, pairs with gh-334):
  asserts ``python -m venv`` succeeds, the resulting venv has both
  launcher executables and the full set of activation scripts, and the
  child's ``sys.base_prefix`` points back at the onedir.

End-to-end verified on ``win-11`` SSH host before the first push: a
baseline v0.23.1 onedir reproduces ``[WinError 2]``; the gh-334 patch
alone unblocks ``python -m venv``; the gh-161 patch on top makes the
child inherit ``RelenvImporter`` on ``sys.meta_path``.
@dwoz
dwoz force-pushed the fix-python-m-venv-on-windows branch from ee72f35 to fc52584 Compare November 20, 2026 08:40

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant