Skip to content

fix(action): start setup-python's interpreter outside GitHub's tool cache - #99

Merged
pasrom merged 1 commit into
mainfrom
fix/action-self-hosted-python
Nov 18, 2026
Merged

pasrom merged 1 commit into
mainfrom
fix/action-self-hosted-python

Conversation

@pasrom

@pasrom pasrom commented Nov 18, 2026 •

Copy link
Copy Markdown
Owner

Problem

On a self-hosted Linux runner the km action fails at its first command:

.../_work/_tool/Python/3.11.18/x64/bin/python: error while loading shared libraries: libpython3.11.so.1.0: cannot open shared object file  (exit 127)

Cause, from the sources at the commits the action pins:

  • actions/python-versions builds CPython with --enable-shared and LDFLAGS=-Wl,--rpath=<tool cache>/Python/<version>/<arch>/lib, the tool cache being GitHub's /opt/hostedtoolcache (builders/ubuntu-python-builder.psm1, Configure()). In any other tool cache the interpreter does not find libpython.
  • setup-python works around that itself: it runs the build's setup.sh with LD_LIBRARY_PATH=<install dir>/lib (src/install-python.ts, installPython()), so the install succeeds, and it exports LD_LIBRARY_PATH for the job, but only with update-environment: true (src/find-python.ts, lines 151-172 at 6d4fd22).
  • The action passes update-environment: false, to leave the job's own python alone, so the next command, python -m venv, has no LD_LIBRARY_PATH.

setup-python's docs ("Using setup-python with a self-hosted runner", docs/advanced-usage.md) call the macOS builds non-relocatable and say to install them in /Users/runner/hostedtoolcache; for Ubuntu they describe AGENT_TOOLSDIRECTORY. The fix that docs path implies, AGENT_TOOLSDIRECTORY=/opt/hostedtoolcache, needs a root-writable /opt, so sudo or an image change.

Fix

The install step moves into action/install.sh (same commands, same -P, PYTHONSAFEPATH and pyyaml==6.0.3). Before venv, it starts the interpreter with the job's LD_LIBRARY_PATH removed. If that fails, the script puts the build's lib folder (absolute, with no empty element, which the loader would read as the working directory) on LD_LIBRARY_PATH for the install, and writes it into the generated km command. On GitHub's runners the interpreter starts as it is, so nothing changes there. A runner needs no sudo, no AGENT_TOOLSDIRECTORY, no system Python and no python3-venv.

Options weighed:

  • System python3 + venv: Ubuntu 24.04 has no python3-venv by default, so pip would have to be fetched some other way, and it does not help a runner without Python.
  • uv with a relocatable Python: this works, but it adds a downloaded binary and a new trust root, and it changes GitHub-hosted behaviour.
  • LD_LIBRARY_PATH scoped to km: chosen. It restores what setup-python itself does, needs no new download, and changes nothing where it is not needed.

README "Running on your own runners" says what the runner needs.

Tests

  • tests/action_smoke.sh (new, offline, in the smoke job) uses stand-in interpreters: one that starts, and one that exits 127 unless its lib folder is on LD_LIBRARY_PATH. It checks:

    • the launcher's exact LD_LIBRARY_PATH with the job's unset and with it set,
    • no empty element,
    • that the job's own LD_LIBRARY_PATH does not decide the probe,
    • that an interpreter that never starts fails with a reason,
    • that a relative path is refused.
  • Mutation check: each of these mutations fails at least one case:

    • drop the export,
    • drop the launcher line,
    • always add the lib,
    • add an empty element in the install or in the launcher,
    • drop the job's value,
    • probe with the job's environment.
  • New CI job action-tool-cache (x64, GitHub's image) moves /opt/hostedtoolcache aside and has setup-python install into /tmp/km-tool-cache. A guard step asserts that the venv's interpreter is the one in that cache and that it does not start on its own, so the job cannot pass vacuously. Its log shows the reported error (libpython3.11.so.1.0 => not found, exit 127). Then km --version and km validate run with no LD_LIBRARY_PATH in the job.

  • Plain ubuntu:24.04 containers, x64 and arm64. Each stages python-versions' 3.11.18 build in /home/runner/actions-runner/_work/_tool with its own setup.sh, as setup-python does. Both variants behave the same:

    Container Unfixed first command install.sh km --version, km validate
    no Python at all exit 127, the reported error ok ok
    python3 3.12 without python3-venv (no ensurepip) exit 127 ok ok
  • All smoke tests pass locally.

Review: /simplify and /code-review were run before opening. Skipped findings:

  • printf %q for the paths in the launcher: this was already the case before this change.
  • Windows pwd paths: the action never supported Windows runners, since the venv has no bin/ there.
  • The CI job's reliance on a patch release absent from GitHub's image: fixed, it now moves that tool cache aside instead.
  • Always set the variable: rejected, because GitHub-hosted behaviour would change.

…ache

setup-python's Linux builds find libpython through an RPATH fixed at build
time to /opt/hostedtoolcache. In any other tool cache, as on a self-hosted
runner, the interpreter starts only with its lib folder on LD_LIBRARY_PATH,
which setup-python exports only with update-environment, and the action turns
that off to leave the job's python alone. So `python -m venv` failed with
exit 127 ("error while loading shared libraries").

The install step moves to action/install.sh: where the interpreter does not
start as it is, it puts the build's lib folder (absolute, no empty element)
on LD_LIBRARY_PATH for the install and in the km command alone. On GitHub's
runners nothing changes. -P, PYTHONSAFEPATH and the PyYAML pin stay.

tests/action_smoke.sh covers both cases offline with stand-in interpreters;
a CI job runs the action with setup-python installing into another tool cache.
@pasrom
pasrom force-pushed the fix/action-self-hosted-python branch from 8a07938 to 1a25d3d Compare November 18, 2026 09:32
@pasrom
pasrom merged commit f82f26b into main Nov 18, 2026
5 checks passed
@pasrom
pasrom deleted the fix/action-self-hosted-python branch November 18, 2026 10:01
@pasrom pasrom mentioned this pull request Nov 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant