Skip to content

feat: run the pilot wrapper tests with pixi - #8903

Open
fstagni wants to merge 8 commits into
DIRACGrid:integrationfrom
fstagni:pilotwrapper-pixi
Open

fstagni wants to merge 8 commits into
DIRACGrid:integrationfrom
fstagni:pilotwrapper-pixi

Conversation

@fstagni

@fstagni fstagni commented Nov 17, 2026

Copy link
Copy Markdown
Contributor

Part of DIRACGrid/.github#5 (adopt pixi everywhere instead of conda in CI).

Replaces the last ad-hoc conda usage in DIRAC's CI: pilotWrapper.yml created its conda environments at run time with setup-miniconda (conda create -c conda-forge python=3.6.9/3.11.4/3.14.6). They now come from the locked pixi environments:

  • a new py36 environment is added to pixi.toml with no-default-feature = true (bare python only, like the previous conda env that deliberately excluded pip — the shared dependencies require python >= 3.9). It solves to python 3.6.15, the last conda-forge build of the 3.6 series
  • py311/py314 reuse the existing default/py314 environments
  • the exact python versions are pinned by pixi.lock, refreshed to include py36

The workflow itself also gets the standard concurrency block, minimal permissions, and drops the now-unneeded conda activation steps.

Note: deploy_CVMFS keeps conda on purpose (gsissh/CVMFS tooling), as agreed in the issue.

Stacked on #8900 (pixi check jobs), which should be merged first.

The CI check jobs now run in the locked pixi environment (pixi.toml +
committed pixi.lock), so make pixi the recommended way to create a
development environment:
- new 'Creating a development environment with pixi' section
  (install, pixi install/shell/run, the py314 environment, and the
  dependency-update policy: pixi.toml + Renovate-managed pixi.lock,
  exclude-newer cooldown)
- note in the conda section that environment.yml now mirrors pixi.toml
  and is still used by readthedocs
- generalize references to 'the conda environment' in the docs-building
  and installation-check pages
…ion workflow

- pixi.lock was still ignored by .gitignore, so it never made it into
  the branch: setup-pixi (cache: true) fails with "Failed to generate
  cache key" on every check/check-py314 job, and the environments were
  not actually locked. Remove it from .gitignore and commit the
  generated lockfile.
- the integration workflow quoted $MATRIX_ARGS, merging all KEY=VALUE
  pairs of matrix.ARGS into a single argument that
  integration_tests.py cannot split, breaking e.g.
  MYSQL_VER=mariadb:11.4.3 ES_VER=... (invalid docker reference) and
  the backward-compatibility job (mangled installation branch). Restore
  the word-splitting.
The plain $COMMAND expansion loses the single quotes around the pytest
-k expression, so pytest tried to open 'test_BaseType_Unicode' as a file
(ERROR: file or directory not found). Use eval, like the conda-based
job did.
Replace the ad-hoc conda environments (created with setup-miniconda at
run time) with the locked pixi environments from pixi.toml, consistent
with the check jobs. A new bare-bones py36 environment (no-default-feature,
python only - like the previous conda env that excluded pip) is added for
the Python 3.6 leg, solving to python 3.6.15 on conda-forge.

Stacked on DIRACGrid#8900.
…e the pixi-spawned shell

- pixi.toml has no 'py311' environment (the py311 one is the 'default'),
  so the py311 matrix legs failed with "unknown environment 'py311'".
- the test exports INVALID_UTF8_VAR=$'\xff' on purpose (the pilot
  wrapper has to cope with invalid UTF-8 in the environment), but pixi
  is now the parent process and panics on non-UTF-8 env vars
  (env::var().unwrap() on "\xFF"). Set the variable inside the
  pixi-spawned shell instead, so only python sees it.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant