[Claptar]

Closes #102. Part of #87. For 0.17.5: merge after 0.17.4 is tagged.

The workflows test what they say, publish only what passed, and hold no token they don't need. main (with B7) is merged in, and B7's floors job has the same pins.

Per finding

• F144 (tests.yml): the unit job sets UV_PYTHON from the matrix, which outranks .python-version (3.12). A step then prints the interpreter and fails if it isn't the leg's. Checked:

  • CI on this PR: the py3.13 leg runs CPython 3.13 and the py3.12 leg 3.12.
  • Locally, the unit suite also passes on 3.13 and 3.14.

• F143 (quay-on-tag.yml, tests.yml, .github/scripts/smoke-image.sh): the image is pushed only after the tests, the version check and the PyPI upload have passed.

  • quay-on-tag.yml is now only a reusable workflow, called by publish.yml after publish-pypi.
  • It builds the image once, runs the smoke script on it, then pushes that same image.
  • The smoke script checks --help, that --version equals the release, a non-root uid, duckdb, and convert csv→h5ad→zarr plus view as the caller on a read-only root.
  • A new container image job in tests.yml builds the image and runs the same script on every PR. It also runs inside Publish's test, so an image that doesn't build or run stops the release before PyPI has it.
  • :latest moves only when the version is the highest X.Y.Z tag, so not for 1.0.0rc1 or a backport.
  • Quay's secrets are passed by name.

  Checked:

  • test_release_pipeline.py evaluates each job's needs/if: (table below).
  • It also runs the :latest script under GitHub's bash -e against a stand-in git, including a failing lookup.
  • Locally, I built the image (arm64 natively, amd64 under emulation) and the smoke script passes on both. It fails with a wrong version.

• F142 (publish.yml):

  • permissions: {} at the top, and each job asks for its own. The reusable test gets checks/pull-requests: write, and a test checks that every called job fits within its caller's grant.
  • publish-pypi needs check-version and github.event_name == 'push' || inputs.target == 'pypi'.
  • check-version also runs for a PyPI dispatch, and refuses a branch.
  • twine is pinned.

  Checked: the scenario table, and a test that runs check-version's script for a tag, a mismatched tag and a branch.

• F084 (benchmark.yml): two jobs.

  • benchmark measures with contents: read, keeps no checkout credentials, uses no uv cache, and installs adata-cli with uv sync/uv run --frozen.
  • publish (contents: write) installs nothing. It renders results.json with the runner's python3 (stdlib + benchmarks/ from main), commits, and edits the release notes.
  • It commits only for a pushed tag, or a dispatch with publish, and only after a measurement.

  Checked: a bare CPython 3.12 runs benchmarks.report --out/--publish on a copy of docs/. test_a_job_that_can_push_installs_nothing covers every workflow, and the commit condition is tested per event.

• F141 (all workflows):

  • every action is pinned to the commit of its current major tag, with the exact version in a comment, so no action is upgraded;
  • every workflow pins uv 0.13.1;
  • the tag and inputs reach scripts through env:.

  Checked: tests, and zizmor (no impostor-commit, ref-version-mismatch or advisory findings).

• F138 (Dockerfile):

  • the base is pinned by index digest: uv:0.9.30-python3.12-bookworm-slim@sha256:e5b6558…. That is the image the floating tag names today, so the base doesn't change;
  • each duckdb zip is checked with sha256sum -c. I took the hashes from the zips as GitHub and install.duckdb.org serve them, and the two agree;
  • the image runs as USER 10001:10001.

  Checked: both arches build (/tmp/duckdb.zip: OK), a wrong hash fails the build, and now container image builds the image on every PR.

• F085 (benchmarks/report.py): "vs previous" is now the release just before, by version and at the same tier. A run that isn't a release compares with the newest release, and the page names the run it compares with. Checked:

  • The old code compared every release with whichever file git wrote last.
  • test_every_published_run_compares_with_the_release_before_it checks every run in docs/benchmarks/ against packaging.version, including when a dispatch's <sha>.json is in the store.
  • Another test sets file mtimes deliberately out of order.

Publishing, run by run

From test_each_run_publishes_exactly_what_passed:

Run	Publishes
tag pushed	PyPI, then the image
dispatch testpypi from a tag	TestPyPI only (main: also PyPI)
dispatch pypi from a tag	PyPI, then the image
dispatch testpypi from a branch	TestPyPI only
dispatch pypi from a branch	nothing, and the run fails (main: PyPI)
tag, tests (incl. image build) fail / tag ≠ version / build fails	nothing (main: the image, and :latest)
tag, PyPI upload fails	nothing more

Against main's workflows, Dockerfile and report.py, 40 of the new and changed tests fail (17 pass, among them the cases where main already published nothing).

Behaviour changes

• The image runs as uid 10001, with working directory /home/adata. Singularity, Apptainer and Docker with -u $(id -u):$(id -g) (as nf-core sets) still run it as the caller.
  • A plain docker run -v "$PWD":/data … now needs --user "$(id -u):$(id -g)" if it writes to the mount. The same applies to reads, if your files aren't world-readable.
  • The same goes for Nextflow with Docker and no runOptions. docs/guides/pipelines.md says so.
  • Dropping the USER line restores root.
• A release now also needs the container image job: it is part of tests.yml, which Publish runs. The workflow lint job is skipped inside a release.
• A manual PyPI dispatch has to be run from the tag; from a branch it fails. "Re-run failed jobs" works as before.
• The image goes out after PyPI, inside the Publish run. There is no image-only dispatch. A transient push failure is fixed with "Re-run failed jobs". A broken image is now caught before PyPI, in test.
• :latest follows the highest X.Y.Z tag; a backport or a pre-release no longer takes it. The pushed image no longer carries buildx's provenance attestation, because it is pushed with docker push.
• Smaller changes: the benchmark artifact is now named benchmark-results, and CI uses uv 0.13.1. The image keeps its base's uv 0.9.30.

Independent review

A fresh subagent checked each workflow on a PR, a push to main, a tag, and each dispatch, and confirmed:

• the action pins and the scenario table;
• no permission errors at workflow start;
• inputs, outputs and artifacts matching across jobs;
• F144 on CI (3.13.17 / 3.12.15).

What it raised, and what I did:

1. workflow lint was red and would block the release. zizmor's only findings were self-repository (use $/ for ./ reusable workflows) on two lines of publish.yml, which exits 12. Fixed: an inline # zizmor: ignore[self-repository], since actionlint 1.7 doesn't know $/.
2. Lint and zizmor's online audits gated the release. Fixed: the job is skipped on tags, and a test checks this.
3. The pushed image was a rebuild, and smoke/ landed in the build context. Fixed: the build runs once and the checked image is pushed. The smoke data lives in $RUNNER_TEMP.
4. A benchmark dispatch with publish would break the F085 oracle test on main (Version("8d779d6")). Fixed, with a test.
5. An image failure after PyPI had no in-repo recovery. Fixed at the root: the image is built and smoke-tested in tests.yml, before PyPI.
6. bash -e without pipefail let a failed ls-remote leave :latest behind silently. Fixed: set -o pipefail. The test now runs under bash -e, with a failing git.

Nits:

• Done:
  • a benchmark dispatch from a tag commits only when asked;
  • twine is pinned;
  • a PyPI dispatch from a branch fails;
  • import yaml instead of importorskip;
  • the test gaps (called-workflow permissions, the benchmark commit condition, more installers).
• Not done:
  • docs/changelog.md/STATUS.md: the plan's rule is that the release PR collects entries from this description.
  • Node 20 deprecation of the pinned actions, and Codecov's tokenless failure: both pre-existing, listed below.
  • The F138 test is still a regex, but container image now builds the image for real on every PR.

Considered, kept

The benchmark page is rendered with main's report.py and page_template.md, not the tag's. The page lives on main, and its wording is edited there (AGENTS.md says so). Rendering with the tag's template would undo wording changes made after the tag.

Only the next tag (0.17.5) can confirm

• PyPI trusted publishing still accepts the upload. publish.yml and the pypi environment are unchanged.
• The Quay login, using secrets passed into the called workflow; the docker push of 0.17.5 and latest.
• publish-unit-test-result-action writing its check run under test's grant.
• The benchmark publish job pushing to main and editing the release notes.

No tag was pushed and no Publish, Benchmark or image workflow was dispatched.

Follow-ups (not done here)

• Python 3.14: add a leg. The suite passes on 3.14 locally, but the classifiers stop at 3.13.
• Keep the pins fresh: Dependabot for github-actions and docker. The pinned actions run on Node 20, which GitHub is retiring, so new majors will be needed.
• Base image: bookworm-slim hasn't been updated since Feb 2026; moving to trixie is its own change.
• Repository settings: the default workflow token is write, and the pypi environment has no protection rules.
• Benchmark baselines: anndata, scanpy, dask and geopandas are still the newest from PyPI, and their results.json is rendered into the docs page. Pinning them, or escaping the rendered strings, would close that.
• Codecov: it needs a token; uploads fail, but not the job.

Changelog entry

Fixed

• The py3.13 CI leg runs Python 3.13 (#102). uv took its interpreter from .python-version before the one on PATH, so the leg ran the suite on 3.12. Each leg now sets UV_PYTHON and checks what it got.
• The image is pushed only when the release passed (#102). It was built and pushed on any tag, moving :latest even when tests failed or the tag named the wrong version. It is now built and smoke-tested with the tests, before PyPI, and pushed after PyPI has the release. :latest moves only to the newest final version.
• A TestPyPI dispatch no longer publishes to PyPI (#102). Run from a tag, it published to both. A PyPI dispatch now has to be run from a tag, and each Publish job asks only for the permissions it uses.
• The benchmark's "vs previous" is the release before (#102). It compared with an arbitrary earlier run (0.17.0 with 0.8.0); it now takes the previous version at the same tier, and the page names it.
• Least privilege in the release workflows (#102). The benchmark no longer runs unpinned packages with a token that can push to main. Actions and uv are pinned, and tags reach scripts through env:.

Changed

• The container image runs as an unprivileged user (#102). It runs as uid 10001 rather than root, and its base is pinned by digest. duckdb's download is checked against a pinned SHA-256. Singularity, Apptainer and Docker with -u still run it as the caller. A plain docker run on a bind mount needs --user "$(id -u):$(id -g)".

🤖 Generated with Claude Code

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	⚠️ Failed 2026-10-29T19:16:48.305196Z We couldn’t complete this review. Comment @codex review to try again.	a90c51d	PR opened

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota.

An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing.

[chatgpt-codex-connector]

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

[github-actions]

Test Results (py3.12)

3 015 tests  +55   3 011 ✅ +55   8m 26s ⏱️ - 3m 12s
    1 suites ± 0       4 💤 ± 0 
    1 files   ± 0       0 ❌ ± 0 

Results for commit 9b12112. ± Comparison against base commit e128b71.

♻️ This comment has been updated with latest results.

[github-actions]

Test Results (py3.13)

3 015 tests  +55   3 011 ✅ +55   11m 12s ⏱️ +19s
    1 suites ± 0       4 💤 ± 0 
    1 files   ± 0       0 ❌ ± 0 

Results for commit 9b12112. ± Comparison against base commit e128b71.

♻️ This comment has been updated with latest results.