Repository navigation
Description
This issue is a collection of sub-tasks for:
-
Minimizing our dependencies to secrets
-
Removing un-necessary CI items
-
Augment the audit
-
Hardening who can do what and how in terms of commits, branches, tags
-
harden branches
-
fork only policy for branches for PRs
-
harden tags
-
check for unused secrets
NPM releases:
- Trusted publisher instead of
NPM_TOKEN: ci: trusted staged publish diracx-web#517- verify with a release
PyPI releases. Use trusted publisher (with environment) instead of PYPI_TOKEN: coming soon (for ref: https://docs.github.com/en/actions/how-tos/secure-your-work/security-harden-deployments/oidc-in-pypi)
-
Use Github token instead of
RELEASE_PLEASE_TOKEN:- intercede: ci: use github token with release-please intercede#41 + gh environment setting + trusted publisher
- verify with a release
- signurlarity: ci: relying on github token with release-please signurlarity#66 + gh environment setting + trusted publisher
- diracx: ci: use GITHUB_TOKEN for release-please and publish with the pypi environment diracx#1187 + gh environment setting + trusted publisher
- DIRAC: ci: publish DIRAC and DIRACCommon to PyPI via trusted publisher DIRAC#8886 (integration), [v9r0] chore: publish DIRAC and DIRACCommon to PyPI via trusted publisher DIRAC#8888 (rel-v9r0), [v8r0] chore: publish DIRAC to PyPI via trusted publisher DIRAC#8887 (rel-v8r0) + gh environment setting + trusted publisher
- WebAppDIRAC: ci: publish WebAppDIRAC to PyPI via trusted publisher WebAppDIRAC#818 (integration), chore: publish WebAppDIRAC to PyPI via trusted publisher WebAppDIRAC#819 (rel-v5r0) + gh environment setting + trusted publisher
- DB12: ci: publish DB12 to PyPI via trusted publisher DB12#18 + gh environment setting + trusted publisher
- diraccfg: ci: publish diraccfg to PyPI via trusted publisher diraccfg#13 + gh environment setting + trusted publisher
- intercede: ci: use github token with release-please intercede#41 + gh environment setting + trusted publisher
-
DIRACOS2 releases: Something with Conda?
-
container-images: still needed? If not, archive?
-
Renovate instead of
CHART_UPDATE_TOKEN? (would likely be more consistent with our current workflows with Renovate) chore: set up Renovate to bump diracx/diracx-web versions diracx-charts#322 along with ci: stop pushing chart bumps to diracx-charts diracx#1183 & ci: stop pushing chart bumps to diracx-charts diracx-web#518- org-wide preset: chore: add the shared DIRACGrid Renovate config preset #7 (merge this first — the
local>references below cannot resolve until it lands onmain) - how to adopt: add a
renovate.jsoncontainingthe preset provides: digest pinning of GitHub Actions, a 7-day release-age cooldown, grouped Actions updates, no automerge by default{ "$schema": "https://docs.renovatebot.com/renovate-schema.json", "extends": ["local>DIRACGrid/.github//renovate-config"] } - migrated (repos that already used Renovate): diracx chore: extend the shared DIRACGrid Renovate config preset diracx#1189, diracx-web chore: extend the shared DIRACGrid Renovate config preset diracx-web#521, DIRACOS2 chore: extend the shared DIRACGrid Renovate config preset DIRACOS2#211, intercede chore: extend the shared DIRACGrid Renovate config preset intercede#44, signurlarity refactor: extend the shared DIRACGrid Renovate config preset signurlarity#69, diracx-charts chore: extend the shared DIRACGrid Renovate config preset diracx-charts#328
- dependabot -> renovate: container-images chore: replace Dependabot with the shared DIRACGrid Renovate config container-images#52 (removes
.github/dependabot.yml) - DIRAC: chore: extend the shared DIRACGrid Renovate config preset DIRAC#8897 (integration), [rel-v9r0] chore: extend the shared DIRACGrid Renovate config preset DIRAC#8898 (rel-v9r0), [rel-v8r0] chore: extend the shared DIRACGrid Renovate config preset DIRAC#8899 (rel-v8r0)
- diracgrid.org: chore: adopt the shared DIRACGrid Renovate config diracgrid.org#6 (merged)
- Pilot: adopted upstream on
devel(withbaseBranchPatternscovering devel + master) - WebAppDIRAC: adopted upstream on
integration(withbaseBranchPatternscovering rel-v5r0) - management: chore: adopt the shared DIRACGrid Renovate config management#78 (CI fix for fork PRs: ci: only push the el9-dirac image from master management#79)
- mkdocs-diracx-plugin: chore: adopt the shared DIRACGrid Renovate config mkdocs-diracx-plugin#10
- diraccfg: chore: adopt the shared DIRACGrid Renovate config diraccfg#14 (CI unblocked by ci: run the CI jobs with pixi diraccfg#15, conda disappeared from the GH runners)
- tornado_m2crypto: chore: adopt the shared DIRACGrid Renovate config tornado_m2crypto#11
- DIRACWebAppResources: chore: adopt the shared DIRACGrid Renovate config DIRACWebAppResources#1
- dirac-webapp-packaging: chore: adopt the shared DIRACGrid Renovate config dirac-webapp-packaging#1
- skip: dormant repos (tornado)
- org-wide preset: chore: add the shared DIRACGrid Renovate config preset #7 (merge this first — the
-
Re-request a review-native mechanism instead ofDRAFT_PAT?: ci: remove draft PR workflows and adjust #4 along with ci: drop the draft conversion on changes requested mkdocs-diracx-plugin#8 & ci: drop the draft conversion on changes requested diracx-charts#323 & ci: drop the draft conversion on changes requested diracx-web#519 & ci: drop the draft conversion on changes requested diracx#1184 -
Stop sending images to docker: Update images-creator.yml to remove Docker Hub steps management#76
- No more sweep and labeler: Remove labeler and pr-sweep GitHub actions DIRAC#8881 & Remove labeler and pr-sweep GitHub actions DIRAC#8882 & Remove labeler and pr-sweep GitHub actions DIRAC#8883 and Remove labeler and pr-sweep GitHub actions WebAppDIRAC#817 & Remove labeler and pr-sweep GitHub actions WebAppDIRAC#816
-
Adopt
pixieverywhere (instead of conda/micromamba in CI): locked environments (pixi.lock) make dependency cooldowns actually enforceable — Renovate's lockfile-maintenance PRs become the only path for dependency changes, withminimumReleaseAgeapplied, instead of CI resolving transitive deps to latest at runtime. One toolchain (prefix-dev/setup-pixi+[tool.pixi]) across the org; DIRACOS2 already builds the full conda distribution with pixi.- already on pixi: diracx, DIRACOS2, intercede, signurlarity (+ diracx-charts/diracx-web use pixi for CI tooling)
- WebAppDIRAC: the
integrationbranch no longer has the conda-based pylint job (removed upstream) — nothing to migrate there; rel-v5r0 migrated: [rel-v5r0] feat: run the pylint job with pixi WebAppDIRAC#822 (stacked on #821) - Pilot: the
develbranch has no conda usage in CI at all (the conda-basedpilotWrapper.ymlonly exists onmaster) — nothing to migrate there - DIRAC: feat: run the check jobs with pixi DIRAC#8900 (integration; stacked on #8891) —
check/check-py314jobs migrated to locked pixi environments (py311 + py314,exclude-newerenforced); thepylint-py27job was already removed (#8894/#8895); ported to the release branches: rel-v9r0 [v9r0] feat: run the check jobs with pixi DIRAC#8905 (stacked on #8892) and rel-v8r0 feat: run the check jobs with pixi DIRAC#8907 (stacked on #8893). Fixes after first CI round: commitpixi.lock(was still gitignored →Failed to generate cache keyon all check jobs) and un-quote$MATRIX_ARGSin the integration workflow (quoting merged theKEY=VALUEpairs ofmatrix.ARGSinto one argument) - DIRAC pilotWrapper: feat: run the pilot wrapper tests with pixi DIRAC#8903 (stacked on #8900) — the last ad-hoc conda usage in DIRAC's CI (run-time
conda createfor py3.6.9/3.11.4/3.14.6) replaced by locked pixi environments, incl. a new bare-bonespy36env (no-default-feature, solves to python 3.6.15); ported to the release branches: rel-v9r0 [v9r0] feat: run the pilot wrapper tests with pixi DIRAC#8906 (stacked on #8905) and rel-v8r0 feat: run the pilot wrapper tests with pixi DIRAC#8908 (stacked on #8907).deploy_CVMFSkeeps conda for the gsissh/CVMFS tooling - pixi audit (Nov 17): all org repos now use pixi or are deliberate exceptions — Pilot (
devel) and DB12 run digest-pinned containers with no conda at all (out of scope); management/mkdocs-diracx-plugin/container-images/.github/DIRACWebAppResources/dirac-webapp-packaging/tornado_m2crypto have no Python CI to migrate; diracgrid.org builds docs with run-timepip install(unlocked but docs-only, no conda) - exclude-newer rollout (set it in every pixi repo, per chore: add the shared DIRACGrid Renovate config preset #7): done for WebAppDIRAC rel-v5r0 (in #822), diraccfg (in the pixi CI migration), intercede chore: enforce the release-age cooldown at solve time (exclude-newer = 7d) intercede#45, signurlarity chore: enforce the release-age cooldown at solve time (exclude-newer = 7d) signurlarity#70, diracx-charts chore: enforce the release-age cooldown at solve time (exclude-newer = 7d) diracx-charts#334, tornado chore: enforce the release-age cooldown at solve time (exclude-newer = 7d) tornado#7 (no lockfile there; also bumps the CI pixi v0.59.0 → v0.84.1, which is required to parse the relative
"7d"form). diracx-web has no pixi manifest of its own — its deployment workflow runs pixi from a diracx-charts checkout, so it is covered by the diracx-charts change. Note: intercede and signurlarity gitignorepixi.lock(CI re-solves at run time, where exclude-newer now enforces the cooldown); if their locks are ever committed, the Renovate pixi lockfile-maintenance rule applies to them too - Renovate follow-up: add the
pixilockfile-maintenance rule (currently only in diracx) to the shared preset in .github - diraccfg: no longer parked — conda is gone from the
ubuntu-latestrunners (last green CI Nov 13), CI migrated to locked pixi environments: ci: run the CI jobs with pixi diraccfg#15 (stacked on ci: publish diraccfg to PyPI via trusted publisher diraccfg#13)
-
add
zizmoreverywhere as pre-commit, to harden the CIs. (already in diracx, intercede: ci: add zizmor (pixi task + pre-commit) and fix its auditor findings intercede#43, signurlarity: ci: add zizmor (pixi task + pre-commit) and fix its auditor findings signurlarity#68)- DB12: chore: add zizmor (pre-commit) and fix its auditor findings DB12#19 (stacked on #18)
- DIRAC: chore: add zizmor (pre-commit) and fix its auditor findings DIRAC#8891 (integration), [v9r0] chore: add zizmor (pre-commit) and fix its auditor findings DIRAC#8892 (rel-v9r0), [v8r0] chore: add zizmor (pre-commit) and fix its auditor findings DIRAC#8893 (rel-v8r0) — stacked on the PyPI PRs
- WebAppDIRAC: chore: add zizmor (pre-commit) and fix its auditor findings WebAppDIRAC#820 (integration), [v5r0] chore: add zizmor (pre-commit) and fix its auditor findings WebAppDIRAC#821 (rel-v5r0) — stacked on the PyPI PRs
- DIRACOS2: chore: add zizmor (pre-commit) and fix its auditor findings DIRACOS2#210
- Pilot: chore: add zizmor (pre-commit) and fix its auditor findings Pilot#291
- container-images: chore: add zizmor (pre-commit) and fix its auditor findings container-images#51
- diracx-charts: chore: add zizmor (pre-commit) and fix its auditor findings diracx-charts#326
- diracx-web: chore: add zizmor (pre-commit) and fix its auditor findings diracx-web#520
- management: chore: add zizmor (pre-commit) and fix its auditor findings management#77
- mkdocs-diracx-plugin: chore: add zizmor (pre-commit) and fix its auditor findings mkdocs-diracx-plugin#9
- diracgrid.org: chore: add zizmor (pre-commit) and fix its auditor findings diracgrid.org#5 (merged)
- .github: chore: add zizmor (pre-commit) and fix its auditor findings #6
- tornado: chore: add zizmor (pre-commit) and fix its auditor findings tornado#5
- diraccfg: todo (CI migration ci: run the CI jobs with pixi diraccfg#15, stacked on ci: publish diraccfg to PyPI via trusted publisher diraccfg#13)
- diracx (already there)
- intercede: ci: add zizmor (pixi task + pre-commit) and fix its auditor findings intercede#43
- signurlarity: ci: add zizmor (pixi task + pre-commit) and fix its auditor findings signurlarity#68
- Keep the pre-commit hook revs updated, and run the hooks the same way everywhere (audit of Nov 17):
- Enable Renovate's
pre-commitmanager in the shared preset: chore: enable Renovate's pre-commit manager in the shared preset #8 — it is off by default in Renovate, so before this the pinned hook revs never got update PRs (several repos still pin 2023-era hooks:black 23.1.0,flynt 0.77,pyupgrade v3.3.1, which crash on Python ≥ 3.12 runners) - Plan: converge all repos on the same CI pattern —
pre-commit/actionruns the hooks,pre-commit-ci/lite-actionpushes auto-fixes to PR branches; Renovate does the rev updates. No more full pre-commit.ci app (it is an external app with write access to the repos):- diracx: ci: run all pre-commit hooks in CI and rely on Renovate for hook updates diracx#1193 — full replacement of the app: all hooks run in CI (pixi-dependent ones via
stages: [manual]+--hook-stage manual), Renovate does the updates; the pre-commit.ci app can be uninstalled once merged (it also covers diracx-charts, where the app is installed: last autoupdate Dec 2025) - DIRAC: ci: run pre-commit via pre-commit/action and lite-action DIRAC#8904 (stacked on #8903 → #8900 → #8891); release branches via the pixi ports #8905 (rel-v9r0) and #8907 (rel-v8r0), which include the same pre-commit change
- WebAppDIRAC: ci: run pre-commit via pre-commit/action and lite-action WebAppDIRAC#825 (rel-v5r0 can follow via #821 once validated)
- already on the pattern: intercede, signurlarity, diracx-charts (
pre-commit/action+pre-commit-ci/lite-action) - remaining plain-style repos (same enforcement, no auto-fix — optional to convert): DB12, diraccfg, diracgrid.org, mkdocs-diracx-plugin, Pilot, .github, container-images, DIRACOS2
- hooks needing Python 3.11 pins until Renovate updates them: DIRAC, WebAppDIRAC, DB12, diraccfg (pin
black/flynt/pyupgradefirst, then move the CI job topython-version: 3.x)
- diracx: ci: run all pre-commit hooks in CI and rely on Renovate for hook updates diracx#1193 — full replacement of the app: all hooks run in CI (pixi-dependent ones via
- Enable Renovate's
- Minimize GITHUB_TOKEN permissions: always set permissions: {} at the workflow level to disable all permissions by default. Then, grant only the specific permissions needed at the job level.
- investigate actions's lockfiles, and make sure we have cooldowns
Activity
[-]On minimizing our dependencies to user tokens[/-][+]On minimizing our dependencies to secrets[/+][-]On minimizing our dependencies to secrets[/-][+]Security hardening (tasks collection)[/+]fstagni commentedon Nov 17, 2026
Status update on the failing PRs (all diagnosed, fixes pushed where needed):
New PRs opened:
ci: run the CI jobs with pixi(stacked on #13). Conda has disappeared from theubuntu-latestrunners (last green CI Nov 13, thenconda: command not found), which was breaking the diraccfg CI (and the checks on #13/#14). diraccfg is therefore un-parked: locked pixi environments (py39/py310/py311, committedpixi.lock), build viapython -m build, actions pinned, minimal permissions.ci: only push the el9-dirac image from master. Fork PRs get a read-onlyGITHUB_TOKEN, so the unconditionaldocker pushalways failed on PRs (installation not allowed to Write organization package) — this is what was red on #78 (and #77 earlier). Same guard as thecertificates-generationjob already uses.Fixes pushed to existing PRs:
pixi.lockwas still in.gitignore→ never committed →prefix-dev/setup-pixifailed withFailed to generate cache keyon allcheck/check-py314jobs (and the environments weren't actually locked). Lockfile now generated and committed.$MATRIX_ARGS, merging allKEY=VALUEpairs ofmatrix.ARGSinto a single argument (MYSQL_VER="mariadb:11.4.3 ES_VER=..."→ invalid docker reference; backward-compat job got a mangledINSTALLATION_BRANCH). Word-splitting restored.upstream/rel-v8r0to resolve the conflict — the only conflict was thepylint-py27job, dropped in favor of the upstream removal (#8895). Now MERGEABLE again.Diagnosed as not caused by the PRs — re-runs triggered:
integration/rel-v9r0. The failed jobs have been re-run.main(cert-manager webhook not ready whenissuer.ymlis applied → diracx/dex pods never healthy →Connection refusedincli_env).mainis green again; failed jobs re-run.mainsince Nov 16 — same cert-manager demo bootstrap failure (secrets "root-secret" not found, webhookconnection refused/x509) plusghcr.io/gubbins/tasks:devanonymous-pull 403 in the gubbins run. Needs a separate infra issue for the diracx demo/chart maintainers; a plain re-run of #521 will likely fail until that's fixed.Still open (unchanged): the zizmor PRs awaiting review,
exclude-newerrollout (intercede, signurlarity, diracx-charts, diracx-web), the pixi lockfile-maintenance rule in the shared Renovate preset, and the diraccfg zizmor PR (now stacked on #13 → #15).chore: disable Renovate's pre-commit manager, pre-commit.ci does the …
chore: run pre-commit via pre-commit/action and lite-action
chore: run pre-commit via pre-commit/action and lite-action
chore: enable Renovate's pre-commit manager in the shared preset (#8)
chore: enforce the release-age cooldown at solve time (exclude-newer …