Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -114,6 +114,16 @@ Set each one up in this order:

`km doctor` notes a missing `mounted-pins` environment in a brain with mounts, and either environment when it is open to more than the default branch.

### Running on your own runners

Every job of the team workflows runs on `${{ vars.KM_RUNNER || 'ubuntu-latest' }}`: set the repository or organisation variable `KM_RUNNER` to one runner label (`my-runner`, say) and they run there; unset, they run on GitHub's runners. bump-mounted-pins reads its own variable `MOUNTED_PINS_RUNNER` first. A brain written by an earlier km gets the change in the shipped workflows through `km upgrade`; `ci.yml` and `staleness.yml` are the brain's own, so change their `runs-on` lines by hand to exactly the template's `runs-on: ${{ vars.KM_RUNNER || 'ubuntu-latest' }}`, and below it, if you like, exactly the template's `timeout-minutes: 15` (km then still counts that `ci.yml` as its own; with any other line `km upgrade` takes it for the brain's own and does not add approve-fork-ci or automerge).

A plain Ubuntu 24.04 VM with git, curl and tar is enough for every workflow but bump-mounted-pins: its jobs run in a container, so its runner needs docker, usable by the runner's user; without docker set `MOUNTED_PINS_RUNNER` to `ubuntu-latest`, and bump-mounted-pins stays on GitHub's runners. Each job that runs `gh`, `jq` or `yq` first fetches the ones the runner lacks (mikefarah's yq v4: Ubuntu's `yq` package is another program), pinned by version and sha256, into the job's temp folder, without sudo; that step runs only while `KM_RUNNER` is set. A runner image that has gh, jq and mikefarah's yq saves those downloads. Where the step fetches gh, that gh runs in jobs that hold a write `GITHUB_TOKEN` (automerge's act without the App, main-guard's issue), as a pinned action would. automerge's act job never fetches them where it merges with the App (`KM_AUTOMERGE_APP`): its token is not handed to a downloaded tool, so there the runner's image must have gh and jq, or nothing is merged and the run fails. The km action installs its Python with `actions/setup-python`, which needs a writable tool cache and network access to GitHub and PyPI. Every job has a `timeout-minutes` (15 at most), so a stuck job does not hold a runner for GitHub's default 6 hours.

Security: approve-fork-ci, review-request-guide and automerge run on `pull_request_target` and never check out the pull request's code, but ci does run on a fork's pull request. If GitHub passes the repository's variables to that run, a fork author's pull request runs on your hardware, in the network the runner sits in, and approve-fork-ci starts that run without a person when the pull request changes no control file (ci then runs km on the fork's docs, nothing from the checkout); otherwise fork ci stays on GitHub's runner. So use `KM_RUNNER` with ephemeral, isolated runners only: one job per fresh VM, with a network that gives the job no more than it needs. On a persistent or non-isolated runner, do not set `KM_RUNNER` while approve-fork-ci auto-approves.

Known limit: whether GitHub passes repository and organisation variables to a fork's or Dependabot's `pull_request` run is not verified yet; until it is, read the security note as if it does. `km doctor` names the runner `KM_RUNNER` sets, and notes it when `ci.yml` does not read it.

## Layout

```
Expand Down
2 changes: 1 addition & 1 deletion skills/km/reference/setup.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ A **team brain** is a shared, served knowledge base: one maintainer merges, the
2. In the empty repo (or an empty clone of a fresh remote): `km init . --team --name <name> --desc "<desc>" --initials <XX> --folder process="<purpose>" --folder ... [--rule "<rule>" ...]`. It writes the docs, indexes, `schema.local.yaml`, the ci, staleness, main-guard, approve-fork-ci, review-request-guide and bump-mounted-pins workflows (the last with its script `.github/scripts/bump-mounted-pins.py`), the pre-commit hook, a Dependabot config and a `.gitattributes` for LF, all pinned by commit to this km's release (or `--pin <version>`), and refuses to run if any file it would write already exists, such as a README the host created with the repo: delete that first.
3. `git add -A`, then `km gen-index` (it reads tracked files only, so add first; it fills the root and folder lists), `git add -A` and `km validate`: expect **0 errors / 0 warnings**.
4. Commit: `chore: initialize <name> (team served knowledge base)`.
5. Tell the user what km does not do: create the repo (private), push `main` (after `pre-commit install` the pre-push hook refuses a push to `main`: the first push of the fresh brain takes `--no-verify`, once; main-guard lets that first push into the empty repository through), give the team **read** access and the maintainer write, and ask members to add the brain to their personal brain with `/km brain add <url>`. Branch protection on a private repo needs a paid GitHub plan; without it the one-merger rule is held by the permission model alone. Members with read access contribute by fork PR, and GitHub does not run Actions on fork PRs of a private repo until "Run workflows from fork pull requests" is enabled (repository or organization settings, Actions): without it those PRs get no CI. With it, the approve-fork-ci workflow approves the waiting ci run of a fork PR that touches no control file (the rule is in the workflow's header); any other fork PR waits for "Approve and run". Dependabot then proposes a PR whenever a newer km or another action is released; mounted brains move through bump-mounted-pins ("Mounted pins on a team brain"). Set the repository variable `KM_MAINTAINERS` to the maintainers' GitHub logins: review-request-guide then comments on a PR whose author asks a maintainer for review before it is `km:ready`, with what `/km pr` and the sign-off still need.
5. Tell the user what km does not do: create the repo (private), push `main` (after `pre-commit install` the pre-push hook refuses a push to `main`: the first push of the fresh brain takes `--no-verify`, once; main-guard lets that first push into the empty repository through), give the team **read** access and the maintainer write, and ask members to add the brain to their personal brain with `/km brain add <url>`. Branch protection on a private repo needs a paid GitHub plan; without it the one-merger rule is held by the permission model alone. Members with read access contribute by fork PR, and GitHub does not run Actions on fork PRs of a private repo until "Run workflows from fork pull requests" is enabled (repository or organization settings, Actions): without it those PRs get no CI. With it, the approve-fork-ci workflow approves the waiting ci run of a fork PR that touches no control file (the rule is in the workflow's header); any other fork PR waits for "Approve and run". Dependabot then proposes a PR whenever a newer km or another action is released; mounted brains move through bump-mounted-pins ("Mounted pins on a team brain"). Set the repository variable `KM_MAINTAINERS` to the maintainers' GitHub logins: review-request-guide then comments on a PR whose author asks a maintainer for review before it is `km:ready`, with what `/km pr` and the sign-off still need. To run the team workflows on the team's own runners, set the repository or organization variable `KM_RUNNER` to a runner label; a fork's pull request then runs ci on that hardware, so only ephemeral runners fit (km's README, "Running on your own runners", lists what a runner needs).

## Mounted pins on a team brain (`bump-mounted-pins`)

Expand Down
36 changes: 36 additions & 0 deletions src/km/doctor.py
Original file line number Diff line number Diff line change
Expand Up @@ -552,6 +552,41 @@ def check_environment(repo: str, name: str, if_missing: tuple[str, str] | None,
f"Settings, Environments, {name}: deployment branches, selected branches, `{default}` only")


RUNS_ON_RUNNER = re.compile(r"(?m)^[ \t]*runs-on:[^#\n]*vars\.KM_RUNNER")


def check_runner(root: Path, repo: str) -> None:
"""The variable KM_RUNNER of the repository, else of its organisation, as far as gh may read them,
beside km's team ci.yml (one that installs km through pasrom/km@): a note naming the runner when the
runs-on lines of ci.yml and staleness.yml read it, or which of them do not. Unset or unreadable,
nothing is said; a repository variable gh may not read is not stood in for by the organisation's."""
files = [p for p in (root / ".github/workflows/ci.yml", root / ".github/workflows/staleness.yml") if p.is_file()]
texts = {p.name: p.read_text(encoding="utf-8", errors="replace") for p in files}
if "pasrom/km@" not in texts.get("ci.yml", ""):
return
for args in ((f"repos/{repo}/actions/variables/KM_RUNNER", "--jq", ".value"),
("--paginate", f"repos/{repo}/actions/organization-variables",
"--jq", '.variables[] | select(.name == "KM_RUNNER") | .value')):
if (res := ask(f"for the variable KM_RUNNER of {repo}", "api", "--hostname", "github.com", *args,
status="note")) is None:
return
if res[0] == 0 and (label := printable(res[1].strip())):
break
if res[0] != 0 and "Not Found" not in res[1]:
return # not readable (a 403): the organisation's would mislead
else:
return
if ignoring := [name for name, text in texts.items() if not RUNS_ON_RUNNER.search(text)]:
report("note", f"the variable KM_RUNNER names the runner {label}, but the runs-on line of {' and '.join(ignoring)} "
f"does not read it: that workflow stays where the line says",
"set their runs-on to `${{ vars.KM_RUNNER || 'ubuntu-latest' }}` (km's README, \"Running on your own runners\")")
else:
report("note", f"the team workflows run on the runner {label} (the variable KM_RUNNER; bump-mounted-pins reads "
f"MOUNTED_PINS_RUNNER first): if GitHub passes the brain's variables to a fork's pull request run, "
f"that ci runs on it too, otherwise on GitHub's runner",
"use ephemeral, isolated runners only, one job per fresh machine (km's README, \"Running on your own runners\")")


def main() -> int:
ap = argparse.ArgumentParser(prog="km doctor", description="check that this computer is set up to work on a brain")
ap.add_argument("--offline", action="store_true", help="skip the checks that need the network")
Expand All @@ -571,6 +606,7 @@ def main() -> int:
check_line_endings(root)
if repo := check_github(origin, args.offline):
check_environments(root, repo)
check_runner(root, repo)
check_git_login(root, origin)
check_km_version(root)
check_plugins(root)
Expand Down
36 changes: 35 additions & 1 deletion src/km/templates/team/approve-fork-ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -42,9 +42,43 @@ jobs:
if: >-
github.event.pull_request.head.repo.full_name != github.repository
&& github.event.pull_request.base.ref == github.event.repository.default_branch
runs-on: ubuntu-latest
runs-on: ${{ vars.KM_RUNNER || 'ubuntu-latest' }}
timeout-minutes: 10
steps:
# GitHub's runner image has gh, jq and yq; a self-hosted runner (KM_RUNNER) may not. A missing one is
# fetched, pinned by version and sha256, into this job's temp folder; the same step in every km workflow
- name: the tools this job runs, where the runner lacks them
if: vars.KM_RUNNER != ''
env:
TOOLS: gh jq
run: |
set -euo pipefail
bin="$RUNNER_TEMP/km-tools"; mkdir -p "$bin"; echo "$bin" >> "$GITHUB_PATH"
case "$(uname -s)-$(uname -m)" in Linux-x86_64) arch=amd64;; Linux-aarch64|Linux-arm64) arch=arm64;; *) arch=;; esac
for tool in $TOOLS; do
if [ "$tool" = yq ]; then # mikefarah's yq v4: Ubuntu's yq package is another program
yq --version 2>/dev/null | grep -q 'mikefarah/yq.* v4\.' && continue
else
command -v "$tool" >/dev/null && continue
fi
gh=https://github.com/cli/cli/releases/download/v2.103.3/gh_2.103.3_linux
jq=https://github.com/jqlang/jq/releases/download/jq-1.8.2/jq-linux
yq=https://github.com/mikefarah/yq/releases/download/v4.54.1/yq_linux
case "$tool-$arch" in
gh-amd64) url="${gh}_amd64.tar.gz" sum=4c806978ffce1cec70853302a56d70ac6da4f19bae7adca4e1a8ab00633dd91d;;
gh-arm64) url="${gh}_arm64.tar.gz" sum=ee32df2323c8847f5fac6cb6b4b69ca75828e9329a8f22f677021f7b3306ce63;;
jq-amd64) url="$jq-amd64" sum=b1c22172dd303f3be49e935aa56aa48a8b7a46e0bc838b4997d3bb451495870f;;
jq-arm64) url="$jq-arm64" sum=8b85c817833814ddca00a144c33705546355afccf0cf39b188f3cdb48b852309;;
yq-amd64) url="${yq}_amd64" sum=8e34fc298390875de416e6a4afcb8cabeceb25d9aa8506c1a2f9353cf702ea5f;;
yq-arm64) url="${yq}_arm64" sum=189088da0c6429ec5178dfaab1a114805f6cab0b61b165ab236efedf1d57a71b;;
*) echo "::error::$tool is missing and km pins no build of it for $(uname -s) $(uname -m): install it on the runner"; exit 1;;
esac
curl -fsSL --retry 3 -o "$bin/$tool.get" "$url"
echo "$sum $bin/$tool.get" | sha256sum -c --quiet -
if [ "$tool" = gh ]; then tar -xzOf "$bin/gh.get" --wildcards '*/bin/gh' > "$bin/gh"; rm "$bin/gh.get"
else mv "$bin/$tool.get" "$bin/$tool"; fi
chmod +x "$bin/$tool"; echo "installed $tool from $url"
done
- name: approve the CI run when the pull request changes nothing CI executes
env:
GH_TOKEN: ${{ github.token }}
Expand Down
Loading
Loading