Repository navigation
Conversation
|
Preview deployment for your docs. Learn more about Mintlify Previews.
💡 Tip: Enable Automations to automatically generate PRs for you. |
DRU-1044: overview, decisions, tests, and open questionsAll results are from OverviewThe problem. A build could hand a person code that nobody had tested:
So a PR could look ready to merge while its CI was failing. The solution. The build now checks the work before anyone is asked to look at it:
The approach. The build learns about CI the way it learns about the review. GitHub's What changed, and why
Two things to know:
TestsAutomated, on
Live, and why. The change sits between GitHub and a durable run: webhooks, a wait that has to survive restarts, and CI that finishes on its own schedule. No unit test spans that, so each case ran a real build on a local install against The test repository's CI has these jobs:
Its
DSE-66 stopped with Not run live, and why:
Pending Questions
|
czpython
left a comment
There was a problem hiding this comment.
Requesting changes on the direction, not on the work. This follows DRU-1044 closely, and the live runs are thorough. The changes come from rethinking the ticket.
The new shape: the implementer pushes and its call ends. The build waits on ci.updated with the VM released, wakes on a webhook or the 60-second timer, and reads only whether CI on the head has finished and whether it is green. When CI is red, the implementer reads the failures itself.
implement lint, typecheck, tests for its diff, commit, push
|
v
head changed? -- no -----------------------> review it pushed nothing
| yes
v
read CI <-----------------+
| |
+-- running --> wait -+ a webhook or the 60 s timer wakes it;
| the VM is released, and no agent runs
+-- green -----------------------------> review
+-- nothing registered after 3 min ----> review
+-- still running after 1 h -----------> review the park label says so
+-- red -+-- revisions left -----------> implement "CI on <sha> is red"
+-- none left ----------------> review
review
|
+-- request changes, revisions left --> implement
+-- approve, or none left ------------> park: the PR leaves draft only if
CI is green and the review after it approved
-
The implementer reads CI; the build only waits. The build reads two facts from GitHub: whether CI on the head has finished, so it knows when to wake the implementer, and whether it is green, for the draft rule. Remove
Check,CheckFailure, the per-check rules inCiReport.judge,get_job_log,list_check_runs,CI_LOG_LINES, and the default-branch read. On red, the revision request is one line: "CI on<sha>is red. Read it withgh pr checksandgh run view --log-failed, fix what your diff caused, and report the rest inchecks."ghin the box acts as the operator App, which can read Actions logs. The box's GitHub MCP server loads only its default toolsets, which leave outactions. -
Read "finished" from check suites, not check runs. Actions creates the check run of a
needs:job only when the jobs it needs finish. On astral-sh/uv run 42286852249, 67 jobs appeared in 9 waves over about 8 minutes. The webhook that wakes the read is an upstream job's completion, so "every check run finished" can be true between waves. Actions opens one check suite per workflow run, and the suite stays open until the run completes.- Finished: every suite with check runs is
completed, and no commit status ispending. - A suite with no check runs counts as running for 3 minutes after the push, then is ignored. Some apps leave a
queuedsuite with no runs for good. Theclaudeapp has one on this PR's head. - Green: every suite is
success,neutral, orskipped, and every status issuccess. - githubkit has
checks.async_list_suites_for_ref.
- Finished: every suite with check runs is
-
Drop the red-on-main rule. It matches check names, and on this repo it fails both ways. The test workflows run only on
pull_request, so main never has their checks. On main,backendis the release-image build, not the tests. Instead, the implementer decides. When a red check is not from its diff, it pushes nothing and reports the check incheckswith the reason. In_implement_phase, a delivery whosehead_shaequals the previous delivery's skips the CI wait and goes to review. The PR stays a draft, and the park label says CI is red. The implement prompt must allow that round with no push. -
Release the VM during the CI wait. Add
release_sandbox: bool = FalsetoWorkflow.wait, and passTruefrom the CI wait. It calls_reap_run(), as a gate park does.releaserevokes the box identity before it deletes the VM, so the next implement call boots a new VM, andRepoWorkspacere-clones before every call anyway. Update thewaitsection ofdocs/writing-an-app.md. -
Drop
pre_push_tests. A fixed list is the same for every diff, so in a large repo it is either too slow or it tests the wrong area. Keep the prompt change: before every push, run lint, typecheck, and the tests narrowed to the files you changed. Remove the key fromdocs/configuration.md. -
Narrow
_approved_work(your question 1). Clear the draft before a merge only on a person's approval. Leaveimplementation_approval: noneto DRU-614. As it is now, withnoneandon_approval: merge,merge_when_readysquash-merges a capped red round on a repo without required checks.
Keep as they are: the check_run and status webhooks, ci.updated and its subscriber, the 60-second timer, _now and _read_ci as steps, the 3-minute and 1-hour limits, the revision cap, is_ready_for_review, and the park label.
Tests: drop the Check.get_log and red-on-default tests. Add one for the suite read, with a needs: wave and a suite with no runs, and one for the unchanged head.
A build reviewed and offered a pull request whose CI had not finished or was red, and
ReviewWork.on_waittook the PR out of draft whatever the round's outcome. Now the implementer runs the repo's tests before every push, and the build waits for CI before the review.config.ymltakes a top-levelpre_push_tests. The implement prompt asks for those tests before every push, or, without the key, for the profile's test commands narrowed to the changed files, and for each result inchecks. The key is not underverification, because a pinnedverificationreplaces every command the profiler found.check_runandstatuswebhooks publishci.updated, which wakes the wait. Each wake, and a poll every minute, reads the checks from GitHub, so a check that finished before the wait began is not missed.success,neutral, orskippedon every check. A red check that is also red on the default branch blocks the round instead. With no check registered three minutes after the push, the repo has no CI, and the round is green. CI still running after an hour blocks the round, and the work park's label gives the reason.max_implementation_revisions. The revision request names each failing check and carries the last 200 lines of its log: the Actions job log, or the check's own output for any other check.A person's approval at the park takes the PR out of draft, whatever CI says, because GitHub does not merge a draft and
on_approval: mergewould otherwise park again.The CI wait adds steps after
implement, so a build that is past its first delivery when this deploys fails on replay and needs a new start.