Repository navigation
Keep an approval when the pushes after it only merge the base branch - #17037
codeconsole wants to merge 14 commits into
Conversation
GitHub moves an approval to the new head after a merge of the base branch that leaves the diff of the pull request as it was, but not after one that had conflicts, since resolving them changes that diff. The workflow then dismissed the approval, although the pull request got no change to review. The workflow now repeats each merge between the approved commit and the head with git merge-tree, in a bare repository holding only the trees of the merge's parents and of their merge base. The approval is kept when every commit on the way has two parents, the second one a commit of the base branch, and differs from git's own merge only in the conflicted files. A merge that changes another file by hand, or that merges another branch, dismisses the approval as before.
A fetch or a merge that hangs would end the job at its timeout, and with it the dismissals of every pull request listed after that one, on each run. Each git command now gets two minutes, after which the approval is dismissed like any other the workflow cannot verify.
jdaugherty
left a comment
There was a problem hiding this comment.
Thanks for this. Update merges with conflicts do dismiss approvals that had nothing new to review, and the approach of repeating the merge without checking anything out is sound: a failed request, a commit with one parent or more than two, a second parent that is not on the base branch, a timeout and too many merges all end in a dismissal.
One problem needs to be solved before this goes in, see the comment on the comparison of the changed files: a conflicted file is accepted with any content, which lets a change nobody reviewed keep an approval. The other two comments are small.
jamesfredley
left a comment
There was a problem hiding this comment.
Requested changes. A push of only base merges still dismisses the approval once there are more than twenty of them.
A merge of the base branch kept an approval when the files it changed by hand were among the conflicted ones, whatever it changed in them, so a merge could add an unreviewed line anywhere in a conflicted file. The merged file git writes holds each conflict between markers labelled with the two parents; the pushed file now has to keep every line outside them, and its mode, as git wrote them. A conflict without markers, of a binary file or of a file deleted on one side, dismisses the approval when it was resolved by hand. A kept approval and the files whose conflicts were resolved now go to the step summary, and the pull request gets one comment per merge naming them, found again by a hidden marker so that the hourly run does not repeat it. The walk over the merges states why it gave up: more than 20 commits, or a request or git command that failed or ran out of the two minutes the walk now has as a whole, is dismissed as not checked rather than as a change of the pull request. Its answer is reused within a run for every approval given to the same commit.
AGENTS.md said that the workflow compares no diff and then described a rule that repeats merges and compares them. The update-merge rule now comes first, and "compares no diff" is said of any other push. Both guides describe the narrower rule: only the lines inside the conflicts may change, and the pull request is told which files were resolved.
jamesfredley
left a comment
There was a problem hiding this comment.
Requested changes. A push of only base merges still dismisses the approval once there are more than twenty of them.
Every fetch of a merge adds a pack to merges.git, and after about fifty of them, across all the pull requests of one run, git starts gc --auto in the background. It holds the lock of the shallow commits, so the next fetch fails and the approval it was checking is dismissed as one whose commits could not be checked. A synthetic run over 61 merges hit it.
The walk stopped after 20 commits and dismissed the approval, although running out of commits to check is no sign of a change. It now goes back to the approved commit, bounded by time only: two minutes for one approval, as before, and seven for all the checks of a run, so that one pull request with a long run of merges, or several slow ones, cannot end the job at its ten-minute timeout before the dismissals after them.
Every merge after the head was fetched as the first parent of the merge before it, so its parents are in its commit object. Reading them there saves one of the three requests a merge cost, which matters now that the number of merges is bounded by time only.
jdaugherty
left a comment
There was a problem hiding this comment.
Thanks for the quick turnaround. The three points from my earlier review are addressed. I replayed the check at 65f9987 against #16877, #16726, #16871 and #16912 and got the results in the description, and ran resolved_within_conflicts against a scratch repository: an edit between two conflicts, an append at the end of the file, a deleted line next to a conflict, a mode change, a change of line endings and a dropped final newline all dismiss, while a resolution inside the conflict keeps the approval, also one that repeats the lines around it.
Two comments are left. The first is a small bug in the reason given for a dismissal, the second is not blocking.
| # The commit was fetched as the first parent of the merge before it, so its parents are read from it. | ||
| read -r first second other < <(bounded env GIT_NO_LAZY_FETCH=1 git -C merges.git cat-file commit "$commit" | sed -n '/^$/q; s/^parent //p' | tr '\n' ' ') | ||
| else | ||
| read -r first second other < <(bounded gh api "repos/$REPO/commits/$commit" --jq '[.parents[].sha] | join(" ")') |
There was a problem hiding this comment.
A failed request here is reported as a change of the pull request. The read takes its input from a process substitution, so the exit status of gh api is lost, and gh api --jq prints the error body on a failed request. For a 422 I get first=[{"message":"No], second=[commit] and the rest of the message in other, so the next line sets "the pull request changed after this approval" and the approval is dismissed with that message, not with "could not be checked". A rate limit or a server error with a JSON body does the same.
The two requests below check their exit status and are fine. Could this one do the same, for example by reading the output into a variable with || return 1 first and splitting it after?
Line 97 has the same mislabel in a smaller window: the step runs without pipefail, so when the deadline passes before git cat-file, awk reads nothing, regions is empty, and the answer is 1, a change, where it should be 2.
There was a problem hiding this comment.
Right on both, thanks for tracing them. Fixed in 58b1f38. The parents are now read into a variable with || return 1 before they are split, both for the head and for the later merges read with cat-file. The cat-file | awk pipeline runs with pipefail in its own subshell, so a timeout or a git failure there answers 2. Line 94 had the same problem: the mode of the pushed file came from a substitution inside the test, so its status was dropped and a timeout there also read as a change. It now fails like line 92. In the scratch repository, a 422 with a JSON body for the head's parents, and a cat-file or ls-tree that runs out of time, all dismiss with "the commits after this approval could not be checked". The earlier cases and the historical merges keep their verdicts.
| local base=$1 approved=$2 commit=$3 deadline=$(( SECONDS + 120 )) fork= first second other behind merged tree conflicts changed path | ||
| (( deadline <= checks_end )) || deadline=$checks_end | ||
| accepted= | ||
| reason="the commits after this approval could not be checked" |
There was a problem hiding this comment.
Not blocking, and fine as a follow-up. GitHub never moves an approval kept by this rule, so it has to be proved again on every hourly run, and anything that keeps one of those runs from finishing dismisses it: one failed request, or the seven minutes of the run being used up by the pull requests before it in the list. An approval that was kept for days can then be lost in an hour in which nothing was pushed, and the cost of one pull request with a long run of merges falls on the others. At today's numbers the seven minutes are far away (a merge took about 7 seconds when I replayed #16877 and #16726 from here).
The comment this workflow already writes could carry the answer: a hidden marker with the approved commit and the head it was kept for, read from the bot's comments like the merge markers. A later run that finds the marker for the same pair has nothing to walk, which ends both the hourly cost and the chance of losing the approval to a failed request.
There was a problem hiding this comment.
The problem holds. The part I could fix without stored state is in b467a46: a failed request of the check is asked once more after five seconds, within the check's deadline, so one passing API error no longer dismisses an approval that was kept for days. In the scratch repository, a 502 once for the head's parents or for a compare keeps the approval; a request that keeps failing still dismisses it as "could not be checked".
I haven't added the marker, because a marker in the bot's comments would decide whether an approval is kept, and a pull request can write those comments. vulnerability-scan.yml runs a labelled same-repo pull request's build, and its own postStickyComment.sh, with pull-requests: write, and a same-repo branch can add a workflow that comments as the bot. Today a forged kept-resolution marker only hides a notice; a forged "kept for A..H" marker would keep an approval over code nobody reviewed. The answer also depends on the base branch, so the key would need that too.
Saving the answer needs a place a pull request can't write to. The simplest is the cache of a default-branch run, but that means using actions/cache, and #17019 promised the workflow uses no action. Since that promise is yours, would you rather keep it, or have the answer cached? The shared seven minutes go away with it.
Three places lost the exit status of the command they read, so a failure there dismissed the approval as a change of the pull request instead of as commits that could not be checked: - The parents of a merge were read from a process substitution. gh prints the body of a failed request, and a 422 split into words looked like a commit with more than two parents. They are now read into a variable, with the status checked, before they are split. - The mode of the pushed file came from a substitution inside a test, so a git command that failed or ran out of time read as another mode. - The step runs without pipefail, so when cat-file failed or ran out of time awk read nothing and the file looked like one without conflicts. That pipeline, and the one reading the parents of a later merge, now set pipefail in their own subshell.
GitHub never moves an approval kept across update merges, so every hourly run checks it again, and one passing error of the API in any of those runs dismissed it although nothing was pushed. A failed request is now asked once more after five seconds, as long as the check has that time left. Only the answer of the request that succeeds is used, never the error body gh prints for the failed one.
jamesfredley
left a comment
There was a problem hiding this comment.
Requested changes. Twenty-one merges of the base no longer dismiss the approval. A line added just before a conflict marker still keeps it.
| /^@@ / { | ||
| split(substr($2, 2), h, ","); at = h[1] + 0; count = (2 in h) ? h[2] + 0 : 1; inside = 0 | ||
| for (i = 1; i < n; i += 2) | ||
| if (count > 0 ? (r[i] <= at && at + count - 1 <= r[i + 1]) : (r[i] - 1 <= at && at <= r[i + 1])) inside = 1 |
There was a problem hiding this comment.
A later push commits a file that still has its conflict markers, and adds a line immediately before the opening marker. This treats that line as part of the conflict and keeps the approval. A line outside the conflict is a change. Dismiss the approval.
There was a problem hiding this comment.
Right. For a line that was only inserted, the check also accepted the spot just before the opening marker and just after the closing one, so a file pushed with its markers still in place and a line added just outside them kept the approval. Fixed in c2b3345: an inserted line now has to land between the markers of a conflict. In the scratch repository your case, and the same with a line just after the closing marker, now dismiss; a line added between the markers still keeps the approval. Resolutions that remove the markers aren't affected, since git reports them as a change of the conflict's own lines. That holds even when they repeat the lines around the conflict, which I checked with four such resolutions. The earlier cases, the failure cases and the 13 historical merges keep their verdicts. One limit stays by design: once the markers are gone, a line written where a marker was is part of the resolution and can't be told apart from it. That is why the pull request gets a comment naming the files whose conflicts were resolved.
A hunk that only inserts lines passed when it sat next to a conflict, just before the opening marker or just after the closing one. A merge pushed with git's conflict markers still in place, and a line added just outside them, therefore kept the approval, although that line is outside the conflict. An inserted line now has to land between the markers. A resolution that removes the markers is not affected: git reports it as a change of the conflict's own lines, also when it repeats the lines around the conflict, so those hunks already had to lie within the markers.
jamesfredley
left a comment
There was a problem hiding this comment.
Requested changes. A line added just before a conflict marker still keeps the old approval.
| BEGIN { n = split(regions, r, " ") } | ||
| /^@@ / { | ||
| split(substr($2, 2), h, ","); at = h[1] + 0; count = (2 in h) ? h[2] + 0 : 1; inside = 0 | ||
| for (i = 1; i < n; i += 2) |
There was a problem hiding this comment.
A later push adds a line immediately before the opening conflict marker, then keeps one side of the conflict. This treats the added line as part of the conflict and keeps the approval. A line outside the conflict is a change. Dismiss the approval.
There was a problem hiding this comment.
You're right, and my last reply called it a limit when it isn't one. Fixed in 9b4d48c: every line a merge writes inside a conflict now has to be a line of one of the conflict's two sides. Which lines it keeps, and in which order, is still up to the resolution. Your case, a line added before the opening marker and then one side kept, now dismisses. So do the same line after the side that is kept, a resolution written by hand, and a copy of a line from around the conflict. Keeping one side, both sides, or both in another order still keeps the approval. This rule costs something real: a line written by hand in a resolution is now a change like any other. Of the seven historical merges that kept their approval, four are now dismissed: #16877 and #16453 renumber a heading of the upgrade guide, #16726 merges a sentence of CONTRIBUTING.md by hand, and #16495 merges Java code by hand. #16958, #16838 and #16552 are still kept. CONTRIBUTING.md now says so.
A resolution could write any line between the markers of a conflict, so a line added just before the opening marker, followed by one side of the conflict, kept the approval: once the markers are gone, that line is part of the resolution and git cannot tell it from one. Every line a merge writes inside a conflict now has to be a line of one of its two sides; which lines it keeps, and in which order, is still up to the resolution. A line written by hand inside a conflict now dismisses the approval like any other change. Of the seven historical merges kept before, the ones of apache#16453, apache#16495, apache#16726 and apache#16877 are now dismissed: they renumber a heading of the upgrade guide, or merge a line of prose or code by hand.
jamesfredley
left a comment
There was a problem hiding this comment.
Requested changes. An extra line that matches one side of a conflict still keeps the old approval.
| FNR == NR { merged[FNR] = $0; next } | ||
| /^@@ / { | ||
| split(substr($2, 2), h, ","); at = h[1] + 0; count = (2 in h) ? h[2] + 0 : 1; conflict = 0 | ||
| for (i = 1; i < n; i += 2) |
There was a problem hiding this comment.
A later push adds a second increment immediately before the opening conflict marker, then keeps the original increment. The file increments twice. This treats the extra line as part of the conflict and keeps the approval. A line outside the conflict is a change. Dismiss the approval.
There was a problem hiding this comment.
Right again, and it showed that checking each line against the sides can't be enough: a line of a side can be repeated, as in your case, or dropped, reordered, or combined with lines of the other side. Fixed in 0b57e0b: each conflict now has to be resolved as a whole, the way git offers to. That means the pull request's side, the base branch's side, or both one after the other, in either order. Your case, the increment made twice, now dismisses. So do part of a side dropped, a side's lines reordered, and a line of a side added between markers that were kept. One side, the other, or both still keep the approval, also when two conflicts in one file are resolved differently. The rule costs more now. Of the seven historical merges that kept their approval before last round, only #16958 and #16552 still do. #16838 is now dismissed as well, since it kept the method signature of one side and the body of the other. A resolution git couldn't have made by itself needs a new review.
Checking each line of a resolution against the lines of the two sides still let a resolution change the code: a line of a side could be repeated, so an increment added before the conflict and the side's own increment made the file increment twice, and lines could be dropped, reordered or combined from both sides. Each conflict is now resolved as a whole, as git offers to: the pull request's side, the base branch's side, or both one after the other, in either order. A conflict left as git wrote it, markers and all, is no change either. Of the historical merges, apache#16838 is now dismissed too: it kept the method signature of one side and the body of the other. apache#16958 and apache#16552 keep their approvals.
🔎 No tests executed 🔎🏷️ Commit: 0b57e0b Learn more about TestLens at testlens.app/docs. |
jamesfredley
left a comment
There was a problem hiding this comment.
Requested changes. A merge with more than one common ancestor can keep an approval for a change outside the conflict.
| [[ -n "$second" && -z "$other" ]] || { reason="the pull request changed after this approval"; return 1; } | ||
| behind=$(requested "repos/$REPO/compare/$second...$base?per_page=1" --jq '.behind_by') || return 1 | ||
| [[ "$behind" == 0 ]] || { reason="the pull request changed after this approval"; return 1; } | ||
| fork=$(requested "repos/$REPO/compare/$first...$second?per_page=1" --jq '.merge_base_commit.sha') || return 1 |
There was a problem hiding this comment.
The two sides have more than one common ancestor. An extra edit outside the conflict is accepted, and a resolution that keeps one whole side is rejected. Use the same base a normal merge would, and dismiss the approval for the extra edit.
Follow-up to #17019.
Why
dismiss-stale-reviews.ymlrelies on GitHub moving an approval to the new head when a push leaves the diff of the pull request as it was. GitHub does that for a clean merge of the base branch, but not for one that had conflicts: resolving them changes the diff. Such an update merge would dismiss the approval, although the pull request got no change to review, only the base branch's.This is common here. Among the pull requests updated recently, 12 approvals on 9 pull requests (#16453, #16495, #16552, #16726, #16838, #16871, #16877, #16912, #16958) were left at an earlier commit by pushes that contained nothing but merges of the base branch. Every one of those merges had conflicts, typically in
dependencies.gradleor in the numbered sections of the upgrade guide. The approvals kept by this pull request are those whose conflicts were resolved by keeping whole sides, as on #16552 and #16958; the other merges wrote resolutions of their own, which still need a new review.What changes
An approval that GitHub has not moved is now also kept when only update merges separate it from the head. Every commit after the approved one has to
So a resolution git could have made by itself is accepted, and nothing else is. Any other resolution is a change like any other: a renumbered heading, lines picked from both sides, a line repeated, dropped, reordered or written by hand. Once the markers are gone, a resolution like that can't be told from a change next to the conflict. A merge that resolves a conflict that way, that changes any other line, that resolves by hand a conflict git writes without markers (a binary file, a file deleted on one side), that merges another branch, or that follows an ordinary commit pushed after the approval dismisses it as before. So does a check that cannot finish, because a request still fails when asked a second time, a git command fails, or the check runs out of time. Such a dismissal says the commits could not be checked, not that the pull request changed.
Nobody has reviewed the resolution, so a kept approval goes to the step summary with the files whose conflicts were resolved, and the pull request gets one comment per merge naming them.
How
For each merge the workflow asks the API whether the second parent is on the base branch and for the merge base of the two parents. It fetches the merge, its parents and their merge base into a bare repository without history and without file contents (
--depth=1 --filter=blob:none), and repeats the merge withgit merge-tree --write-tree, which prints the merged tree and the conflicted files. Only the head's parents come from the API; every later merge was already fetched as the first parent of the one before it, so its parents are read from its commit. A changed conflicted file is compared with the file git wrote, conflict markers included: the markers are matched with their labels, the full ids of the two parents, every changed line has to lie inside a conflict, and what replaces each conflict has to be one side, the other, or both.The number of merges is not limited, only the time: each approval's check gets two minutes, and all the checks of a run together get seven, so the job always reaches the dismissals after them. Within a run the answer is reused for every approval given to the same commit. The comment on the pull request carries a hidden marker with the merge, read only from this workflow's bot, so the hourly run does not repeat it. Auto-gc is off in the bare repository, since a background gc after many fetches would hold the lock the next fetch needs.
Nothing of the pull request is checked out or run:
git merge-treeworks on objects and runs no hook and no filter. The token keepscontents: readandpull-requests: write.An approval kept this way stays at its commit, so the hourly run checks it again: two requests and one small fetch per merge, about four seconds a merge when measured from a workstation.
Testing
The check was run against earlier pushes of this repository:
In a scratch repository, with a stand-in for
gh: a resolution that keeps one side, the other, or both in either order keeps the approval, also with two conflicts in a file resolved differently, and so does a file pushed with its conflict markers as git wrote them; 21 and 61 merges of the base branch after it keep it too. A line written by hand in a resolution, one added before or after the side it keeps, a line of a side repeated (an increment made twice), part of a side dropped, a side's lines reordered, a line of a side added between kept markers, a copy of a line from around the conflict, an edit far from the conflict, an edit after a line that only looks like a conflict marker, a line added just before the opening marker or just after the closing one of a conflict pushed with its markers, a mode change, a deleted file, a hand-resolved binary conflict, and an ordinary commit followed by merges all dismiss it. A second approval at the same commit costs no request. A request that fails once is asked again and the approval is kept. A request that keeps failing with an error body, as a 422 for an unknown commit does, a git command that runs out of time while reading a conflicted file or a merge's parents, and a run whose time is used up all dismiss as "could not be checked", not as a change.The step's script was also run as it is in the workflow file, in dry-run mode, against this repository. Earlier it kept the approval on #16958 and would have dismissed 19 others, each followed by ordinary commits. On the last run no open pull request had an approval at another commit than its head, so there was nothing to judge.
As with #17019, the workflow itself only runs from the default branch, so it could not be run from this pull request.
AGENTS.mdandCONTRIBUTING.mddescribe the rule; the latter says that a resolution keeps the approval only when it keeps one side, the other, or both, and that any other resolution, like any other change, dismisses it.