Skip to content

ci: run no CI on draft pull requests - #2130

Merged
kanat merged 3 commits into
mainfrom
ci/skip-draft-prs
Nov 20, 2026
Merged

kanat merged 3 commits into
mainfrom
ci/skip-draft-prs

Conversation

@kanat

@kanat kanat commented Nov 19, 2026 •

Copy link
Copy Markdown
Collaborator

Why

Every push to a PR starts CI, and a push to a draft does too. Agents push to drafts many times during review: the author pushes, fixers push, and the base gets merged in. On 2026-11-19, 43 PR branches started 128 CI (unit) runs (median 3 per branch, max 8), and 834 of the 2,925 jobs that day ended cancelled. Bursts of these runs filled the org-wide limit of 60 running jobs (Slack thread in #video-ai, follow-up to #2126).

How it works

push to a PR
├─ PR is a draft        -> each workflow starts, its jobs are skipped (no runner)
└─ PR is ready
   ├─ ready_for_review  -> CI runs once on the head
   └─ each later push   -> CI runs again (cancel-in-progress drops the old run)
push to main            -> unchanged: draft != true holds on a push
Workflow Job given if: github.event.pull_request.draft != true
CI (Go) go
CI (Python) unit (calls run_tests.yml)
CI (SDKs) sdks
CI (client SDKs) changes. kotlin, dart and swift need it, so they skip with it
Integration (router) integration (all 6 groups)
Python compatibility check resolve

Each workflow's pull_request gets types: [opened, synchronize, reopened, ready_for_review]. Without ready_for_review in the list, marking a draft ready would start nothing. The path filters stay as they are: on a pull request they are matched against the whole PR diff, so the ready event starts the workflows the PR touches.

No status check is required on main (the two rulesets require 1 approval and resolved threads, nothing else), so a skipped job blocks no merge.

Changes

  • The six PR workflows above: the types line and the if on their top job.
  • The pr skill: a draft runs no CI workflow (CodeQL, the labeler and CodeRabbit still run), so gh pr ready once the branch is done.
  • The parallel-agents skill: Integration runs once the PR is ready, a skipping check on a draft is not a pass, and push → wait for the push's runs → gh pr ready. Both events share one concurrency group per PR, so a draft run created after the ready run would cancel it.

What it does not change

  • CodeQL (3 jobs) and the labeler (1) still run on drafts. CodeQL is the repo's default setup, a setting, not a workflow file.
  • Publish Kotlin SDK is left as it is: its only PR job is skipped unless the branch is release-kotlin-*.

Validation

  • actionlint on the six files: no finding on a changed line.
  • This PR tested itself, since its diff touches all six workflows (comment):
Event (head 87da79d794) CI (Go) CI (Python) CI (SDKs) CI (client SDKs) Integration (router) Python compatibility
opened as a draft skipped skipped skipped skipped skipped skipped
marked ready success success success swift failed (base) success, 6/6 success
  • The swift failure (Build swift_demo, stale StreamWebRTC module) was on main too: push runs 44595177842 and 44615508387. main passed again at ebc1995df7, which is merged into this branch.
  • Review: round 1 NO-GO (R1.1 the parallel-agents skill still described CI on drafts, R1.2 the pr skill line), both fixed in 8e4508ace5, delta GO.

🤖 Generated with Claude Code

Every pull_request workflow now lists ready_for_review in its types and
skips its jobs while the pull request is a draft. CI starts when a pull
request is marked ready and runs on every push after that.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kanat
kanat marked this pull request as ready for review November 19, 2026 17:00
@coderabbitai

coderabbitai Bot commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

Explain Change →

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 52 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 8 included reviews currently available.

Learn how review limits work.

Review configuration:

Run configuration

Commits

Reviewing files that changed from the base of the PR and between cbf9dce and 8e4508a.

Files selected for processing: 2

  • .claude/skills/parallel-agents/SKILL.md
  • .claude/skills/pr/SKILL.md
Walkthrough

Six GitHub Actions workflows now listen for ready_for_review events and skip specified jobs when a pull request is a draft. The PR skill documents that draft pull requests run only CodeQL and the labeler, and that marking a pull request ready starts CI.

Merge Risk: Low · previous review 87da7

Draft PRs will skip CI as intended and run it once marked ready. The skill note slightly overpromises, since PRs that match no path filter still run no CI. This is a minor documentation fix and not a merge blocker.

- [ ] Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Review info

Run configuration

Commits

Reviewing files that changed from the base of the PR and between 58b951a and 87da79d.

Files selected for processing: 7

  • .claude/skills/pr/SKILL.md
  • .github/workflows/ci-go.yml
  • .github/workflows/ci-python.yml
  • .github/workflows/ci-sdks.yml
  • .github/workflows/client-sdks.yml
  • .github/workflows/integration.yml
  • .github/workflows/python-versions.yml

Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 4 remain after this review.

Comment thread .claude/skills/pr/SKILL.md Outdated
## Creating

- Always `gh pr create --draft`. Push the branch first.
- A draft runs no CI, only CodeQL and the labeler. `gh pr ready` starts CI, and every push after it runs CI again, so mark it ready once the branch is done, not before.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor · Functional Correctness · Quick win

Qualify when marking a PR ready starts CI.

The six workflows retain path filters. A PR that changes only this skill file does not match any of them, so gh pr ready starts none of those workflows. State that ready events and later pushes run the workflows only when their filters match. (docs.github.com)

@kanat

kanat commented Nov 20, 2026

Copy link
Copy Markdown
Collaborator Author

Validation on this PR (head 87da79d794)

Its diff touches all six workflows, so each one ran on both events.

Event Time (UTC) CI (Go) CI (Python) CI (SDKs) CI (client SDKs) Integration (router) Python compatibility check
opened as a draft 16:59:14 skipped skipped skipped skipped skipped skipped
marked ready 17:00:31 success success success failure, swift only success, 6/6 groups success
  • Only CodeQL and the labeler ran on the draft, as expected.
  • CI (client SDKs): changes, kotlin and dart passed. swift failed at Build swift_demo with error: file '…/StreamWebRTC.framework/Headers/RTCPeerConnection.h' has been modified since the module file. That is a base failure: the main push runs at ccc66d50aa (run 44595177842) and 9fc96ff9de (run 44615508387) fail the same step with the same error.

Source: gh api "repos/GetStream/Agents/actions/runs?head_sha=87da79d794ac2d8f9ea1ff379e3e548d6fa0e2b6", plus each run's /jobs.

@kanat kanat left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed at cbf9dce6f7 (87da79d79 plus a clean merge of main 36d5e346f; the PR diff is the same 7 files). CI on cbf9dce6f7 was still queued. On 87da79d79 only swift failed, at Build swift_demo with the RTCPeerConnection.h has been modified since the module file error. main push runs 44595177842 and 44615508387 fail at the same step with the same error, so this failure comes from main.

[Should fix] R1.1 .claude/skills/parallel-agents/SKILL.md:70-79 and :106-108 describe the old CI behavior. They say Integration (router) runs "on a PR that changes acceleration/**", tell agents to "read CI's Integration (router) check" before the final review, and say "A check that did not run leaves nothing to wait for". With this PR, a draft runs no CI, and the pr skill keeps a PR a draft until the branch is done. So the final review of a draft sees only skipping checks, and that text reads them as nothing to wait for. Fix: say that CI runs only on a ready PR (after gh pr ready), that a skipping check on a draft is not a pass, and that the integration result is read after the PR is marked ready. Found by git grep -n CI .claude/skills/parallel-agents/SKILL.md.

[Nit] R1.2: see the inline comment on .claude/skills/pr/SKILL.md:18.

CodeRabbit's open thread on .claude/skills/pr/SKILL.md:18 (path filters) is correct and not fixed at this head. I count it as part of R1.2.

Checked and sound:

  • Push to main. A missing property evaluates to '' (contexts), and loose equality turns '' into 0 and true into 1 (expressions), so draft != true holds on a push. Live check: renovatebot/renovate push run 44688929422 ran a job gated only on github.repository == ... && github.event.pull_request.draft != true.
  • client-sdks. On the draft (run 44572190330), changes, kotlin, dart and swift were all skipped. Their if gets the implicit success(), so a skipped needs skips them. On a push, changes runs as before.
  • unit in ci-python. jobs.<id>.if is a supported keyword for a job that calls a reusable workflow (reusing workflows). On the draft, unit was skipped (run 44572190309).
  • Path filters on the ready event. ready_for_review came at 17:00:27Z with no new commit, and all six workflows started at 17:00:31 on the same sha. So the filters were matched against the whole PR diff.
  • Concurrency. The groups are keyed on the PR branch or ref. A draft push cancels a running run only when the PR was converted back to draft. converted_to_draft is not in types, so converting a PR does not cancel its running CI.
  • publish-sdk-kotlin. It needs no change. On a PR, version runs only for a release-kotlin-* head, publish requires push, and upload needs publish. labeler is pull_request_target.
  • Required checks. The rulesets require none: main has deletion, non_fast_forward and pull_request (1 approval, resolved threads), and classic protection returns 404.
  • actionlint. No finding on a changed line.

Verdict: request changes, for R1.1.

Comment thread .claude/skills/pr/SKILL.md Outdated
## Creating

- Always `gh pr create --draft`. Push the branch first.
- A draft runs no CI, only CodeQL and the labeler. `gh pr ready` starts CI, and every push after it runs CI again, so mark it ready once the branch is done, not before.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Nit] R1.2 "A draft runs no CI, only CodeQL and the labeler" leaves out CodeRabbit, which reviews drafts (.coderabbit.yaml:45 sets drafts: true). "gh pr ready starts CI" also holds only for the workflows whose paths match the PR, which is CodeRabbit's thread above. Suggested text: "A draft runs no CI workflow; CodeQL, the labeler and CodeRabbit still run. gh pr ready starts the workflows whose paths the PR touches …"

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@kanat kanat left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No findings at 8e4508ace5 (delta cbf9dce6f7..8e4508ace5, docs only).

  • R1.1 fixed. .claude/skills/parallel-agents/SKILL.md:70-83 now says: Integration (router) runs once the PR is ready; gh pr ready starts it; CI is read after the PR is ready; a skipping check on a draft is not a pass. It also says to push, wait until the push's runs are listed, then gh pr ready. The wait always ends, because the labeler (pull_request_target) gets a run on every push. The reason the text gives is correct: all six concurrency groups are keyed per PR and set cancel-in-progress: true. :110-112 now marks the PR ready before Integration (router) is read.
  • R1.2 fixed. .claude/skills/pr/SKILL.md:18 names CodeRabbit (.coderabbit.yaml sets drafts: true). It also says that gh pr ready starts only the workflows whose paths the PR touches, which answers CodeRabbit's thread.
  • No other change. The delta touches no workflow file.

CI on 8e4508ace5 was still pending when I reviewed.

Verdict: approve. This is a COMMENT review only because GitHub does not let an author approve their own PR.

@kanat
kanat merged commit 4204477 into main Nov 20, 2026
16 of 18 checks passed
@kanat
kanat deleted the ci/skip-draft-prs branch November 20, 2026 10:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant