Repository navigation
feat(verify): prefer make check-diff where a repo has it - #67
Conversation
…r reruns only what confirms a finding The verify skill sent every run to the full make check. Where the repo root has a check-diff target, it now commits the work first and runs make check-diff, which picks the steps the committed diff needs; a step it did not pick is "not selected", never "passed". The full suite runs on request and in CI on the default branch. Repos without check-diff keep make check. The reviewer reruns only the checks that confirm or rule out a finding, preferring make check-diff, since CI is already green on the PR. tests/run.sh checks the new text, that the skill stays provider-neutral, and that the verify skill and the agents name no repo outside the allowed list. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
orion-setkernel
left a comment
There was a problem hiding this comment.
No merge blockers. The diff matches its description and touches only the verify skill, the reviewer agent and tests/run.sh.
Checks I ran on the PR head: bash tests/run.sh gave 832/832 passed (the PR body says 803; the count has grown since, probably from the merge of main). bin/generate --check said the generated files are current. shellcheck -S warning over the AGENTS.md file list, including skills/request-review/*.sh, exited 0. I did not run dash or the JSON check.
What is good
- Step 0 of the verify skill now prefers
make check-diff. Repos that only havemake checkbehave as before. - Committing first is needed for a committed-diff scope, and the skill says so. A step that was not picked is reported as "not selected", never "passed". That fits the org rule about reporting checks you did not run.
- The new
expectlines each check something that can break: the new text, no vendor names in the skill, andunlisted_reposon the verify skill and the agents (a function that already exists, line 100).
Nits (non-blocking)
- Step 2 of the verify skill still says that before anything reaches the default branch you run "the same steps CI runs". With step 0 that now reads as contradicting it. Add "unless
make check-diffselected them" or similar. - The reviewer text says CI is "already green". That holds for the PR review flow, but the reviewer agent is also used pre-PR through the review-blockers skill, where no CI has run. Consider "where CI is already green".
- The AGENTS.md traps section does not mention that the verify skill must name no vendor or model. The test enforces it, so this is optional.
Process note: this changes agent runtime behaviour on every seat once merged, through auto-update. Whether it needs R3 handling is for Raghav and the risk policy to say.
Automated: @orion-setkernel's agent on orion-dev.
|
Approvals that count toward the two a merge needs: 1 (orion-setkernel) on head Automated: @orion-setkernel's agent on |
|
Handed back to @realRaghavGupta: the merge gate won't merge this PR because the merge gate is off in agent-plugins (its mode in hq's policy/autonomy.json). |
lyra-setkernel
left a comment
There was a problem hiding this comment.
No blockers. The diff matches the PR description: the verify skill's step 0 now prefers make check-diff (commit first, unpicked steps reported as "not selected", full make check only on request), make check stays the fallback, and the reviewer reruns only checks that confirm or rule out a finding. The new expect lines in tests/run.sh are cheap text guards.
Checks I ran on the PR head (merged with current main): bash tests/run.sh 832/832 passed; dash tests/run.sh 832/832 passed; bin/generate --check "generated files are current"; shellcheck -S warning over the AGENTS.md file list exit 0. The PR body says 803, because main added tests after it was written.
Nits (non-blocking):
- The opt-in Stop gate (
plugins/setkernel/hooks/verify-gate.sh:59, README.md:166) still runs the fullmake check, so it keeps the cost this PR cuts. It is outside this issue; a follow-up could addcheck-diffthere. - The skill now says the full
make checkruns "only when you are asked to", while the org rule says done means the repo's checks pass on the reported tree. Defensible, sincecheck-diffis the repo's own scope, but a one-line note that CI on the default branch covers the rest would remove the tension. - "Commit the work first" is a local commit only; consider saying "locally" so nobody reads it as a push.
Merging releases this to every seat via plugin auto-update (no version), so it's on the Raghav / merge-gate path as the PR says.
Automated: @lyra-setkernel's agent on lyra-dev.
|
Approvals that count toward the two a merge needs: 2 (lyra-setkernel orion-setkernel) on head Automated: @lyra-setkernel's agent on |
What
check-difftarget, commit the work first and runmake check-diff. It picks the steps the committed diff against the default branch needs. A step it did not pick is reported as "not selected", never "passed". The fullmake checkruns only on request, and in CI on the default branch. Repos withoutcheck-diffkeepmake checkas before.make check-diff.tests/run.sh. Newexpectlines check the new text, that the verify skill names no model, CLI or vendor, and that the verify skill and the agents name no repo outside the allowed list.Why
Agent runs ran the whole suite on every work and review step and hit their time limits. A scoped run checks what the diff can break; CI still runs everything.
Checks
bash tests/run.shdash tests/run.shbin/generate --checkshellcheck -S warning bin/generate plugins/setkernel/hooks/*.sh tests/run.shNo
versionbump:plugin.jsonhas none by design (AGENTS.md).Merging releases this to every seat through plugin auto-update.
— Claude Lead
🤖 Generated with Claude Code