From 5c9f260def714c32bce9d939477b1d5620fcdbd0 Mon Sep 17 00:00:00 2001 From: Raghav Gupta Date: Tue, 3 Nov 2026 22:04:20 -0400 Subject: [PATCH] feat(verify): prefer make check-diff where a repo has it; the reviewer reruns only what confirms a finding The verify skill sent every run to the full make check. Where the repo root has a check-diff target, it now commits the work first and runs make check-diff, which picks the steps the committed diff needs; a step it did not pick is "not selected", never "passed". The full suite runs on request and in CI on the default branch. Repos without check-diff keep make check. The reviewer reruns only the checks that confirm or rule out a finding, preferring make check-diff, since CI is already green on the PR. tests/run.sh checks the new text, that the skill stays provider-neutral, and that the verify skill and the agents name no repo outside the allowed list. Co-Authored-By: Claude Opus 5.5 --- plugins/setkernel/agents/reviewer.md | 4 +++- plugins/setkernel/skills/verify/SKILL.md | 15 ++++++++++----- tests/run.sh | 8 ++++++++ 3 files changed, 21 insertions(+), 6 deletions(-) diff --git a/plugins/setkernel/agents/reviewer.md b/plugins/setkernel/agents/reviewer.md index 1f54937..e752e1a 100644 --- a/plugins/setkernel/agents/reviewer.md +++ b/plugins/setkernel/agents/reviewer.md @@ -38,7 +38,9 @@ the code before reporting it; drop anything you cannot substantiate. For each bl - why it is wrong (input → wrong result); - how to show it fails (a command, an input, a test), and the smallest fix. -Run the repo's checks (see "Commands" in `AGENTS.md`) and report their exact result lines. Stay +CI is already green on the PR, so rerun only the checks that confirm or rule out a finding: +`make check-diff` if the repo has that target, else the checks under "Commands" in `AGENTS.md`. Report +their exact result lines; a step `make check-diff` did not pick is "not selected", never "passed". Stay read-only: `git`, reads and the checks only; never edit files. If there are no blockers, say "No blockers" and list at most three non-blocking notes. End with "merge", "merge after fixes" or "don't merge". diff --git a/plugins/setkernel/skills/verify/SKILL.md b/plugins/setkernel/skills/verify/SKILL.md index 3f43290..2681827 100644 --- a/plugins/setkernel/skills/verify/SKILL.md +++ b/plugins/setkernel/skills/verify/SKILL.md @@ -10,11 +10,16 @@ that tree, with its exact result. ## Find the checks -0. If the repo root has a `check` target (the first of `GNUmakefile`, `makefile`, `Makefile` that - exists, with a line matching `^check[[:space:]]*:`), run `make check` and use it instead of - reading the workflows and running steps yourself: it runs every check CI runs. Report its - per-step PASS/FAIL lines and its final count. Without a `check` target, use steps 1–2. - Steps 3 and 4 apply either way: run the setup step before `make check`. +0. If the repo root has a `check-diff` target (the first of `GNUmakefile`, `makefile`, `Makefile` + that exists, with a line matching `^check-diff[[:space:]]*:`), commit the work first, then run + `make check-diff`: it picks the steps the committed diff against the default branch needs. + Report its per-step PASS/FAIL lines and its final count; a step it did not pick is + "not selected", never "passed". Run the full `make check` only when you are asked to; CI runs + it on the default branch. Otherwise, if the repo root has a `check` target (a line matching + `^check[[:space:]]*:`), run `make check` and use it instead of reading the workflows and running + steps yourself: it runs every check CI runs. Report its per-step PASS/FAIL lines and its final + count. With neither target, use steps 1–2. + Steps 3 and 4 apply either way: run the setup step before `make check-diff` or `make check`. 1. Read the "Commands" section of the root `AGENTS.md` (and any area `AGENTS.md` for the files you touched), then the CI workflow(s) in `.github/workflows/`. 2. While iterating, run the smallest set of checks that can catch what the change could break. diff --git a/tests/run.sh b/tests/run.sh index 0c178c2..5ba6c7a 100755 --- a/tests/run.sh +++ b/tests/run.sh @@ -1022,6 +1022,14 @@ skill=$root/plugins/setkernel/skills/verify/SKILL.md expect 0 "run \`make check\`" "the verify skill runs make check when the repo has the target" -- grep -F 'run `make check`' "$skill" expect 0 "needs Docker (or OrbStack)" "the verify skill reports a Docker-less run as not run" -- grep -F 'needs Docker (or OrbStack)' "$skill" expect 0 "Steps 3 and 4 apply either way" "the verify skill keeps the grep and setup steps with a check target" -- grep -F 'Steps 3 and 4 apply either way' "$skill" +# It prefers `make check-diff` where a repo has it: committed work first, unpicked steps "not selected". +expect 0 "make check-diff" "the verify skill runs make check-diff when the repo has the target" -- grep -F 'make check-diff' "$skill" +expect 0 "commit the work first" "the verify skill commits before make check-diff" -- grep -F 'commit the work first' "$skill" +expect 0 "not selected" "the verify skill reports unpicked steps as not selected" -- grep -F 'not selected' "$skill" +expect 1 "" "the verify skill names no model, CLI or vendor" -- grep -iE 'claude|codex|anthropic|openai' "$skill" +expect 0 "make check-diff" "the reviewer reruns make check-diff where the repo has it" -- grep -F 'make check-diff' "$root/plugins/setkernel/agents/reviewer.md" +expect 0 "^none unlisted$" "the verify skill names no setkernel repo outside the allowed list" -- unlisted_repos "$root/plugins/setkernel/skills/verify" +expect 0 "^none unlisted$" "the agents name no setkernel repo outside the allowed list" -- unlisted_repos "$root/plugins/setkernel/agents" # The suite passes with SETKERNEL_GUARD_NO_PUSH=1 in its environment (as in sk-agent runs). One nested run, not recursive. if [ -z "${SETKERNEL_TESTS_NESTED:-}" ]; then