Skip to content

docs(rules): judge work by its output; a test only where logic can silently break - #65

Merged
realRaghavGupta merged 1 commit into
mainfrom
lead/output-over-tests
Nov 3, 2026
Merged

realRaghavGupta merged 1 commit into
mainfrom
lead/output-over-tests

Conversation

@realRaghavGupta

Copy link
Copy Markdown
Contributor

Draft for Raghav's review. This changes org policy, so Raghav merges it.

Why

The work is done for an output. Tests are there to show it works as it should; they are not the goal. A process that makes nearly every PR add a test file and a check step is feeding itself.

Over the last week, more lines of tests and check steps were added than lines of everything else. Three rules drove it:

  • "new behaviour needs a test";
  • the reviewer agent blocking on "missing tests for new behaviour";
  • the worker prompt's "Add or update tests".

What changes

  • rules/org-rules.md: "new behaviour needs a test" becomes:
    • Done means the output works where it is used, shown by a run, a screenshot or a live check.
    • Tests prove the output works; they are not the output.
    • Add a test only where logic can silently break, in an existing test file when one fits.
    • Never add a script, check step or test for its own sake.
    • The file stays at 64 lines.
  • plugins/setkernel/agents/reviewer.md:
    • It blocks on no proof the change works, or on logic that can silently break with no test.
    • It never blocks on a missing test when the PR shows the change working.
    • It flags tests, scripts or check steps that only restate the code, as a non-blocking note.
  • managed/settings.json: regenerated with bin/generate.

Unchanged: the test-diff guard and its allow lines.

After merge

Raghav pastes managed/settings.json into the org's managed settings. The reviewer change reaches seats through plugin auto-update.

Checks

bash tests/run.sh: "795/795 passed".

🤖 Generated with Claude Code

…lently break

Raghav, 2026-11-03: work is for an output, and tests prove it works. They
are not the output. The old line "new behaviour needs a test", with the
reviewer blocking on missing tests, made almost every PR add a test file and
a check step.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@realRaghavGupta
realRaghavGupta marked this pull request as ready for review November 3, 2026 12:12
@realRaghavGupta
realRaghavGupta merged commit fe4bf97 into main Nov 3, 2026
1 check passed
@realRaghavGupta
realRaghavGupta deleted the lead/output-over-tests branch November 3, 2026 12:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant