From 823dacb5dfa0e620cde7c8d2bb8a975669166bc3 Mon Sep 17 00:00:00 2001 From: TresPies-source Date: Sat, 31 Oct 2026 08:53:36 -0500 Subject: [PATCH] =?UTF-8?q?fix(dojo-protocol):=200.1.1=20=E2=80=94=20the?= =?UTF-8?q?=20judge=20reports=20before=20its=20turn=20limit;=20briefs=20qu?= =?UTF-8?q?ote=20rules=20verbatim=20(2.4.0,=20unreleased)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The judge has maxTurns: 25 and nothing telling it to report before then. In a session on 2026-10-22/23 one ran out of turns and returned no report; it took a hand resume with a call cap to get verdicts. Two of that judge's findings were its own errors, both from a brief that paraphrased a rule instead of quoting it. - judge.md: report by about the 20th tool call and mark the rest `unverified (not reached)`. maxTurns stays 25, so there is headroom to write the report. A not-reached claim counts as unverified in the final line, so the report can't end in `no defects found`. - delegate (new item 7) and refute: when a judge or reviewer checks work against rules, paste the rules verbatim with source path and line. - Tests: the judge's body must carry the rule, and the version pin moves to 0.1.1. The new assertion fails on main's judge.md. Live check, one run each, same prompt and harness, only judge.md differs: 30 path:line claims, one Read per claim, three false on purpose (4, 12, 29). - main: 25 Reads, no report ("stopped at its 25-turn limit"). - this branch: 20 Reads, then a report: 18 confirmed, 2 refuted (4 and 12), 10 unverified (not reached), including 29. Checks: - WSL Ubuntu, Python 3.12: dojo-protocol 71 tests, 0 skipped, OK. Self-test 296 tests, 2 failures, both from running 3.12 instead of 3.9. - Windows, Claude Code 2.1.305: validate 0 errors on all 9 plugins; 190 mod tests pass. - Always-on: ~1,856 of 2,000 (listing ~1,288 plus SessionStart ~568), with descriptions and PROTOCOL.md unchanged. The row itself reads ~1,288 on Windows, because cmd.exe doesn't expand the hook's plugin-root variable and always_on.py counts a failed hook as 0, so the 568 was measured separately. - The release tables and failing test ids match main on both platforms. - Not run: the denylist (its list isn't on this machine). OVERALL is not green on either platform here, with or without this change. Co-Authored-By: Claude Opus 5.5 --- CHANGELOG.md | 4 ++++ plugins/dojo-protocol/.claude-plugin/plugin.json | 2 +- plugins/dojo-protocol/agents/judge.md | 3 ++- plugins/dojo-protocol/skills/delegate/SKILL.md | 1 + plugins/dojo-protocol/skills/refute/SKILL.md | 1 + plugins/dojo-protocol/tests/test_session_start.py | 4 +++- 6 files changed, 12 insertions(+), 3 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 4299f7a..804885d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -50,6 +50,10 @@ method, the rubric and the tables. It started from five misfires reported from o ### Fixed +- **dojo-protocol 0.1.1** — the judge reports by about its 20th tool call and marks what it didn't check + `unverified (not reached)`. In a session on 2026-10-22/23 a judge hit its turn limit and returned no report at + all. `delegate` and `refute` now say to paste the rules a judge or reviewer applies verbatim, with source path and + line: two false findings in that session came from a paraphrased rule. - **dojo-tally 0.1.2** — a workflow agent that answers through StructuredOutput counts as returned. Its last turn has no text, so the tally called every such agent "failed — it returned an empty answer": on 2026-10-27 a run of 16 healthy scout, verify and synthesize agents read "returned 2 · failed 16". The arguments of the agent's last diff --git a/plugins/dojo-protocol/.claude-plugin/plugin.json b/plugins/dojo-protocol/.claude-plugin/plugin.json index 3b23304..7313f5e 100644 --- a/plugins/dojo-protocol/.claude-plugin/plugin.json +++ b/plugins/dojo-protocol/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "dojo-protocol", - "version": "0.1.0", + "version": "0.1.1", "description": "The Dojo Protocol for Claude Code: ten rules at session start, a scout→contract→build→verify spine, and role agents pinned to the right model.", "author": { "name": "Dojo Genesis", diff --git a/plugins/dojo-protocol/agents/judge.md b/plugins/dojo-protocol/agents/judge.md index 10dc14d..d840c41 100644 --- a/plugins/dojo-protocol/agents/judge.md +++ b/plugins/dojo-protocol/agents/judge.md @@ -17,5 +17,6 @@ Rules you work by (nothing else will tell you them): - Test the negatives: if a probe came back empty, prove it can see a known positive before accepting the empty result. Count what was returned, not what was dispatched. - You are read-only by instruction. Use Bash to run checks, never to change files. Only your tools list is a hard limit. - Do not invent defects. If every claim holds up under your checks, say `no defects found` and list the checks you ran and what each returned. +- Report by about your 20th tool call. If you can't check every claim by then, return verdicts for the ones you checked and mark the rest `unverified (not reached)`. A judge that runs out of turns returns nothing, and that looks the same as a clean pass. -Return shape: a table or list with one line per claim (verdict, evidence), then a final line that is either `no defects found` or the count of refuted and unverified claims. +Return shape: a table or list with one line per claim (verdict, evidence), then a final line that is either `no defects found` or the count of refuted and unverified claims. A `not reached` claim counts as unverified. diff --git a/plugins/dojo-protocol/skills/delegate/SKILL.md b/plugins/dojo-protocol/skills/delegate/SKILL.md index 88525ea..2bd74ee 100644 --- a/plugins/dojo-protocol/skills/delegate/SKILL.md +++ b/plugins/dojo-protocol/skills/delegate/SKILL.md @@ -26,6 +26,7 @@ Template: `${CLAUDE_PLUGIN_ROOT}/skills/delegate/brief-template.md`. 4. **Facts you already hold**: `path:line` citations, decisions from the contract file. Do not make it rediscover them. 5. **Return shape**: what to send back (files changed, commands run with exit codes, open items), and its size. Big deliverables go to a file; reply with the path and a one-line summary. 6. **Rules it must carry**: subagents do not get the injected protocol. State the ones that apply: read before editing, verify before saying done, cite `path:line`, no git add/commit/push unless asked. +7. **Rules it will apply**: when a judge or reviewer checks work against rules, paste the rules verbatim with their source path and line, never a paraphrase. It will judge your paraphrase, and report the difference as a defect. ## Set `model` every time diff --git a/plugins/dojo-protocol/skills/refute/SKILL.md b/plugins/dojo-protocol/skills/refute/SKILL.md index a17d975..0d1f319 100644 --- a/plugins/dojo-protocol/skills/refute/SKILL.md +++ b/plugins/dojo-protocol/skills/refute/SKILL.md @@ -16,6 +16,7 @@ A review can end clean. If there is no real defect, the answer is `no defects fo - Default: `dojo-protocol:reviewer` (sonnet). Fresh context, Read/Glob/Grep/Bash. Give it the goal and the files, not your conclusion. - Claims that need a hard second look (an agent's "all green", a risky merge): `dojo-protocol:judge` (opus). It tries to refute, and it re-runs checks itself. - The reviewer is read-only by instruction. Its tools list is the only hard limit; Bash can still write if told to. +- When a judge or reviewer checks work against rules, paste the rules verbatim with their source path and line, never a paraphrase. ## Steps diff --git a/plugins/dojo-protocol/tests/test_session_start.py b/plugins/dojo-protocol/tests/test_session_start.py index c95ad94..983b783 100644 --- a/plugins/dojo-protocol/tests/test_session_start.py +++ b/plugins/dojo-protocol/tests/test_session_start.py @@ -623,7 +623,7 @@ class TestManifestAndComponents(unittest.TestCase): def test_plugin_json(self): data = json.loads(read(os.path.join(PLUGIN, ".claude-plugin", "plugin.json"))) self.assertEqual(data["name"], "dojo-protocol") - self.assertEqual(data["version"], "0.1.0") + self.assertEqual(data["version"], "0.1.1") self.assertEqual(data["description"], PROMISE) self.assertLessEqual(len(data["description"]), 200) self.assertEqual(data["author"], {"name": "Dojo Genesis", "email": "cruz@trespiesdesign.com"}) @@ -683,6 +683,8 @@ def test_agent_bodies_carry_their_own_rules(self): _fm, scout = parse_frontmatter(os.path.join(PLUGIN, "agents", "scout.md")) self.assertIn("path:line", scout) self.assertIn("not found", scout) + _fm, judge = parse_frontmatter(os.path.join(PLUGIN, "agents", "judge.md")) + self.assertIn("unverified (not reached)", judge) def test_skills_and_budget(self): skills_dir = os.path.join(PLUGIN, "skills")