Repository navigation
feat(optimize-skill): prove pointers and estimate what a run reads - #2000
Merged
Merged
Conversation
Eval suites that paste moved text into the prompt with also_include show a rule still works, never that an agent reaches it. Tell optimize-skill to prove each pointer with a pointer-only eval and a copy with the pointers deleted, to compare prose with a tool line by line before cutting it, and to estimate a run cost per path, since the SKILL.md body alone understates what an invocation reads. Add pointer-routing steps for the pointers still untested: the issue-stale-sweep failure-modes pointer and seven pre-PR review sites. Generated-by: Claude Opus 5
potiuk
enabled auto-merge (squash)
November 5, 2026 01:43
potiuk
disabled auto-merge
November 5, 2026 01:47
potiuk
enabled auto-merge (squash)
November 5, 2026 01:47
This was referenced Nov 5, 2026
potiuk
added a commit
that referenced
this pull request
Nov 5, 2026
Describe the ASF network scoping from #1978, including the claim it made false that containers share the sandbox's host allow-list. Record the run cost and pointer proof from #2000, the verify-rc pane from #2007, and the nine-skill pr-management family. Recheck every Known gaps bullet in eleven specs: delete those now closed, rewrite the stale ones, and annotate the rest with gap-closes-when. Generated-by: Claude Opus 5
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
optimize-skillwhat eperf(skills): load conditional shared text only when it applies #1996–perf(security-issue-sync): read conditional Step 1 sub-steps only when they apply #1998 had to learn the hard way: how to prove a pointer works, and to compare prose with a tool before cutting it.SKILL.mdbody understates what an invocation reads:security-issue-synchas a body of about 6k tokens, but a routine run reads about 46k.pointer-routingeval steps for the pointers still untested.New
optimize-skillguidance:The guidance sits inside the existing Split section, so no heading is added and no
surface_hashmoves.Type of change
.claude/skills/<name>/) — eval fixtures updated belowTest plan
prek run --all-files --skip workspace-pytestpasses, includinglycheeworkspace-pytestpasses for the changed filessurface_hashchangedoptimize-skillevals: 7/7 before and afterpointer-routingsteps, 8 suites: 19/19 on the real skills; with the pointers deleted, all 11 cases that need a file fail and all 8 that need none passsecurity-model-verifycase-1failed once on the real skill, returning two files. Step 6 useshelper.mdandtemplates.md, so the fixture now says they are already open, with the same expected answer. It then passed 3/3 and failed on the copy without pointers. The runner does not show the extra file, so the cause is likely rather than confirmed.Linked issues
Follows #1996, #1997, #1998.
Notes for reviewers
optimize-skillsuite cannot prove the guidance works. It only shows that diagnosis did not change.Step 1still ranks by cap overflow and smells, which is whatstep-diagnosereads.pointer-routingrecords a stated decision. The agent says which files it would open; it does not run with file tools.🤖 Generated with Claude Code