Skip to content

Keep Anthropic tool prefixes cacheable when system context changes - #263

Open
qduc wants to merge 1 commit into
mainfrom
codex/anthropic-tool-prefix
Open

qduc wants to merge 1 commit into
mainfrom
codex/anthropic-tool-prefix

Conversation

@qduc

@qduc qduc commented Nov 12, 2026

Copy link
Copy Markdown
Owner

Term2 marks later message prefixes for caching, so a changed system message can invalidate reuse of the unchanged tool definitions before it. Add a five-minute breakpoint to the last eligible function tool. Preserve caller-owned cache controls, the four-breakpoint allowance, TTL ordering and all tool schemas/behavior.

A native Claude Haiku 5.5 test used one cold, warm and changed-system request per arm with a synthetic ~20k-token tool prefix. On the changed-system request, the candidate read 19,934 cached tokens versus zero for control; estimated cost was $0.000216 versus $0.002508. This demonstrates conditional prefix reuse, not general task savings or Pi parity. Provider cache state and minimum prefix size still govern eligibility.

Validation: registry → installed SDK → loopback HTTP coverage; full provider suite 273 passed/5 skipped; fake-Codex E2E 18 passed; typecheck, lint, static and baseline gates pass. Independent verification accepted cache controls, TTL/breakpoint protections, failure/recovery and all six retained live request/usage captures. Original experimental worktree preserved.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant