Repository navigation
Agent harness adoption Part B: tool loop on @qduc/agent-core (flagged) + provider replay e2e - #42
Merged
Merged
Conversation
…agent-core behind a flag Installs @qduc/agent-core 0.2.0-alpha.1 from a vendored tarball (backend/vendor/, packed from qduc/term2) until the alpha is on npm, and deletes the local contracts mirror so the Part A wrappers compile against the package's own declarations. AGENT_HARNESS_TOOL_LOOP=true routes streaming tool requests to harness/harnessToolsStreaming.ts: it builds a TurnSpec from the request, runs startTurn, and feeds every accepted event to TurnEventSink. Default off; the legacy loop is unchanged. - Per-user max iterations -> TurnSpec.limits.maxModelCalls. - Parallel tool settings -> TurnDeps.maxParallelToolCalls. - Reasoning-only responses are re-asked inside the model wrapper (2x, then the legacy warning). The engine's model_behavior recovery refuses to replay once reasoning has streamed, so it cannot cover this case. - Requests are self-contained for now (no previous_response_id). - Interaction requests are denied rather than left waiting. Packaging: both Dockerfiles copy vendor/ before npm ci, and Electron (which repackages backend dependencies) declares the package too. The backend lockfile's root devDependency specs are also synced to package.json. A parity test drives both loops through /v1/chat/completions against the same scripted upstream: tool call then answer, reasoning-only retry and exhaustion, iteration cap, upstream error. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JqbJBAaQKKFnHRJoDCGExb
Replays sanitized recordings of real provider traffic, copied from qduc/term2's provider black-box fixtures, from a loopback server through the real /v1/chat/completions route, ChatForge's provider adapters, tool execution and persistence. No real API calls. Recordings: OpenAI Chat Completions (OpenCode Go), OpenAI Responses (o3-mini), Anthropic Messages (OpenCode Go) and Gemini, all the two-turn tool-continuation probe. Each runs through the legacy loop and the agent harness loop and is checked against what the recording says (tool name, arguments and call id, answer text) and against the other loop (stream, persisted message and tool rows). The wire checks pin the known continuation difference: legacy chains Responses-API turns with previous_response_id, the harness sends full history. A mutation that stops the harness streaming tool calls fails 8 of the 12 tests. Run: npm run test:provider-replay (also part of npm test). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JqbJBAaQKKFnHRJoDCGExb
pipe() does not forward source errors, so a provider connection that dropped mid-answer left consumers waiting for the idle timeout (~30s) instead of failing in ~100ms. Destroy the passthrough with the source error. Also read error messages by duck typing in toModelFailure: errors from Node internals can come from another realm and fail instanceof Error. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JqbJBAaQKKFnHRJoDCGExb
- Synthetic parallel-call fixtures derived from the real recordings for Chat (sequential and interleaved), Responses, Anthropic and Gemini, shaped after the official SDKs' own tests and parsers (sources in fixtures/PROVENANCE.md). - Oracle test serving every fixture to the official openai, @anthropic-ai/sdk and @google/genai SDKs; real recordings act as calibration. - Replay server can stall or drop a turn mid-stream; Stop and drop scenarios run through both tool loops. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JqbJBAaQKKFnHRJoDCGExb
The harness graduated from Term2 to qduc/agent-runtime. Replace the tarball packed from Term2 with the one packed there (same version and code; repository metadata and README differ) and update the lockfile integrity in backend and electron. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JqbJBAaQKKFnHRJoDCGExb
0.2.0-alpha.1 is now published. The registry tarball has the same sha512 integrity as backend/vendor/qduc-agent-core-0.2.0-alpha.1.tgz, so the installed code is identical. - backend and electron pin the exact npm version - lockfiles resolve from registry.npmjs.org (integrity unchanged) - delete backend/vendor and the COPY vendor lines in both Dockerfiles - update the harness doc and tick the checklist item Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JqbJB9PPzDrqCr5EPj6yNc
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Part B of #40. When
AGENT_HARNESS_TOOL_LOOP=true(off by default), the streaming tool loop runs on@qduc/agent-core. The PR also adds end-to-end replay tests that run both tool loops against recorded provider traffic, with no real API calls. While building the tests I found and fixed a real bug in the upstream logging tee.Changes
Harness loop (
212cc13)backend/src/lib/harness/harnessToolsStreaming.tsruns the flagged loop.openaiProxy.jsroutes to it when the flag is on.providerModel.tsretries a reasoning-only response twice, then emits the existing incomplete-reasoning warning.Provider replay e2e (
50542e5,9e73ce2)integration/providerReplay.test.jsreplays sanitized real recordings from Term2 for Chat Completions, Responses, Anthropic and Gemini. Each one goes through the real route, adapters, tools and persistence, once per loop.fixtures/PROVENANCE.md.integration/syntheticFixtureOracle.test.jsserves every fixture to the officialopenai,@anthropic-ai/sdkand@google/genaiSDKs:Bug fix (
92c2016)upstreamLogger.js:pipe()doesn't forward errors from the source stream. When a provider connection dropped mid-answer, the client waited for the 30s idle timeout. The fix destroys the passthrough with the source error, so the drop is now detected in about 100ms.toModelFailurenow reads error messages by duck typing, because errors from another realm failinstanceof Error.Install
@qduc/agent-corefrom npm (2b0cfff)0.2.0-alpha.1is published. Backend and electron pin that exact version; both lockfiles resolve from registry.npmjs.org.3e322ba), so the installed code is identical.backend/vendor/and theCOPY vendorlines in both Dockerfiles are gone.Verification
npm run lintandnpm run typecheckare clean. The full jest suite passes: 1352 passed, 3 skipped, 15 todo.npm run test:provider-replay: 60/60.npm ciin electron installs0.2.0-alpha.1from npm.id;Known limits
previous_response_idand ignoresprovider_stream:false.tool_outputis a string in the harness loop and an object in the legacy loop.Related
ccr-c2e20a05-677q07. term2 still installs a vendored tarball and needs the same switch to npm.Generated by Claude Code