Skip to content

fix(handler,mcp): a COMMIT whose connection broke says unknown on every write path, the confirm notes its write at the commit, and the remaining writes name write_outcome (#483) - #490

Merged
GottZ merged 11 commits into
rootfrom
issue/483-write-line-followups
Nov 22, 2026

Conversation

@GottZ

@GottZ GottZ commented Nov 22, 2026

Copy link
Copy Markdown
Owner

What this changes

A COMMIT whose connection broke says unknown, not rolled_back (point 1). pgx v5.11 reports a connection that breaks while a COMMIT waits for its answer as pgconn.ErrConnClosed — the lock error of a call on a closed connection, marked safe to retry (its MultiResultReader.NextResult drops the read error). writeOutcome and statementOutcome read that as "never sent" and said rolled_back for a write the database then committed: on every labeled bracket (MCP store, update, archive, guard_resolve, store_batch, the staged batch, the confirmed store/update/archive/batch, REST manage update, guard-resolve with ids, ingest block mode) and on the issue tools. Both classifiers now say unknown for it (client_abort.go, mcp_write_line.go). pgx also sends a cancel request on a new connection after such a break, so whether the held COMMIT lands is a race in production; unknown is the only honest word.

A test per write path holds its classifier (point 1). write_outcome_cut_integration_test.go: the handlers' pool runs through a TCP proxy; a deferred constraint trigger holds the commit of the path's write on an advisory lock; once it waits, the proxy closes both sides and refuses new connections. Every path's failure line and MCP tool line must say unknown, and the held commit must land once the lock goes away — 12 statement paths, 13 labeled paths, the 11 paths of point 8 and the 7 admin writes of point 9 that take statementOutcome. A second test runs probe B of the #482 review: a COMMIT whose deadline passes while it waits ends with pgconn's timeout: context deadline exceeded (not safe to retry), statementOutcome says unknown, and writeOutcome on an unlabeled bracket or single statement would say rolled_back. A type split between the two classifiers was checked and is not small: pgxdb.WriteOpts passes fn's error through unchanged (its documented contract), so a bracket marker would change the identity of every fn error in every package.

The confirm tool notes its write at the commit (points 2, 13). mcpConfirmHandler noted the outcome after executeConfirm returned, so a panic in the afterwork of a confirmed store left the line at write_outcome=none for a block that stands. The confirm cores note where the write ends — committed right after the commit, before the afterwork; a failed execute with its store function's classifier next to the failure line, and the block (update, archive) or blob (blob_link) the card names, which the line lacked on confirmExecErr. Sweep of the eleven write tools: only the confirm had code between its commit and its note.

write_outcome on the eight remaining paths (point 8). manage delete and dream-link-resolve (labeled brackets, writeOutcome); guard-resolve of one id, chunk ingest, blob delete, the chat's staged store and update (one statement each, statementOutcome); the REST issue writes — manage issue-create, issue-update, issue-comment-create and the three project issue routes — through writeIssueWriteError (statementOutcome, the issue transaction has no commit label); a purge says committed when an earlier batch removed something. The census allows no path without an outcome.

writes.batch_max_items at most 1000 (point 3). No path classifies a block later but its next write (no scheduler, guard or dream backfill for type_name). Measured afterwork on a test database on a loaded host: about 3 ms per block — 100 blocks 0.29 s, 1000 blocks 3.0 s, 5000 blocks 16.1 s, past the 15 s drain, so a bound tied to the batch size could not cover it either. V67 refuses a larger value (boot abort for the environment value, 422 for a settings write, a psql row dropped with a WARN), like V43 did for dispatch.external_max_wait.

Gaming toggle (point 4). The backend pool reload and the settings reload after the toggle each run detached on 5 s of their own (together inside the 10 s bound and the 15 s drain); a hung backend reload no longer ends the settings reload before it starts. Either reload failing — the backend pool's carries the toggle on this instance — answers 500 {"success": false, "persisted": true, "error": "persisted but reload failed — the toggle is written …"} instead of a bare "internal error" (and a failed backend reload no longer answers 200).

The staged blob_link names the stored id (point 5). store.BlobWriteScope also returns the blob's id as stored; the staged card and the stage line carry it (the answer text keeps the client's spelling).

Census (point 7). It now sees if x, err := pkg.F(…); err != nil {…}, an aliased store import and writes in switch/select cases; return store.F(…) stays counted, not followed.

Admin write failures log (point 9, tenants, scopes, grants, quotas). tenant-create, tenant-update, tenant-delete, tenant-limit-set, scope-create, tenant-grant-create, tenant-grant-delete, block-grant-create, block-grant-revoke and tenant-quota-set write one ERROR line manage: <action> failed with the error, write_outcome (unknown for tenant-delete, whose statements commit one by one) and request_id — no request argument, name or key.

POST /api/digest discards its body (point 11). The CLI sends {"trigger":"manual"}; net/http watches the connection only once the body is consumed, so such a client was noticed only when the answer was written. The run now ends with a client that leaves.

Docs (point 6 and the above). docs/operations.md: rolled_back assumes no synchronous standby; the paths, the batch ceiling, the gaming answer, the admin lines, the digest. docs/api.md, docs/mcp.md: the ceiling and the gaming answer.

Why

Part of #483. Open: point 9 for backend-* and embed-migration-*, points 10 and 12 (listed in the issue comment). Rejected alternative for point 3: a bound proportional to the batch size, capped inside the drain — the measured 5000-block afterwork (16.1 s) exceeds any cap inside the 15 s drain.

Behaviour changes (intended)

  • A write whose COMMIT broke on the wire logs write_outcome=unknown (was rolled_back).
  • writes.batch_max_items above 1000 is refused (an environment value aborts the boot). The live .env sets no value (default 100).
  • The gaming toggle's failed-reload answer carries persisted: true and a new error text (500 as before for a failed settings reload; a failed backend-pool reload answered 200 and now answers 500).
  • A staged blob_link card's id is the stored spelling: its payload hash differs for an upper-case or braced input, and two spellings of one link stage (re-arm) one card instead of two; the answer text is unchanged.
  • A client that leaves POST /api/digest during the run is seen leaving: the run ends with it, as every request does (its classifications after the upsert stay detached).
  • The REST issue writes' 500 line, the purge's failure line and the other point-8 lines gain write_outcome; ten admin routes gain a failure line.

Consumers of the changed contracts

Changed Callers and readers Covered by
writeOutcome, statementOutcome (ErrConnClosed) every write_outcome site in internal/handler (census: 33 write calls) TestWriteOutcome, TestStatementOutcome, TestWriteOutcomeBrokenCommit, the cut tests
confirm cores note the tool line; noteConfirmOutcome removed mcpConfirmHandler, ConfirmHandler (REST, notes are no-ops there) TestMCPConfirmLine, TestMCPWriteLineEveryWriteToolCommits
store.BlobWriteScope returns the stored id mcpStageBlobLink; store test TestMCPStagedBlobLinkNamesTheStoredID, TestBlobBlockRef…
writeIssueStoreError(…, attrs), writeIssueWriteError manage issue handlers, ProjectIssuesHandler TestWriteOutcomeCutRemainingPaths
writeBootstrapError(w, r, err) handleTenantCreate TestAdminWriteFailureLines
config.WritesBatchMaxItemsCeiling, V67 config.Validate (boot, settings PUT, reload) TestWritesBatchMaxItemsCeiling, TestFollowupBoundBatchAfterworkFits
gaming-mode answer; reloadBackendPool returns its error web UI and CLI (cli/eject.go) read success/error only; reloadAfterMutation drops the error as before TestFollowupBoundGamingReloadFailureSaysWritten, TestFollowupBoundGamingReloadsHaveTheirShares

Cost

No new statement on a write path. BlobWriteScope selects one more column. The digest route reads at most the 1 MB its middleware allows. The new log lines are written only on failure.

Verification

  • Red on the base 168b7840, green after: the cut tests — 13 labeled paths and the 3 issue tools said rolled_back for a commit that then landed (TestWriteOutcomeBrokenCommit: both classifiers rolled_back); the 11 point-8 paths had no write_outcome; TestMCPConfirmLine (write_outcome=none after a panic past the commit, no block_id/blob_id on a failed execute); TestMCPStagedBlobLinkNamesTheStoredID (upper-case and braced id on the stage line); TestFollowupBoundGamingReloadsHaveTheirShares (settings reload ran on an ended context) and …FailureSaysWritten (bare "internal error"); TestAdminWriteFailureLines (10/10 without a line); TestWritesBatchMaxItemsCeiling (1001 and 5000 accepted); TestFollowupBoundDigest with the CLI's body (the server saw the client go only when the handler returned). Unit: TestWriteOutcome, TestStatementOutcome with pgx's ErrConnClosed shape. Green on the base by design (they pin correct code): TestStatementOutcomeDeadlineAtCommit and the single-statement cut paths.
  • Mutants: 28, all killed — the three surviving mutants of the fix(handler,mcp): an MCP write tool writes one server line with its write_outcome, a handler that wrote nothing logs handler_status=0, and the work after a commit ends at a bound, not with the client #482 review (DeadlineExceeded counted as rolled back in statementOutcome; the confirmed blob ops noted with noteToolWrite; the staged blob_link noted as labeled) and 25 of our own (each classifier swap on a statement path, the ErrConnClosed branches, the confirm notes, the purge outcome, V67, the gaming shares and answer, the stored blob id, the digest drain, the admin lines and classifiers, the census forms).
  • bash go/gates.sh wave --final --base 168b7840: rc 0 — build, vet, lint, doccheck, short (231 s), census, race (319 s), integration of ./internal/handler/ (579 s), ./internal/store/ (659 s), ./cmd/ctxd/, ./internal/config/, ./internal/handler/v1door/, ./internal/testdb/ (6/6 ok), python, deadcode, deadcode-testonly.
  • The new tests -count=5, with and without -race (the two slow digest/batch tests -count=2).
  • Sample lines from a test run:
    • level=ERROR msg="mcp: store error" error="store: upsert block: commit: conn closed" write_outcome=unknown request_id=…
    • level=INFO msg="mcp tool write" tool=store write_outcome=unknown duration_ms=20 request_id=…
    • level=ERROR msg="manage: tenant-grant-create failed" error="store: create tenant grant: ERROR: probe fault (SQLSTATE P0001)" write_outcome=rolled_back request_id=sample-req
  • A fresh reviewer agent read the diff; its medium finding (a failed backend-pool reload of the toggle answered 200) and three low/notes (the deadline test's landing raced pgx's cancel request, the admin classifiers were unpinned, the census skipped case bodies) are fixed.

Not verified

  • A synchronous standby (point 6) — documented only.
  • The afterwork measurement comes from a test database on a loaded host, not production; a settings row for writes.batch_max_items on the live instance was not read (needs an admin key).
  • Points 9 (backend-, embed-migration-), 10 and 12 are not built (see the issue comment).
  • The project issue update and comment routes share writeIssueWriteError with the cut-tested manage routes; they have no cut test of their own.

🤖 Generated with Claude Code

GottZ and others added 11 commits November 22, 2026 00:12
…own, not rolled_back, and a test per write path holds its classifier

Part of #483 (point 1).

pgx v5.11 reports a connection that breaks while a COMMIT waits for its
answer as pgconn.ErrConnClosed, the lock error of a call on a closed
connection, marked safe to retry (MultiResultReader.NextResult drops the
read error). writeOutcome and statementOutcome read that as "never sent"
and said rolled_back for a write the database then committed - on every
labeled bracket (MCP store, update, archive, guard_resolve, store_batch,
the staged batch, the confirmed store/update/archive/batch, REST manage
update, guard-resolve with ids, ingest block mode) and on the issue tools.
Both classifiers now say unknown for it.

A TCP proxy between the handlers' pool and a real PostgreSQL cuts the
connection while a deferred constraint trigger holds the commit; every
write path's failure line and MCP tool line must say unknown and the held
commit must land afterwards. A second test runs probe B of the #482 review:
a COMMIT whose deadline passes while it waits ends with pgconn's
"timeout: context deadline exceeded", statementOutcome says unknown, and
writeOutcome on an unlabeled bracket would say rolled_back.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…mmit, and a failed execute names the block or blob of the card

Part of #483 (points 2 and 13).

mcpConfirmHandler noted the confirm's outcome after executeConfirm returned,
so a panic in the afterwork of a confirmed store (the classification after
the upsert) left the tool line at write_outcome=none for a block that
stands. The confirm cores now note where the write ends, as the other
write tools do: committed right after the commit, before the afterwork,
and a failed execute with the classifier of its store function next to the
failure line - together with the block (update, archive) or blob
(blob_link) the card names, which the line lacked on confirmExecErr. A
confirmed batch names its item count also when it fails.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ame it, and the census allows none

Part of #483 (point 8).

manage delete and dream-link-resolve take writeOutcome (labeled brackets);
guard-resolve of one id, the chunk ingest, the blob delete and the chat's
staged store and update take statementOutcome (one statement each); the
REST issue writes (manage issue-create, issue-update, issue-comment-create
and the three project issue routes) take statementOutcome through
writeIssueWriteError (the issue transaction carries no commit label); a
purge says committed when an earlier batch removed something, else what
its failing statement left behind. The census's list of paths without an
outcome is empty. Each path has a cut test: its commit is held and its
connection cut, and the failure line must say unknown.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…largest batch whose afterwork fits its one follow-up bound

Part of #483 (point 3).

A batch classifies and enriches all its written blocks after the commit
under one 10 s bound inside the 15 s shutdown drain, and nothing classifies
a block later but its next write (no scheduler, guard or dream backfill for
type_name). The setting had no upper limit. Measured on a test database on
a loaded host: about 3 ms per block - 100 blocks 0.29 s, 1000 blocks 3.0 s,
5000 blocks 16.1 s, past the drain, so a bound tied to the batch size could
not have covered it either. V67 refuses a value above 1000 (boot abort for
the environment value, 422 for a settings write, a psql row dropped with a
WARN), and the handler test now measures a batch of the ceiling's size
inside the bound (2.7-2.8 s).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…the follow-up bound, and a failed reload answers that the toggle is written

Part of #483 (point 4).

The backend pool reload and the settings reload after the eject/gaming
toggle shared one 10 s follow-up context: a backend reload that hung for
the whole bound ended the settings reload before it started, and the
client got a generic 500 "internal error" for a toggle that had committed.
Each reload now runs detached on 5 s of its own (the pair stays within the
10 s bound inside the 15 s drain), and a failed settings reload answers
500 with persisted=true and a text that says the toggle is written.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…, not as the client spelled its id

Part of #483 (point 5).

store.BlobWriteScope, the stage-time scope lookup of a blob_link card, also
returns the blob's id as stored; the card carries it and the MCP line of
the stage names it, as the direct path names blob.ID since 0f31fed. A test
stages a link with an upper-case and a braced uuid and reads both the stage
line and the confirm line.

The cut tests' proxy refuses new connections until the held backend is
gone: pgx sends a cancel request on a new connection after a broken one,
which could cancel the held COMMIT before the test released it and made
the landed check race.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Part of #483 (point 6).

A COMMIT whose backend is terminated (57P01) while it waits for a
synchronous standby has committed locally although the client receives an
error. Not built (no such deployment); the write_outcome paragraph says it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…and under an aliased store import

Part of #483 (point 7).

The census matched only `x, err := store.F(...)`, `err = store.F(...)` and
`return store.F(...)` under the name "store". It now also follows
`if x, err := pkg.F(...); err != nil {...}` (the if's body is the failure
branch) and reads the name each file imports the store package by. A
synthetic file holds both forms; the package still has 33 write calls,
none of them in the new forms. `return store.F(...)` stays counted, not
followed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…RROR line with its write_outcome

Part of #483 (point 9).

Ten admin write routes answered a bare 500 with no line: tenant-create,
tenant-update, tenant-delete, tenant-limit-set, scope-create,
tenant-grant-create, tenant-grant-delete, block-grant-create,
block-grant-revoke and tenant-quota-set - rights and budgets that changed
or did not, with no trace. Each now writes "manage: <action> failed" with
the error, write_outcome (writeOutcome for the labeled tenant bootstrap and
scope assignment, statementOutcome for the single statements and the
unlabeled quota bracket, unknown for tenant-delete, whose statements commit
one by one) and the request id; no request argument, name or key. The
backend-* (4) and embed-migration-* (8) routes are left for a follow-up:
their failures run through helpers without the error (writeBackendTxFailure)
or answer 4xx and 500 from one mapper (embedMigrationMechErr).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… client that leaves is seen leaving

Part of #483 (point 11).

The route takes no body, but the CLI sends {"trigger":"manual"}, and
net/http watches a connection for its client going away only once the
request body is consumed: such a client was noticed only when the answer
was written. The handler now drains the body first (bounded by the route's
1 MB MaxBodySize). The digest tests send the CLI's body; on the base the
server saw the client go only when the handler returned, past the
classification bound.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… gaming toggle answers persisted, the admin paths' classifiers and the deadline test hold under a cut

Part of #483.

- The gaming toggle's backend pool reload carries the toggle; its failure
  now answers 500 with persisted=true as a failed settings reload does
  (reloadBackendPool returns its error), and the text names the NOTIFY
  listeners.
- The cut tests cover the seven admin writes that take statementOutcome
  (tenant-update, tenant-limit-set, tenant grants, block grants, quota).
- TestStatementOutcomeDeadlineAtCommit runs on the proxied pool and refuses
  pgx's cancel request until the held commit went on.
- The census also walks switch and select cases; V67's comment states the
  measurement and the live setting; the docs say a digest run ends with its
  client.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@GottZ
GottZ requested a balanced review from Copilot November 22, 2026 00:52
@coderabbitai

coderabbitai Bot commented Nov 22, 2026

Copy link
Copy Markdown

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 19 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: e2b491e4-2bd2-4792-8fbf-de2a64e0a00a
Commits

Reviewing files that changed from the base of the PR and between 53b8f59 and 0b950c8.

Files selected for processing: 36

  • docs/api.md
  • docs/mcp.md
  • docs/operations.md
  • go/internal/config/config.go
  • go/internal/config/descriptions.go
  • go/internal/config/guard_search_test.go
  • go/internal/config/validate.go
  • go/internal/handler/admin_write_line_integration_test.go
  • go/internal/handler/backends_manage.go
  • go/internal/handler/blob.go
  • go/internal/handler/block_grant_manage.go
  • go/internal/handler/chat_stage.go
  • go/internal/handler/client_abort.go
  • go/internal/handler/client_abort_rule_test.go
  • go/internal/handler/confirm_core.go
  • go/internal/handler/context_gaming.go
  • go/internal/handler/context_manage.go
  • go/internal/handler/context_manage_dreamlinks.go
  • go/internal/handler/context_manage_issues.go
  • go/internal/handler/digest.go
  • go/internal/handler/followup_bound_integration_test.go
  • go/internal/handler/ingest.go
  • go/internal/handler/mcp_blob.go
  • go/internal/handler/mcp_confirm.go
  • go/internal/handler/mcp_confirm_line_integration_test.go
  • go/internal/handler/mcp_write_line.go
  • go/internal/handler/mcp_write_line_test.go
  • go/internal/handler/project_issues_write.go
  • go/internal/handler/quota_manage.go
  • go/internal/handler/scope_purge_manage.go
  • go/internal/handler/store_batch.go
  • go/internal/handler/tenant_manage.go
  • go/internal/handler/write_outcome_census_test.go
  • go/internal/handler/write_outcome_cut_integration_test.go
  • go/internal/store/blob_block_ref_integration_test.go
  • go/internal/store/blobs.go
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

@GottZ
GottZ merged commit f13ec0e into root Nov 22, 2026
30 checks passed
@GottZ
GottZ deleted the issue/483-write-line-followups branch November 22, 2026 01:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants