Skip to content

Release a chunk when the request body read fails - #2383

Open
pavel-ptashyts wants to merge 1 commit into
AsyncHttpClient:mainfrom
maygemdev:claude/multipart-upload-flake
Open

pavel-ptashyts wants to merge 1 commit into
AsyncHttpClient:mainfrom
maygemdev:claude/multipart-upload-flake

Conversation

@pavel-ptashyts

Copy link
Copy Markdown
Contributor

Summary

CI job test (windows-latest, 11) failed on #2378 (run 42294836025) in MultipartUploadTest.testSendFileInputStreamZeroCopyUnknownContentLength. A leaked ByteBuf then failed every later test class through the leak-detector extension, so one flaky test showed up as more than 20 failing classes. Both causes are fixed here.

1. Test server race (the failing test)

  • The test servlet answered as soon as commons-fileupload reached the multipart closing boundary. For a chunked request body (unknown content length), the terminating chunk can arrive after that: NettyBodyBody.write sends the ChunkedInput and LastHttpContent as two writes.
  • Jetty 11 HttpConnection.onCompleted() calls the non-blocking _input.consumeAll(). When the request is not fully consumed on a persistent connection, it calls _channel.abort(new IOException("unconsumed input")).
  • On Windows, the reset discards the response before the client reads it. The client sees An established connection was aborted, retries on a new connection, and the retry reads the InputStreamPart the first attempt already closed: IOException: Stream closed.
  • Fix: the servlet reads the request to EOF before it answers. For requests with a Content-Length, that read returns EOF at once.

Locally, on Windows with JDK 17, a temporary @RepeatedTest of the chunked upload failed once in 492 repetitions without the change, with the same Stream closed and leak. It passed 1,400 repetitions with it.

2. ByteBuf leak in BodyChunkedInput (the cascade)

  • BodyChunkedInput.readChunk allocated a chunk and did not release it when Body.transferTo threw. ChunkedWriteHandler releases only chunks it is handed, so any failing request body read on HTTP/1.1 leaked a buffer.
  • It now releases the chunk and rethrows the original exception, as the HTTP/2 path (NettyBodyBody.BodyChunkSource.nextChunk) already does.
  • New BodyChunkedInputTest: a body that writes part of a chunk and then fails. It checks that the exception is rethrown as is and that the buffer's refCnt() is 0. Without the fix, refCnt() is 1.

Not in this PR

A retry of a multipart request whose InputStreamPart was already consumed still fails with Stream closed rather than a clear message. Redirects already refuse such bodies (Redirect30xInterceptor.ensureBodyReplayable), and plain InputStream bodies fail fast since #2312. Doing the same for retries changes retry policy, so it belongs in a separate change.

Verification

  • mvnw.cmd -B -ntp clean verify -Dgpg.skip=true on JDK 17.0.2 (Windows): 1,940 tests, 0 failures, 0 errors, 35 skipped. Error Prone, NullAway, Javadoc and Revapi passed. Local verify ran on JDK 17; CI covers JDK 11. -Dgpg.skip is set only because gpg is not installed on this machine.
  • No public API change.

Second-agent review

Both reviewers ran read-only, in fresh sessions, on git diff upstream/main...HEAD.

Reviewer Agent and model Effort Commit reviewed Result
Guided (against AGENTS.md) Codex CLI 0.161.0, gpt-6-astra medium 03221ef63 No actionable findings
Independent Claude Code 2.1.316, claude-opus-5-5 medium 03221ef63 No actionable findings

Notes the reviewers raised and that were left as they are:

  • The default branch of the switch in readChunk throws without releasing the chunk, but it cannot be reached for the three-value BodyState.
  • If the servlet's parser threw FileUploadException on a malformed body from a client that keeps the connection open, the drain would wait for Jetty's idle timeout. No test sends such a body.
  • Codex notes that local verification ran on JDK 17, not JDK 11; CI covers JDK 11.

Claude Code on behalf of Pavel Ptashyts

🤖 Generated with Claude Code

MultipartUploadTest failed on windows-latest JDK 11 (run 42294836025
on PR AsyncHttpClient#2378), and a leaked ByteBuf then failed every later test class.

Two causes. The test servlet answered once commons-fileupload reached
the closing boundary, which for a chunked body can come before the
terminating chunk. Jetty 11 then aborts the connection for unconsumed
input (HttpConnection.onCompleted), and on Windows the reset discards
the response before the client reads it. The client replayed the
request on a new connection, and the replay read the InputStreamPart
the first attempt had already closed: "Stream closed". The servlet now
reads the request to the end before it answers. Locally, on Windows,
the chunked upload failed once in 492 repetitions without that and
passed 1,400 repetitions with it.

BodyChunkedInput.readChunk allocated a chunk and did not release it
when Body.transferTo threw, so that IOException leaked the buffer.
ChunkedWriteHandler releases only chunks it is handed. The HTTP/2 body
pump already releases its buffer on failure.

Local mvnw.cmd clean verify ran on JDK 17; CI covers JDK 11.

Claude Code on behalf of Pavel Ptashyts

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant