Repository navigation
Conversation
When the connection closed while a suspended response held compressed input back, Http1ContentDecompressor decompressed all of it at once and handed it on. For a body that the close cut off, that work is wasted: the response cannot complete, and the request fails or is retried right after. The input held back is now dropped unless the end of the body has arrived, either held with the input or queued behind it. A body that has ended is handed on at the close as before: a held last chunk, the end of a body delimited by the connection, and a last chunk that failed to parse. A dropped body is not decompressed first, so it no longer fails on a corrupt or oversized remainder. The request is handled like any other whose connection closed before the end of its body: it is retried or fails with a RemotelyClosedException, and the IOExceptionFilters see a ChannelClosedException. Whether an orderly close by the server (a TCP FIN) is noticed while reads are suspended depends on the transport: native transports such as epoll notice it, NIO only once the input held back has been used up. A close on the client's side, for example when the client is closed, reaches the decompressor on every transport. Claude Code on behalf of Matthias Kurz Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
Author
|
I also have a local prototype for what is still open here: a body that has ended, held back when the server closes the connection in an orderly way (a TCP FIN). While compressed input is held back, it turns on half-closure, so that close only shuts the input down. The rest of the body then still waits for demand, and the connection is not reused. Tests cover epoll and io_uring, over plaintext and TLS; kqueue is untested. A reset, a close on the client's side and HTTP/2 are not covered. Would you like it as a separate PR after this one? |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Problem
Http1ContentDecompressorholds compressed input back while a response is suspended. When the connection closes, Netty tears the pipeline down in the same task, so onmaineverything held back is decompressed and handed on at once, bounded only bymaxDecompressedResponseSize.For a body that the close cut off, a
Content-Lengthor chunked body that never reached its end, that is wasted work: the response cannot complete, and AHC fails the request (RemotelyClosedException) or retries it right after. A suspended handler still receives the whole remainder first.Whether an orderly close by the server (a TCP FIN) is noticed while the response is suspended depends on the transport. A handler that suspends after its first part and never resumes, 16 MiB of zeros as gzip, and a server that closes the connection:
mainRemotelyClosedExceptionRemotelyClosedExceptionNIO honors auto-read, so it notices the server's close only once the input held back has been used up. epoll keeps reading after
EPOLLRDHUPregardless of auto-read (EpollRecvByteAllocatorHandle.continueReading), and io_uring behaves the same here. AHC selects epoll on Linux whenever the native library is on the classpath. A close on the client's side, for example when the client is closed, reaches the decompressor on every transport.Change
channelInactive, the input held back is decompressed only if the end of the body has arrived: aLastHttpContentheld with the input, or queued behind it. That covers the end of a body delimited by the connection, whichHttpObjectDecoderhands on just before the close, and a last chunk that failed to parse, which still fails the request with its own error. Otherwise the input is dropped with the rest of the response state.DecompressionException, which is never retried. The request is handled like any other whose connection closed before the end of its body: it is retried (maxRequestRetry, or anIOExceptionFilterthat replays it) or fails with aRemotelyClosedException, and theIOExceptionFilters see aChannelClosedException.ResponseBodyControl.suspend()and of the class say so.suspend()also says that native transports notice an orderly close by the server while reads are suspended, and NIO only once the input held back has been used up.Still open: a body that has ended, including one delimited by the connection, is still decompressed and handed on at once when the connection closes while its input is held back, on native transports, bounded by
maxDecompressedResponseSizeas the Javadoc already said. Keeping it to demand needs the input held back to outlive the close. Either the channel allows half-closure, so that the server's close only shuts its input down and the pipeline stays until the body has been delivered and the channel is closed, or the input held back is moved out of the pipeline before it is torn down. Either is a larger change, left for a separate PR. HTTP/2 is separate.AI disclosure
Claude Code on behalf of Matthias Kurz. The commit includes
Co-Authored-ByperAGENTS.md.Test plan
main, pass with this PR:Http1ContentDecompressorSuspensionTest.inputHeldBackOfABodyThatTheCloseCutsOffIsDropped(gzip, deflate, br, zstd, snappy): a body without its end, suspended after its first part, with a limit that decompressing the rest would exceed. At the close nothing more is handed on, no error, every input buffer released, andchannelInactivepassed on. Onmain, all five fail with the size limit.Http1DecompressionSuspensionTest.aSuspendedBodyThatTheConnectionCutsOffFailsWithoutTheRestOfIt(Linux with epoll; aContent-Lengthand a chunked body, half of it sent, then closed): the handler gets 1 part, the request fails withRemotelyClosedException, and theIOExceptionFiltersees oneChannelClosedException. Onmain, it fails with the size limit'sDecompressionException.mainand with this PR:Http1ContentDecompressorSuspensionTest.inputHeldBackOfABodyThatHasEndedIsHandedOnAtTheClose(gzip, zstd; the end of a body delimited by the connection, and a failed last chunk): the whole body is handed on at the close, then the end with its decoder result.channelClosureHandsOnHeldInputBeforeChannelInactive,aTruncatedBodyHeldWhenTheConnectionClosesFailsandaHeldDirectInputIsReleasedWhenTheResponseEndsEarly, for a held last chunk../mvnw -B -ntp -Dgpg.skip=true clean verifyon JDK 11: 2,003 tests, 0 failures, 0 errors, 11 skipped; Revapi passed. No test-skipping flags were used.Http1ContentDecompressor*Test,Http2ContentDecompressorTest,Http1Decompression*TestandResponseBodyControlTestclasses compiled on JDK 11 and run on JDK 17, 21 and 25 (Maven Surefire's-Djvm): 134 tests each, 0 failures.aSuspendedBodyThatTheConnectionCutsOffFailsWithoutTheRestOfIt, 5 more runs: all passed.