Skip to content

Drop held input of a body cut off by the close - #2386

Open
mkurz wants to merge 1 commit into
AsyncHttpClient:mainfrom
mkurz:fix/decompression-premature-close
Open

mkurz wants to merge 1 commit into
AsyncHttpClient:mainfrom
mkurz:fix/decompression-premature-close

Conversation

@mkurz

@mkurz mkurz commented Nov 7, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Part of the second follow-up promised on fix(netty): stop inflating a suspended HTTP/1.1 response #2362 (discussion), output left over when the connection closes.
  • When the connection closes while a suspended HTTP/1.1 response holds compressed input back, and the body has not ended, that input is now dropped instead of decompressed and handed on at once. That response cannot complete anyway.
  • A body that has ended is handed on at the close as before. That is the case from the discussion, so this does not finish the follow-up; see "Still open" below.

Problem

Http1ContentDecompressor holds compressed input back while a response is suspended. When the connection closes, Netty tears the pipeline down in the same task, so on main everything held back is decompressed and handed on at once, bounded only by maxDecompressedResponseSize.

For a body that the close cut off, a Content-Length or chunked body that never reached its end, that is wasted work: the response cannot complete, and AHC fails the request (RemotelyClosedException) or retries it right after. A suspended handler still receives the whole remainder first.

Whether an orderly close by the server (a TCP FIN) is noticed while the response is suspended depends on the transport. A handler that suspends after its first part and never resumes, 16 MiB of zeros as gzip, and a server that closes the connection:

Transport Body cut in half Complete body, then close Body delimited by the close
NIO 1 part; the close is not read yet 1 part 1 part
epoll, io_uring on main 129 parts (8 MiB), then RemotelyClosedException 257 parts (16 MiB), completed 257 parts (16 MiB), completed
epoll, io_uring with this PR 1 part, then RemotelyClosedException unchanged unchanged

NIO honors auto-read, so it notices the server's close only once the input held back has been used up. epoll keeps reading after EPOLLRDHUP regardless of auto-read (EpollRecvByteAllocatorHandle.continueReading), and io_uring behaves the same here. AHC selects epoll on Linux whenever the native library is on the classpath. A close on the client's side, for example when the client is closed, reaches the decompressor on every transport.

Change

  • In channelInactive, the input held back is decompressed only if the end of the body has arrived: a LastHttpContent held with the input, or queued behind it. That covers the end of a body delimited by the connection, which HttpObjectDecoder hands on just before the close, and a last chunk that failed to parse, which still fails the request with its own error. Otherwise the input is dropped with the rest of the response state.
  • A dropped body is not decompressed first, so a corrupt or oversized remainder no longer fails it with a DecompressionException, which is never retried. The request is handled like any other whose connection closed before the end of its body: it is retried (maxRequestRetry, or an IOExceptionFilter that replays it) or fails with a RemotelyClosedException, and the IOExceptionFilters see a ChannelClosedException.
  • The Javadoc of ResponseBodyControl.suspend() and of the class say so. suspend() also says that native transports notice an orderly close by the server while reads are suspended, and NIO only once the input held back has been used up.

Still open: a body that has ended, including one delimited by the connection, is still decompressed and handed on at once when the connection closes while its input is held back, on native transports, bounded by maxDecompressedResponseSize as the Javadoc already said. Keeping it to demand needs the input held back to outlive the close. Either the channel allows half-closure, so that the server's close only shuts its input down and the pipeline stays until the body has been delivered and the channel is closed, or the input held back is moved out of the pipeline before it is torn down. Either is a larger change, left for a separate PR. HTTP/2 is separate.

AI disclosure

Claude Code on behalf of Matthias Kurz. The commit includes Co-Authored-By per AGENTS.md.

Test plan

  • Fail on main, pass with this PR:
    • Http1ContentDecompressorSuspensionTest.inputHeldBackOfABodyThatTheCloseCutsOffIsDropped (gzip, deflate, br, zstd, snappy): a body without its end, suspended after its first part, with a limit that decompressing the rest would exceed. At the close nothing more is handed on, no error, every input buffer released, and channelInactive passed on. On main, all five fail with the size limit.
    • Http1DecompressionSuspensionTest.aSuspendedBodyThatTheConnectionCutsOffFailsWithoutTheRestOfIt (Linux with epoll; a Content-Length and a chunked body, half of it sent, then closed): the handler gets 1 part, the request fails with RemotelyClosedException, and the IOExceptionFilter sees one ChannelClosedException. On main, it fails with the size limit's DecompressionException.
  • Pass on main and with this PR:
    • Http1ContentDecompressorSuspensionTest.inputHeldBackOfABodyThatHasEndedIsHandedOnAtTheClose (gzip, zstd; the end of a body delimited by the connection, and a failed last chunk): the whole body is handed on at the close, then the end with its decoder result.
    • channelClosureHandsOnHeldInputBeforeChannelInactive, aTruncatedBodyHeldWhenTheConnectionClosesFails and aHeldDirectInputIsReleasedWhenTheResponseEndsEarly, for a held last chunk.
  • ./mvnw -B -ntp -Dgpg.skip=true clean verify on JDK 11: 2,003 tests, 0 failures, 0 errors, 11 skipped; Revapi passed. No test-skipping flags were used.
  • The Http1ContentDecompressor*Test, Http2ContentDecompressorTest, Http1Decompression*Test and ResponseBodyControlTest classes compiled on JDK 11 and run on JDK 17, 21 and 25 (Maven Surefire's -Djvm): 134 tests each, 0 failures.
  • aSuspendedBodyThatTheConnectionCutsOffFailsWithoutTheRestOfIt, 5 more runs: all passed.

When the connection closed while a suspended response held compressed
input back, Http1ContentDecompressor decompressed all of it at once and
handed it on. For a body that the close cut off, that work is wasted:
the response cannot complete, and the request fails or is retried
right after. The input held back is now dropped unless the end of the
body has arrived, either held with the input or queued behind it.

A body that has ended is handed on at the close as before: a held last
chunk, the end of a body delimited by the connection, and a last chunk
that failed to parse.

A dropped body is not decompressed first, so it no longer fails on a
corrupt or oversized remainder. The request is handled like any other
whose connection closed before the end of its body: it is retried or
fails with a RemotelyClosedException, and the IOExceptionFilters see a
ChannelClosedException.

Whether an orderly close by the server (a TCP FIN) is noticed while
reads are suspended depends on the transport: native transports such
as epoll notice it, NIO only once the input held back has been used
up. A close on the client's side, for example when the client is
closed, reaches the decompressor on every transport.

Claude Code on behalf of Matthias Kurz

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@mkurz

mkurz commented Nov 7, 2026

Copy link
Copy Markdown
Contributor Author

I also have a local prototype for what is still open here: a body that has ended, held back when the server closes the connection in an orderly way (a TCP FIN). While compressed input is held back, it turns on half-closure, so that close only shuts the input down. The rest of the body then still waits for demand, and the connection is not reused. Tests cover epoll and io_uring, over plaintext and TLS; kqueue is untested. A reset, a close on the client's side and HTTP/2 are not covered. Would you like it as a separate PR after this one?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant