Repository navigation
Drop undecodable codings from Accept-Encoding - #2381
Open
pavel-ptashyts wants to merge 6 commits into
Open
pavel-ptashyts wants to merge 6 commits into
pavel-ptashyts wants to merge 6 commits into
Conversation
With automatic decompression on, a caller-set Accept-Encoding is meant to lose br and zstd when their native libraries are missing. Three bugs defeated that: - The zstd filter ran on the original value and overwrote the Brotli result, so with both libraries missing (both are optional dependencies) br was offered again. This dates from AsyncHttpClient#1950. - Each filter only cut a trailing ", br" or ", zstd". The browser-style "gzip, deflate, br, zstd" kept br even with the filters chained, and so did a coding listed first, without a space, in another case or with a weight. - Only the first Accept-Encoding field was read, and whenever either library was missing the header was replaced by that field alone, dropping the caller's other fields. A server that picks br or zstd then sends a body AHC cannot decode: the response decompressor leaves such a body as it is, so the handler receives compressed bytes where it expects decoded ones. Filter each list element by its coding token, chain the filters on the running value, and filter all fields as one list, setting the header only when something was dropped. The check moves into a package-private helper so tests can drive it with both libraries missing, as both are on the test classpath. Claude Code on behalf of Pavel Ptashyts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Resolve the HttpUtilsTest conflict by keeping both the cookie pair test from upstream and the Accept-Encoding filter tests, and spell the tab in one filter test as an escape instead of a raw tab. Claude Code on behalf of Pavel Ptashyts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Bring the branch up to date with main before opening the pull request. Claude Code on behalf of Pavel Ptashyts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The filters dropped every element naming br or zstd, including one with weight 0. Such an element refuses the coding, so dropping it next to a wildcard turned "*, br;q=0" into "*" and let a server pick the coding AHC cannot decode, while main left that header alone. Elements with q=0, in any spelling RFC 9110 allows, are now kept. A single Accept-Encoding field, the common case, is also no longer copied through String.join on every request when a library is missing. Found by the independent review of AsyncHttpClient#2381. Claude Code on behalf of Pavel Ptashyts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Dropping an element that offers br or zstd with a non-zero weight let a "*" element offer the coding at the wildcard's weight instead, so "*, zstd;q=0.5" became "*" and raised zstd from 0.5 to 1. When the list has a "*" that offers codings, the dropped elements are now replaced by one explicit "<coding>;q=0" refusal. Found by the second independent review of AsyncHttpClient#2381. Claude Code on behalf of Pavel Ptashyts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
hyperxpro
requested changes
Nov 7, 2026
Brotli and zstd are both on the test classpath, so nothing exercised the filter call in newNettyRequest: removing it, swapping its arguments or dropping the automatic-decompression check left every test green, as review pointed out. The factory now reads Brotli.isAvailable() and Zstd.isAvailable() once in its constructor, and a package-private constructor takes both flags. New tests build requests through newNettyRequest with each combination of the flags, with automatic decompression on and off, and with compression enforced. Claude Code on behalf of Pavel Ptashyts Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
Author
|
The failing What happened in that job:
The other 12 checks passed. #2383 fixes this: the test servlet drains the request before answering, and Claude Code on behalf of Pavel Ptashyts |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
With automatic decompression on, a caller-set
Accept-Encodingis meant to losebrandzstdwhen their native libraries are missing. Three bugs defeated that, sobr(and in some caseszstd) was still offered:brcame back. This dates from support zstd compression #1950.", br"or", zstd". The browser-stylegzip, deflate, br, zstdkeptbreven with the filters chained, and so did a coding listed first, without a space, in another case or with a weight.Accept-Encodingfield was read, and whenever either library was missing the header was replaced by that field alone, dropping the caller's other fields.Impact: a server that picks
brorzstdsends a body AHC cannot decode.PullDecompressionleaves such a body as it is, so the handler receives compressed bytes where it expects decoded ones.Change
HttpUtils.filterOutBrotliFromAcceptEncoding/filterOutZstdFromAcceptEncodingnow drop every list element that offers the coding, wherever it is and in any case or spacing (RFC 9110 section 12.5.3). Signatures are unchanged; the value is returned as is when nothing is dropped. Dropping the only element leaves an empty value, which accepts no content coding.br;q=0,BR ; Q = 0.000) refuses the coding and is kept: dropping it next to*would let the server pick the coding, whilemainleft*, br;q=0alone.*that offers codings, the dropped elements are replaced by one<coding>;q=0refusal, so*, zstd;q=0.5becomes*, zstd;q=0rather than*, which would raise zstd to the wildcard's weight.NettyRequestFactorymoves the check into a package-privatefilterOutUndecodableAcceptEncodings(headers, brotliAvailable, zstdAvailable)that chains the filters on the running value and filters all fields as one comma-joined list (RFC 9110 section 5.3), without joining when there is one field. It returns at once when both libraries are available and sets the header only when something was dropped.Brotli.isAvailable()andZstd.isAvailable()once, in its constructor; a package-private constructor takes both flags so tests can drivenewNettyRequestwith either library missing.No public API change.
Tests
Brotli and zstd are both on the test classpath, so
NettyRequestFactoryAcceptEncodingTestpasses the availability flags to the helper and to the factory. ItsnewNettyRequesttests cover each combination of the flags with automatic decompression on, the header left untouched with it off, and the enforced-compression header.HttpUtilsTestcovers position, separators, case, weights, zero-weight refusals and wildcards. Onmain7 of the new assertions fail, for examplegzip, deflate, br, zstdwith neither library becomesgzip, deflate, br; the zero-weight and wildcard tests each failed before their fix.mvnw.cmd clean verifypasses locally on JDK 17 (no JDK 11 on the host): 1953 tests, 0 failures, Revapi green. CI covers JDK 11.Not changed
*still offers every coding the list does not refuse.gzip, *is left as is, andbr, *with both libraries missing becomesbr;q=0, *, where the*still coverszstd. Left for a separate change.Http2ContentDecompressordecodes only gzip and deflate, so abrorzstdbody reaches the handler compressed even with the library present. Needs its own issue.HttpContentCompressormay still answergzip, br;q=0, *withbr.maingives the same result, andbr;q=0is the refusal RFC 9110 section 12.5.3 defines, so it is kept.Second-agent review
Two agent reviewers, each a fresh read-only CLI run on
git diff upstream/main...HEAD, reasoning effort medium; the model and effort below are as reported by each run.gpt-6-astraclaude-opus-5-5*, br;q=0became*, letting a server pickbr(fixed in 93f51c0);String.joinper request for one field (fixed in 93f51c0)*, zstd;q=0.5became*, raising zstd's weight (fixed in d3a31c6)Optional notes from pass 3 left as they are:
*;q=0, broridentity;q=0, brwith Brotli missing becomes*;q=0/identity;q=0, which accepts nothing, so a strict server may answer 406. The reviewer marked it optional and not a regression (maingave the same result). Not changed: rewriting the caller's explicitidentity;q=0would override their intent, and a 406 is easier to diagnose than a body that arrives still compressed.headers.getAllstill allocates a list per request when a library is missing. The reviewer marked it as needing no action: the path is taken only then, and the filters return early when the coding is absent.clean verifyon JDK 11; locally it ran on JDK 17, and CI covers JDK 11.Claude Code on behalf of Pavel Ptashyts
🤖 Generated with Claude Code