Repository navigation
feat: split only the beginning of the FakeTLS response (network.client-mss-bytes) - #591
Open
AndreyOsipuk wants to merge 7 commits into
Open
AndreyOsipuk wants to merge 7 commits into
AndreyOsipuk wants to merge 7 commits into
Conversation
Some DPI boxes (seen on a Russian ISP) recognise the FakeTLS ServerHello when it arrives in full-size TCP segments and then block the server ip:port, but let it through when the same bytes come in small segments, as with a client MSS of 92. Keeping a small MSS for the whole session is slow and costs a lot of packets per second, so only the ServerHello is split. [network] client-mss (0 disables, 48..1460) and client-mss-bulk (0 or 536..65495, 1400 by default): - bulk > 0: TCP_MAXSEG = bulk on the listening socket; the ServerHello is written in chunks that fit one segment at MSS = client-mss, each chunk only after the previous one has left the socket queue (tcpi_notsent_bytes = 0), otherwise the kernel merges chunks when it hits the congestion window or autocorking; - bulk = 0: TCP_MAXSEG = client-mss on the listening socket for the whole session, like iptables TCPMSS on the client SYN. The order cannot be reversed (small MSS on the listener, then a larger one): on an established connection setsockopt(TCP_MAXSEG) does not change the segment size, mss_clamp is fixed during the TCP handshake (listener 92, then 1400 - tcpi_snd_mss stays at 80). Linux only, a no-op elsewhere. Disabled by default.
…enerMSS NewListener no longer takes the listener MSS. utils.SetListenerMSS sets TCP_MAXSEG on the already listening socket right after NewListener and before the PROXY protocol wrapper; accepted connections inherit it the same way. This keeps the listener constructor free for other changes.
… client-mss The small-segment ServerHello writer now waits until the same handshake deadline that is set on the client socket, so a client that never reads cannot hold the stream past the handshake timeout. Tests: a non-reading client with a tiny receive window (writer and full proxy), the handshake behind the PROXY protocol listener (v1 and v2), tcpi_rcv_mss as an unprivileged signal of the largest received segment, and an AF_PACKET loopback capture that checks every ServerHello segment (skipped without CAP_NET_RAW).
… queue Without MSG_EOR the kernel may collapse adjacent small skbs on retransmission (tcp_retrans_collapse is on by default), so on a lossy path the retransmitted ServerHello left in full-size segments. Polling tcpi_notsent_bytes between writes did not prevent that. Each chunk is now one sendmsg(MSG_EOR|MSG_NOSIGNAL) through RawConn. EAGAIN waits on the poller under the handshake deadline, EINTR retries, and a partial send finishes the current chunk, so chunk boundaries stay fixed. Tests: a fake sendmsg checks the flags and boundaries of every call, including partial sends, EINTR and EAGAIN; a full send buffer on a real socket; and a retransmission test that drops incoming packets with a socket BPF filter and checks that retransmitted segments are not larger than a chunk (fails on the previous code: 1360 bytes in 3 segments).
…t-mss-bytes) client-mss split the whole server response - ServerHello, ChangeCipherSpec and 2.5-4.7 KB of ApplicationData noise - into 35-60 small segments. On a path with 2% loss at least one of them is lost in about 64% of handshakes (about 6% for three segments), and with IW10 the response takes extra round trips of slow start. Now only the first client-mss-bytes bytes are split; by default (0) that is the ServerHello record itself (~130 bytes, two segments at client-mss = 92). The rest leaves in one sendmsg at the bulk MSS. Every small chunk keeps MSG_EOR, so the rest is never appended to the last one. A value above the response size (for example 65535) restores splitting of the whole response. The option is ignored with client-mss-bulk = 0.
AndreyOsipuk
force-pushed
the
feat/client-mss-prefix
branch
from
November 9, 2026 13:23
68962c6 to
f939db4
Compare
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## master #591 +/- ##
==========================================
+ Coverage 68.31% 69.54% +1.23%
==========================================
Files 112 118 +6
Lines 4087 4279 +192
==========================================
+ Hits 2792 2976 +184
+ Misses 1146 1143 -3
- Partials 149 160 +11 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Builds on #580 (its commits are included here; the change of this PR is the last commit, f939db4). Happy to rebase once #580 is merged.
Problem.
client-msscurrently splits the whole server response - ServerHello, ChangeCipherSpec and 2.5-4.7 KB of ApplicationData noise - into 35-60 small segments. On a path with 2% loss at least one of them is lost in roughly 64% of handshakes (vs ~6% for three segments), and with IW10 sending the response takes a few extra round trips of slow start. For DPI resistance it is enough that the ServerHello record does not leave in a single packet.Change. Only the first
client-mss-bytesbytes are split; by default (0) that is the ServerHello record itself (~130 bytes, 2 segments atclient-mss = 92). The rest goes in one sendmsg in regular segments. Every small chunk keeps MSG_EOR, so the large rest is never appended to the last one, including on retransmission. A value above the response size (e.g. 65535) restores splitting of the whole response. The option is ignored withclient-mss-bulk = 0, where the whole session runs at client-mss anyway.Tests. Unit tests for the split length and chunk boundaries (fake sendmsg); a loopback check that the default response leaves in ~5 segments instead of ~55; an AF_PACKET capture test (CAP_NET_RAW, skipped otherwise) asserting that segments carrying the first N bytes are <= client-mss and end exactly at N, while the rest goes in large segments; config and CLI tests. Documented in
example.config.toml.go vet,go test -race ./...on Linux and golangci-lint pass.