Repository navigation
feat: opt-in small-segment ServerHello (network.client-mss) - #580
Open
AndreyOsipuk wants to merge 1 commit into
Open
AndreyOsipuk wants to merge 1 commit into
AndreyOsipuk wants to merge 1 commit into
Conversation
Some DPI boxes (seen on a Russian ISP) recognise the FakeTLS ServerHello when it arrives in full-size TCP segments and then block the server ip:port, but let it through when the same bytes come in small segments, as with a client MSS of 92. Keeping a small MSS for the whole session is slow and costs a lot of packets per second, so only the ServerHello is split. [network] client-mss (0 disables, 48..1460) and client-mss-bulk (0 or 536..65495, 1400 by default): - bulk > 0: TCP_MAXSEG = bulk on the listening socket; the ServerHello is written in chunks that fit one segment at MSS = client-mss, each chunk only after the previous one has left the socket queue (tcpi_notsent_bytes = 0), otherwise the kernel merges chunks when it hits the congestion window or autocorking; - bulk = 0: TCP_MAXSEG = client-mss on the listening socket for the whole session, like iptables TCPMSS on the client SYN. The order cannot be reversed (small MSS on the listener, then a larger one): on an established connection setsockopt(TCP_MAXSEG) does not change the segment size, mss_clamp is fixed during the TCP handshake (listener 92, then 1400 - tcpi_snd_mss stays at 80). Linux only, a no-op elsewhere. Disabled by default.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Some DPI boxes recognise the FakeTLS ServerHello when it arrives in full-size TCP segments and then block the server
ip:port, but let the same bytes through when they come in small segments. We hit this on a Russian ISP (Rostelecom): replaying a real ClientHello of Telegram iOS against mtg, only 3 of 8 handshakes completed and the port got banned; with the ServerHello split into 80-byte segments all 8 of 8 completed and the port was not banned. Keeping a small MSS for the whole session works too, but it is slow and multiplies packets per second, so this PR splits only the ServerHello.Disabled by default:
How it works (Linux only, a no-op with a warning elsewhere):
client-mss-bulk > 0: the listening socket getsTCP_MAXSEG = client-mss-bulk; accepted connections inherit it. The ServerHello is written in chunks that fit one segment at MSS =client-mss(minus the timestamps option when it is negotiated: 80 bytes for 92), and each next chunk is written only after the previous one has left the socket queue (tcpi_notsent_bytes == 0). Without that wait the kernel merges chunks once it hits the congestion window or autocorking: at 100 ms RTT 4 KB written in 80-byte chunks left in 13 segments, with the wait in 52. The wait is bounded by the handshake timeout. After the ServerHello nothing changes, the rest of the session uses full-size segments.client-mss-bulk = 0:TCP_MAXSEG = client-msson the listening socket for the whole session, the same asiptables ... TCPMSS --set-msson the client SYN (client-mssmust then be at least 88, the kernel minimum).*net.TCPConnis used); for a non-TCP connection the ServerHello is written as before.Why not "a small MSS on the listener, then raise it with
setsockopt(TCP_MAXSEG)" (or the other way round): on Linux the segment size is fixed during the TCP handshake (mss_clamp), andsetsockopt(TCP_MAXSEG)on an established connection only changesuser_mss, which no longer affects outgoing segments. Checked on 6.12: listener MSS 92, thensetsockopt1400 -tcpi_snd_mssstays at 80. There is a test for this (TestTCPMaxSegCannotBeRaisedAfterAccept). Hence the bulk MSS goes on the listener and only the ServerHello is split in user space.The idea and the option names follow
client_mss/client_mss_bulkin telemt. This is a separate implementation in Go; the chunking mechanism differs (here the socket MSS is not touched after accept, chunks are paced bytcpi_notsent_bytes).Documented in
example.config.toml. The listener options are read at start.Tests (Linux, real loopback sockets):
tcpi_snd_mss>= 1000; chunks are not merged when the client has a 2 KB receive window and does not read for 300 ms; a realfake.SendServerHellothrough the writer is parsed as three TLS records; PROXY protocol unwrapping; fallback for non-TCP;TCP_MAXSEGis advertised in the SYN-ACK, 0 keeps the default, too small values are rejected, and the MSS cannot be raised after accept;client-mssthe ServerHello (3-4.5 KB, the size is randomised) arrives in 2 segments, withclient-mss = 92in about 50-60;client-mss-bulk = 0, defaults) and the mapping of the options to listener/ServerHello MSS.gofmt,go vet ./...,go test -race ./...,GOOS=windows go build ./...,GOOS=darwin go vet ./...andgolangci-lint run(v2.12.2) pass.