Skip to content

feat: opt-in small-segment ServerHello (network.client-mss) - #580

Open
AndreyOsipuk wants to merge 1 commit into
9seconds:masterfrom
AndreyOsipuk:feat/client-mss-upstream
Open

AndreyOsipuk wants to merge 1 commit into
9seconds:masterfrom
AndreyOsipuk:feat/client-mss-upstream

Conversation

@AndreyOsipuk

Copy link
Copy Markdown

Some DPI boxes recognise the FakeTLS ServerHello when it arrives in full-size TCP segments and then block the server ip:port, but let the same bytes through when they come in small segments. We hit this on a Russian ISP (Rostelecom): replaying a real ClientHello of Telegram iOS against mtg, only 3 of 8 handshakes completed and the port got banned; with the ServerHello split into 80-byte segments all 8 of 8 completed and the port was not banned. Keeping a small MSS for the whole session works too, but it is slow and multiplies packets per second, so this PR splits only the ServerHello.

Disabled by default:

[network]
client-mss = 92         # 0 (default) disables; 48..1460
client-mss-bulk = 1400  # 0 or 536..65495, 1400 by default when client-mss is set

How it works (Linux only, a no-op with a warning elsewhere):

  • client-mss-bulk > 0: the listening socket gets TCP_MAXSEG = client-mss-bulk; accepted connections inherit it. The ServerHello is written in chunks that fit one segment at MSS = client-mss (minus the timestamps option when it is negotiated: 80 bytes for 92), and each next chunk is written only after the previous one has left the socket queue (tcpi_notsent_bytes == 0). Without that wait the kernel merges chunks once it hits the congestion window or autocorking: at 100 ms RTT 4 KB written in 80-byte chunks left in 13 segments, with the wait in 52. The wait is bounded by the handshake timeout. After the ServerHello nothing changes, the rest of the session uses full-size segments.
  • client-mss-bulk = 0: TCP_MAXSEG = client-mss on the listening socket for the whole session, the same as iptables ... TCPMSS --set-mss on the client SYN (client-mss must then be at least 88, the kernel minimum).
  • Works through the PROXY protocol wrapper (the underlying *net.TCPConn is used); for a non-TCP connection the ServerHello is written as before.

Why not "a small MSS on the listener, then raise it with setsockopt(TCP_MAXSEG)" (or the other way round): on Linux the segment size is fixed during the TCP handshake (mss_clamp), and setsockopt(TCP_MAXSEG) on an established connection only changes user_mss, which no longer affects outgoing segments. Checked on 6.12: listener MSS 92, then setsockopt 1400 - tcpi_snd_mss stays at 80. There is a test for this (TestTCPMaxSegCannotBeRaisedAfterAccept). Hence the bulk MSS goes on the listener and only the ServerHello is split in user space.

The idea and the option names follow client_mss / client_mss_bulk in telemt. This is a separate implementation in Go; the chunking mechanism differs (here the socket MSS is not touched after accept, chunks are paced by tcpi_notsent_bytes).

Documented in example.config.toml. The listener options are read at start.

Tests (Linux, real loopback sockets):

  • the writer: a 4 KB ServerHello leaves in at least one segment per 80-byte chunk while the session stays at tcpi_snd_mss >= 1000; chunks are not merged when the client has a 2 KB receive window and does not read for 300 ms; a real fake.SendServerHello through the writer is parsed as three TLS records; PROXY protocol unwrapping; fallback for non-TCP;
  • the listener: TCP_MAXSEG is advertised in the SYN-ACK, 0 keeps the default, too small values are rejected, and the MSS cannot be raised after accept;
  • end-to-end through the proxy with a real ClientHello snapshot: without client-mss the ServerHello (3-4.5 KB, the size is randomised) arrives in 2 segments, with client-mss = 92 in about 50-60;
  • config parsing and validation (bounds, explicit client-mss-bulk = 0, defaults) and the mapping of the options to listener/ServerHello MSS.

gofmt, go vet ./..., go test -race ./..., GOOS=windows go build ./..., GOOS=darwin go vet ./... and golangci-lint run (v2.12.2) pass.

Some DPI boxes (seen on a Russian ISP) recognise the FakeTLS ServerHello
when it arrives in full-size TCP segments and then block the server
ip:port, but let it through when the same bytes come in small segments,
as with a client MSS of 92. Keeping a small MSS for the whole session is
slow and costs a lot of packets per second, so only the ServerHello is
split.

[network] client-mss (0 disables, 48..1460) and client-mss-bulk (0 or
536..65495, 1400 by default):

- bulk > 0: TCP_MAXSEG = bulk on the listening socket; the ServerHello is
  written in chunks that fit one segment at MSS = client-mss, each chunk
  only after the previous one has left the socket queue
  (tcpi_notsent_bytes = 0), otherwise the kernel merges chunks when it
  hits the congestion window or autocorking;
- bulk = 0: TCP_MAXSEG = client-mss on the listening socket for the whole
  session, like iptables TCPMSS on the client SYN.

The order cannot be reversed (small MSS on the listener, then a larger
one): on an established connection setsockopt(TCP_MAXSEG) does not change
the segment size, mss_clamp is fixed during the TCP handshake (listener
92, then 1400 - tcpi_snd_mss stays at 80). Linux only, a no-op elsewhere.
Disabled by default.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant