iPhone: FTW Cloud over QUIC, racing the WebSocket (QUIC step 2)

[frahlg]

QUIC, the iPhone's side: the session with FTW Cloud dials QUIC first and falls back to the WebSocket, and nothing above the socket changes. It is paired with srcfl/ftw#1653 (protocol and box), srcfl/ftw-cloud#22 (service) and srcfl/ftw-webapp#91 (registry). main is merged in, with the review's fixes (B2/S5, S1, S2). iPhone 0-RTT stays off.

What changes

• QUICCloudSocket is a WebSocketConnection on iOS 26's NetworkConnection<QUIC>, ALPN ftw-cloud-app/1, to the WebSocket URL's host and port over UDP.
  • The control stream starts with the target (the URL's path and query). After that, each message goes with its QUIC varint length.
  • The box's bulk comes on a stream the service opens and reaches onBinary as before. hb and live may come as datagrams, which reach the new WebSocketEvents.onDatagram.
  • A keepalive every 15 s holds the NAT mapping.
  • The service's close code (4403, 4408…) comes back from applicationError, so CloudCarrier's close handling is the same. Only the control stream and the writer end the session.
• CloudSockets.racing races it against URLSessionWebSocket. QUIC starts first, and the WebSocket starts after 250 ms, or at once if QUIC is refused. The first to open carries the session, and the other is closed. CloudTransportMemory keeps the winner per network for a day, and only the last day's networks. The network is named by NetworkWatch (interfaces and gateways) and kept only as a hash.
• Demotion (B2, S5). A QUIC session that ends before its first message, or that the path drops within a minute of it, demotes QUIC on that network for a day: the WebSocket goes first, and no QUIC win, the learning dial's included, moves QUIC back meanwhile. Closes from 1000 up other than 1005 and 1006 do not count. So a door that finishes the handshake and closes, as one with the sessions off did, no longer holds the phone on QUIC.
• AppModel takes makeCloudSocket. The app passes the race; tests leave it nil and get makeSocket as before. The relay still uses the WebSocket.
• The Noise window (S1) is 1024 (16 words), as appwire.ReplayWindow in srcfl/ftw, since a bulk burst can overtake a control message. CloudCarrier drops a replay only when it came as a datagram; on a stream it ends the session, as a broken message does. It also says how it travels (transportName).
• CloudView (S2) takes the table of a late live whose srcRev is newer and keeps the newer values (Freshness case 17), so a table overtaken by a datagram is not lost.
• FTW Cloud's settings read "Connected · QUIC" or "Connected · WebSocket". The carrier stays tunnel or cloud, and freshness stays carrier plus source state.
• The names come from the registry and the vectors:
  • Contract.cloudTransports and protocol/registry.yaml from registry: cloud_transports, how a session with FTW Cloud travels (QUIC step 2) ftw-webapp#91 (ContractTests checks the new line);
  • CloudWire.alpnApp;
  • cloud-quic-vectors.json from FTW Cloud over QUIC: the protocol, and the box's uplink racing the WebSocket with 0-RTT (QUIC steps 2–3) ftw#1653 as it stands (QUICFramingTests replays it).
• LiveCloudTests' shell helper is macOS only, so the FTWKit tests also build for the simulator.

No release is forced. Against today's service, which has no UDP listener, QUIC gets no answer and the WebSocket opens 250 ms later, once per network per day. An older app keeps the WebSocket.

Numbers: today's WebSocket against QUIC

A real ftwcloudd, a box running Core's own cloudlink, appuplink and command handler, and a test phone in Go (go/protocol and quicdoor.Dial) that pairs with it and speaks the app protocol inside the tunnel. The WebSocket rows are today's path on both legs; the QUIC rows are QUIC on both. Everything runs in one Linux container whose loopback carries netem: 20 ms each way, so every leg has a 40 ms round trip, and the loss applies to each packet on both legs. The readable copy is on. 30 connects and 30 taps per row, 90 s of frames.

The connect rows come from a run with the review's fixes in, where a phone's session waits for the QUIC handshake. The other rows come from the run before the fixes, which do not touch a session once it is open.

link	what	WebSocket p50 / p95 ms	QUIC p50 / p95 ms
40 ms	cold connect → first live number	326 / 333	236 / 248
40 ms	reconnect → first live number	326 / 335	235 / 247
40 ms	tap → ack	101 / 107	97 / 109
40 ms	tunnel frame, box → phone	49 / 55	52 / 58
40 ms	copy live, box → phone	52 / 57 (89 of 90 arrived)	54 / 60 (89 of 90)
40 ms	network switch → live again (Go test phone)	355 (new session)	46 (same session, 0 ticks missed)
40 ms, 1 % loss	cold connect → first live number	326 / 608	260 / 401
40 ms, 1 % loss	reconnect → first live number	327 / 419	265 / 383
40 ms, 1 % loss	tap → ack	104 / 168	101 / 110
40 ms, 1 % loss	tunnel frame, box → phone	50 / 57	52 / 58
40 ms, 1 % loss	copy live, box → phone	51 / 59 (90 of 90)	52 / 59 (88 of 90)
40 ms, 1 % loss	network switch → live again (Go test phone)	375 (new session)	47 (same session, 0 ticks missed)
40 ms, 5 % loss	cold connect → first live number	610 / 2057	380 / 568
40 ms, 5 % loss	reconnect → first live number	434 / 1190	385 / 585
40 ms, 5 % loss	tap → ack	104 / 360	104 / 202
40 ms, 5 % loss	tunnel frame, box → phone	52 / 308	52 / 149
40 ms, 5 % loss	copy live, box → phone	53 / 309 (90 of 90)	52 / 57 (82 of 90)
40 ms, 5 % loss	network switch → live again (Go test phone)	689 (new session)	50 (same session, 0 ticks missed)

What it says: QUIC saves round trips where a connection starts, and it keeps loss on one stream from holding up the others. Cold, it is two round trips less: the TCP handshake and the WebSocket upgrade. A phone's reconnect now costs what a cold connect does, since its session waits for the handshake; 0-RTT still saves a box a round trip, whose challenge comes back in the first flight. Under loss the tails shrink everywhere, and a network switch costs no new session. With no loss, a frame or a tap takes one round trip either way.

The copy's live at 5 % loss is not a clean win. QUIC sends it as a datagram, never again, and lost 8 of the 90; the WebSocket lost none but brought its late ones up to 309 ms after. A lost one shows as a gap in the freshness rules (hb.liveSeq), never as a live value. Neither row measures the age of the newest value at the phone, which is what a person sees.

• Cold connect → first live number: a new connection with no TLS ticket, through the outer Noise handshake (with the box session's message 1 in now.tun), the box session's handshake and hello with its subscription, to the snapshot.
• Reconnect: the same phone again, with the service's ticket. Over QUIC it may send the target and message 1 early, but the session opens only once the handshake is done; the WebSocket resumes TLS.
• Tap → ack: a mode change through the tunnel to the box's cmd.ack.
• Tunnel frame: a lane 0 tick, from the moment the box sent it to its arrival at the phone, on the control stream either way.
• Copy live: the readable copy's live, from the box's wallMs to the phone. Over QUIC it goes as a datagram; the counts say how many of the 90 the box sent arrived.
• Network switch → live again: the Go test phone moves its QUIC connection to a new socket (a new source address, as Wi-Fi to cellular) and the session goes on; the time is the path probe and switch. The WebSocket has to dial a new session; the time is from the switch to its first live number. The iPhone's own migration is not measured.

On the phone: FTWKit in the iOS simulator

LiveQUICTests (in this PR) runs the app's own stack (AppModel, PairModel, CloudCarrier, the tunnel and the session) against a real ftwcloudd and a real Core box with the relay unreachable. It runs in an iPhone 17 simulator. The service and the box run in a container whose netem shapes only the phone's leg, 20 ms each way plus OrbStack's own few ms (ping 56–100 ms); the box reaches the cloud unshaped. The readable copy is off. Each row is one pairing, 30 taps and 6 returns from the background.

phone's link	transport	scan → streaming ms	tap → ack p50 / p95 ms	background → streaming p50 / max ms
40 ms	WebSocket	308	52 / 61	274 / 295
40 ms	QUIC	195	54 / 59	165 / 193
40 ms	race (QUIC won)	167	53 / 59	182 / 187
40 ms, 1 % loss	WebSocket	292	53 / 60	277 / 290
40 ms, 1 % loss	QUIC	171	53 / 59	181 / 321
40 ms, 1 % loss	race (QUIC won)	191	54 / 60	184 / 194
40 ms, 5 % loss	WebSocket	303	55 / 254	286 / 785
40 ms, 5 % loss	QUIC	199	55 / 165	184 / 479
40 ms, 5 % loss	race (QUIC won)	191	53 / 154	191 / 382

The Mac's own swift test gave the same at 40 ms: 305 against 180 ms from scan to stream, and 269 against 184 ms back from the background. In an earlier 5 % run the race gave one session to the WebSocket, because one lost QUIC packet made QUIC miss its 250 ms lead. A QUIC dial that loses now goes on to learn, so that one loss no longer keeps QUIC second for a day.

Keeping an idle box linked

The box uplink stayed idle for 180 s with the readable copy off, at 40 ms, counted packet by packet on the loopback (IP bytes):

transport	what goes every 15 s	bytes per day
WebSocket	service's ping, box's pong, an ACK: 3 packets	1.2 MB
QUIC, once path MTU discovery is done	box's PING (53 B), service's ACK (60 B)	0.65 MB

In the first two minutes, both ends also send path MTU probes of about 1.4 KB beside each keepalive. Over the 180 s window that came to 10 KB, a rate of 4.9 MB a day that stops once discovery ends. A PING every 15 s keeps a 30 s NAT mapping open.

Not checked

• A real phone on LTE, and a real Wi-Fi to cellular move. The switch here is quic-go's own path migration in the Go phone, which proves the service follows the path. Apple's side of it is untested.
• 0-RTT from the iPhone. Network.framework made no TLS resumption against quic-go on macOS, with an IP literal or localhost and a pinned root (DidResume false), so the iPhone's reconnect is a full handshake here. The box and the Go phone use 0-RTT. The simulator with the root in its trust store and no pinning did not resume either: across four app connections, one of them a return from the background, the service logged DidResume false on each. The likely cause is the API. iOS 26's QUIC.TLS has no ticketsEnabled or earlyDataEnabled, though TLS and DTLS have both. The older NWProtocolQUIC.Options exposes securityProtocolOptions, where sec_protocol_options_set_tls_tickets_enabled can turn tickets on. That is a follow-up.
• The age of the newest value at the phone, the cost of a real cellular path, and Apple's QUIC on watchOS.

The rig is not in any repository. It is the box as a test binary in Core's appuplink (with Core's own stubs), a phone in Go over go/protocol and quicdoor.Dial, run.sh, and netem in an Alpine container on OrbStack. Landing it as a check is a follow-up.

Checks

• swift test in FTWKit: 460 tests in 74 suites and 254 in 21, all pass after the merge (the live suites are off without their environment). New: the window against a plain record of 3,000 shuffled frames, a burst of a hundred overtaking a control message, a datagram past the window, what counts as QUIC failing, demotion, the memory's day, Freshness case 17.
• xcodebuild build -scheme FTW -destination 'generic/platform=iOS Simulator', as CI runs it: builds.
• LiveQUICTests against a local ftwcloudd and Core box passes on macOS, and in the iPhone 17 simulator at 40 ms with 0, 1 and 5 % loss. The rows are above.
• The app in the iPhone 17 simulator, paired through a local ftwcloudd with a Core box on QUIC: About → FTW Cloud reads "Connected · QUIC" in light and dark mode. At the largest text size "· QUIC" wraps to a second line and stays readable. The test box answers GET /api/cloud with a stub, since it has no HTTP API.
• LiveTunnelTests (step 2) still passes over the WebSocket with the readable copy off. With the copy on, its six returns from the background did not stream again within 20 s. That happened with the box on the WebSocket too. main (4b595e7) fails the same way against the same stack (box on the WebSocket, copy on): pairing and 50 taps pass, then the first return from the background does not stream within 20 s. So it predates this branch.

🤖 Generated with Claude Code

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-10-15T19:31:53.886398Z	441b283	PR opened

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 441b283848

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Retry QUIC after a remembered WebSocket win

When WebSocket is the remembered first transport and opens within 250 ms, cancelling lead prevents QUIC from ever being started, while memory.remember(transport) refreshes the WebSocket timestamp on every reconnect. Consequently, a single transient QUIC loss can make WebSocket win indefinitely rather than expiring after one day. Keep the delayed QUIC learning probe scheduled when WebSocket starts first, or avoid refreshing the remembered WebSocket result until QUIC has actually been retried.

Useful? React with 👍 / 👎.