From d74e8d2c59b39644dca7d4bb9a648c21678ea414 Mon Sep 17 00:00:00 2001
From: Fredrik Ahlgren <fredrik@sourceful-labs.com>
Date: Thu, 15 Oct 2026 19:10:36 +0200
Subject: [PATCH 1/5] feat(cloud): follow FTW Cloud to the instance that holds
 the home, and spread the redial after a restart

The wire for sharding FTW Cloud and for deploys without a reconnect storm:

- cloudwire: CloseWrongShard 4421 (reason: the host), CloseRestart 4503
  (reason: spread in ms), StatusWrongShard 421 with Ftw-Shard for the
  HTTP routes, CapShard, ShardHostOK (the host must be cloud.ftw.energy or
  one label under it), RestartSpreadMs.
- cloudlink: the box follows 4421 once, keeps the host in its link record,
  forgets it after a second redirect in a row or three failed dials, and
  redials within the spread after 4503 without growing its backoff.
- docs/protocol-cloud.md: "Shards and restarts", the close codes, the
  extension token's home prefix and the numbers.

Paired with srcfl/ftw-cloud (feat/scale).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---
 .changeset/cloud-shard-redirect.md   |   5 +
 docs/protocol-cloud.md               |  67 ++++++-
 go/internal/cloudlink/conn.go        | 116 +++++++++++-
 go/internal/cloudlink/grant.go       |   3 +
 go/internal/cloudlink/link.go        |  13 +-
 go/internal/cloudlink/link_test.go   |  36 +++-
 go/internal/cloudlink/shard_test.go  | 268 +++++++++++++++++++++++++++
 go/internal/cloudlink/tunnel_test.go |   2 +-
 go/protocol/cloudwire/shard.go       |  89 +++++++++
 go/protocol/cloudwire/shard_test.go  |  47 +++++
 10 files changed, 635 insertions(+), 11 deletions(-)
 create mode 100644 .changeset/cloud-shard-redirect.md
 create mode 100644 go/internal/cloudlink/shard_test.go
 create mode 100644 go/protocol/cloudwire/shard.go
 create mode 100644 go/protocol/cloudwire/shard_test.go

diff --git a/.changeset/cloud-shard-redirect.md b/.changeset/cloud-shard-redirect.md
new file mode 100644
index 00000000..559f57b1
--- /dev/null
+++ b/.changeset/cloud-shard-redirect.md
@@ -0,0 +1,5 @@
+---
+"ftw": patch
+---
+
+The box follows FTW Cloud to the instance that holds its home. When the service closes the uplink with 4421, the box dials the host named in the close reason once, only if it is `cloud.ftw.energy` or one label under it, and keeps that host with its link so a restart goes there first. It goes back to `cloud.ftw.energy` after a second redirect in a row or three failed dials. When the service closes with 4503 before a deploy, the box redials at a random moment within the spread the service names, instead of all boxes coming back within half a second. The box lists `shard` in `hello.caps`. The service still proves itself with the TLS certificate and the service key the box was built with, so a redirect grants nothing.
diff --git a/docs/protocol-cloud.md b/docs/protocol-cloud.md
index 5a3341d3..2f7676c0 100644
--- a/docs/protocol-cloud.md
+++ b/docs/protocol-cloud.md
@@ -49,7 +49,9 @@ for both live in `go/protocol`, under Apache-2.0
 - No proxy may end TLS in front of the service, because that breaks the
   handshake binding. A plain TCP load balancer is fine.
 - Reconnects back off with full jitter from 0.5 s to 60 s. The attempt
-  counter resets after 60 s connected.
+  counter resets after 60 s connected. Two closes are not failures: 4421
+  names the instance that holds the home, and 4503 asks the box to come back
+  within a spread (see [Shards and restarts](#shards-and-restarts)).
 - The service sends a WebSocket ping every 15 s. The box's read deadline is
   45 s and its write deadline 10 s.
 - WebSocket compression is off.
@@ -661,6 +663,8 @@ nothing about which check failed.
 | 4409 | The roster changed what this device may read | Reconnects at once, once; then retries with backoff |
 | 4429 | The home has as many sessions as the service allows, as at the relay | Retries with backoff |
 | 1013 | The phone fell too far behind | Retries with backoff and gets a fresh snapshot |
+| 4421 | Another instance holds the home. The reason is its host | Follows once and keeps the host for the home (see [Shards and restarts](#shards-and-restarts)) |
+| 4503 | The service is about to stop. The reason is the spread in milliseconds | Redials at a random moment within the spread, without backoff |
 
 - **Backoff** is full jitter from 0.5 s to 60 s, as on the uplink. It also
   covers a lost socket.
@@ -865,7 +869,10 @@ through the relay instead, and prices and the plan wait for the phone.
 
 ## Extension token
 
-- 32 random bytes, base64url. The service stores its SHA-256, never the token.
+- The home id, a dot, and 32 random bytes in base64url. The service stores
+  the SHA-256 of the random bytes, never the token, and refuses a token whose
+  home is not the one its row is bound to. The home id lets any instance tell
+  which shard holds the token. To the extension the token is opaque.
 - Bound to one roster entry and valid for 7 days. The app renews it at each
   foreground and background refresh.
 - It dies when the roster drops the device, the device's epoch changes, an
@@ -1090,6 +1097,59 @@ app's, so the grant screen claims no more than an owner's session.
   After 90 days without a valid roster it deletes the home as in a purge.
 - **A new home** that sends no roster within 24 hours is deleted.
 
+## Shards and restarts
+
+Every home lives on exactly one instance of the service, its shard. Its box,
+phones, watch and extensions all meet there, so nothing about a home crosses
+between instances: no shared pub/sub and no shared storage, and readings stay
+off disk. `cloud.ftw.energy` is the default host. With one shard it is the
+only host and nothing below happens.
+
+**Wrong shard.** An instance that gets a connection or a request for a home
+it does not hold names the host that does:
+
+| Where | When the service knows the home | Answer |
+|---|---|---|
+| `/v1/box` | After `hello`, from the key that signed it, before the registry is touched | Close 4421, reason the host |
+| `/v1/app` | From `?home=`, before message 1 is read | Close 4421, reason the host |
+| `/v1/app/once` | From `?home=`, before the body is read | 421, the host in `Ftw-Shard`, empty body |
+| `/v1/now` | From the home id in the token | 421, the host in `Ftw-Shard`, empty body |
+
+A client follows a wrong-shard answer only to `cloud.ftw.energy` itself or
+one DNS label of lowercase letters, digits and hyphens directly under it,
+such as `s2.cloud.ftw.energy` (`cloudwire.ShardHostOK`). The redirect carries
+no trust: the box still checks the TLS certificate for that host and
+`hello_ok` against the service keys it was built with, and the phone still
+runs Noise IK to the service Noise key it carries.
+
+- The client keeps the host for that home. The box stores it with its link,
+  so a restart dials the shard first.
+- It follows once. A second wrong-shard answer before it is connected again
+  sends it back to `cloud.ftw.energy` after backoff, where it may follow once
+  more.
+- After 3 dials in a row that fail at a kept host, the client forgets it and
+  starts again at `cloud.ftw.energy`.
+- A box lists `shard` in `hello.caps` when it follows 4421 and spreads its
+  redial after 4503. A second shard holds homes only once their boxes list
+  it.
+
+**Directory.** Each instance holds the same directory: the shards' hosts, a
+shard for each of 256 buckets of homes, and single homes moved on their own.
+A home's bucket is the first byte of SHA-256 of `ftw.cloud.shard\0` and the
+home id. Moving homes means copying their registry rows to the new shard,
+then updating the directory on every instance. Their clients follow the
+redirect at their next connection, and extension tokens keep working,
+because the rows hold their hashes.
+
+**Restart.** An instance that is about to stop, for a deploy or a move, stops
+accepting connections and closes every socket with 4503. The reason is a
+spread in milliseconds, in decimal, which the service sets from how many
+sockets it holds and how fast the next instance can take them. The client
+redials at a uniformly random moment within the spread, at most 60 s, and
+does not count the close as a failed attempt. A missing or unreadable reason
+means 5 s. The service sends no `box.unreachable` for boxes it closed this
+way.
+
 ## Service state
 
 Per home, the service keeps in memory the newest envelope of each class, the
@@ -1143,6 +1203,9 @@ the registry's roles would remove that copy.
 | What | Value |
 |---|---|
 | Reconnect backoff, box and phone | 0.5 s to 60 s, full jitter |
+| Redial after 4503 | Uniform within the reason's spread; 5 s without one; at most 60 s |
+| Wrong-shard redirects followed | One per connection; a kept host is forgotten after 3 failed dials |
+| Shard buckets | 256 |
 | Phone retry after 4403 | Half to all of min(15 s × 2ⁿ, 5 min) |
 | Ping, read deadline, write deadline | 15 s, 45 s, 10 s, on both sockets |
 | Period per class | 5 s; 1 s while watched; `lp` at least 2 s |
diff --git a/go/internal/cloudlink/conn.go b/go/internal/cloudlink/conn.go
index a27e7238..bbc6cdad 100644
--- a/go/internal/cloudlink/conn.go
+++ b/go/internal/cloudlink/conn.go
@@ -8,6 +8,8 @@ import (
 	"errors"
 	"fmt"
 	"math/big"
+	"net"
+	"net/url"
 	"slices"
 	"sync"
 	"time"
@@ -45,18 +47,20 @@ func (l *Link) dialer() *websocket.Dialer {
 		HandshakeTimeout:  dialTimeout,
 		TLSClientConfig:   l.tlsConfig(),
 		EnableCompression: false,
+		NetDialContext:    l.opts.NetDial,
 	}
 }
 
 // once dials, serves until the connection ends and says how long to wait.
 func (l *Link) once(ctx context.Context, snap grantSnapshot) (time.Duration, error) {
 	dialCtx, cancel := context.WithTimeout(ctx, dialTimeout)
-	conn, resp, err := l.dialer().DialContext(dialCtx, l.opts.Endpoint, nil)
+	conn, resp, err := l.dialer().DialContext(dialCtx, l.endpoint(), nil)
 	cancel()
 	if resp != nil && resp.Body != nil {
 		_ = resp.Body.Close()
 	}
 	if err != nil {
+		l.hostFailed()
 		return l.backoff(), fmt.Errorf("dialling FTW Cloud: %w", err)
 	}
 	defer conn.Close()
@@ -64,6 +68,7 @@ func (l *Link) once(ctx context.Context, snap grantSnapshot) (time.Duration, err
 	connectedAt := time.Now()
 	err = l.serve(ctx, conn, snap)
 	l.mu.Lock()
+	up := l.connected
 	l.connected = false
 	if time.Since(connectedAt) >= backoffCap {
 		l.attempt = 0
@@ -72,9 +77,115 @@ func (l *Link) once(ctx context.Context, snap grantSnapshot) (time.Duration, err
 	if errors.Is(err, errLinkEnded) || errors.Is(err, errClosed) {
 		return 0, nil
 	}
+	// The service's two closes that are not failures: another instance
+	// holds the home, or this one is about to stop.
+	var closed *websocket.CloseError
+	if errors.As(err, &closed) {
+		switch closed.Code {
+		case cloudwire.CloseWrongShard:
+			if l.followShard(closed.Text) {
+				return 0, nil
+			}
+		case cloudwire.CloseRestart:
+			l.mu.Lock()
+			l.attempt = 0
+			l.mu.Unlock()
+			return spread(cloudwire.RestartSpreadMs(closed.Text)), nil
+		}
+	}
+	if !up {
+		l.hostFailed()
+	}
 	return l.backoff(), err
 }
 
+// spread is a uniformly random wait within ms milliseconds.
+func spread(ms int64) time.Duration {
+	if ms <= 0 {
+		return 0
+	}
+	n, err := rand.Int(rand.Reader, big.NewInt(ms*int64(time.Millisecond)))
+	if err != nil {
+		return time.Duration(ms) * time.Millisecond / 2
+	}
+	return time.Duration(n.Int64())
+}
+
+// endpoint is the uplink at the host the service last named for this home,
+// or at Endpoint's own host.
+func (l *Link) endpoint() string {
+	l.mu.Lock()
+	host := ""
+	if l.link != nil {
+		host = l.link.Host
+	}
+	l.mu.Unlock()
+	u, err := url.Parse(l.opts.Endpoint)
+	if host == "" || err != nil {
+		return l.opts.Endpoint
+	}
+	if port := u.Port(); port != "" {
+		host = net.JoinHostPort(host, port)
+	}
+	u.Host = host
+	return u.String()
+}
+
+// followShard keeps the host a wrong-shard close named, once. A second
+// wrong-shard close before the box is connected again, or a host off the
+// service's domain, sends the box back to Endpoint's host after backoff,
+// where it may follow once more.
+func (l *Link) followShard(host string) bool {
+	base := ""
+	if u, err := url.Parse(l.opts.Endpoint); err == nil {
+		base = u.Hostname()
+	}
+	l.mu.Lock()
+	defer l.mu.Unlock()
+	if l.link == nil {
+		return false
+	}
+	if l.shardHops > 0 || !cloudwire.ShardHostOK(base, host) {
+		l.shardHops = 0
+		l.keepHostLocked("")
+		return false
+	}
+	l.shardHops = 1
+	l.keepHostLocked(host)
+	l.log.Info("FTW Cloud: another instance holds this home; following", "host", host)
+	return true
+}
+
+// hostFailed counts a dial or handshake that failed at a kept host. After
+// cloudwire.ShardHostFailures in a row the box forgets it, and Endpoint's
+// host names the right one again.
+func (l *Link) hostFailed() {
+	l.mu.Lock()
+	defer l.mu.Unlock()
+	if l.link == nil || l.link.Host == "" {
+		return
+	}
+	l.hostFails++
+	if l.hostFails >= cloudwire.ShardHostFailures {
+		l.log.Info("FTW Cloud: the kept host stopped answering; starting again at the default host")
+		l.keepHostLocked("")
+	}
+}
+
+// keepHostLocked stores the host the uplink dials. The caller holds l.mu.
+func (l *Link) keepHostLocked(host string) {
+	l.hostFails = 0
+	if l.link.Host == host {
+		return
+	}
+	rec := *l.link
+	rec.Host = host
+	if err := saveJSON(l.opts.KV, linkKey, rec); err != nil {
+		l.log.Warn("could not store the FTW Cloud host", "err", err)
+	}
+	l.link = &rec
+}
+
 // backoff is full jitter from 0.5 s to 60 s.
 func (l *Link) backoff() time.Duration {
 	l.mu.Lock()
@@ -172,6 +283,7 @@ func (l *Link) serve(ctx context.Context, conn *websocket.Conn, snap grantSnapsh
 	tunOut, tunBulk := make(chan []byte, tunQueue), make(chan []byte, tunBulkQueue)
 	l.mu.Lock()
 	l.connected = true
+	l.shardHops, l.hostFails = 0, 0
 	if l.opts.Tunnel != nil {
 		l.tunOut, l.tunBulk = tunOut, tunBulk
 	}
@@ -366,7 +478,7 @@ func (w *writer) send(snap grantSnapshot, t string, body any) error {
 // tunnel's own budget, and was counted. Noise transport survives a lost
 // message, so a session goes on after ErrTunDropped.
 var (
-	ErrNoUplink   = errors.New("cloudlink: no uplink to FTW Cloud")
+	ErrNoUplink         = errors.New("cloudlink: no uplink to FTW Cloud")
 	ErrTunDropped error = tunDropped{}
 )
 
diff --git a/go/internal/cloudlink/grant.go b/go/internal/cloudlink/grant.go
index e5313602..9b825025 100644
--- a/go/internal/cloudlink/grant.go
+++ b/go/internal/cloudlink/grant.go
@@ -38,6 +38,9 @@ type linkRecord struct {
 	// Ending means the box asked the service to delete the home and waits
 	// for the receipt. The key stays until it arrives.
 	Ending bool `json:"ending,omitempty"`
+	// Host is the instance of the service that holds this home, as a
+	// wrong-shard close named it. Empty is Endpoint's own host.
+	Host string `json:"host,omitempty"`
 }
 
 // KV is the part of the state store the grant needs.
diff --git a/go/internal/cloudlink/link.go b/go/internal/cloudlink/link.go
index 7329e2d7..94dd0b17 100644
--- a/go/internal/cloudlink/link.go
+++ b/go/internal/cloudlink/link.go
@@ -28,6 +28,7 @@ import (
 	"errors"
 	"fmt"
 	"log/slog"
+	"net"
 	"os"
 	"slices"
 	"strings"
@@ -169,6 +170,7 @@ type Options struct {
 	ServiceKeys []*ecdsa.PublicKey
 	RootCAs     *x509.CertPool
 	Tick        time.Duration
+	NetDial     func(ctx context.Context, network, addr string) (net.Conn, error)
 }
 
 // Link is the box's one uplink to FTW Cloud.
@@ -207,8 +209,13 @@ type Link struct {
 	lpSeq      uint64
 	resyncAt   time.Time
 	attempt    int
-	purgedAt   int64
-	rosterDue  bool
+	// shardHops counts wrong-shard closes since the box was last
+	// connected, and hostFails dials in a row that failed at the kept
+	// host (linkRecord.Host).
+	shardHops int
+	hostFails int
+	purgedAt  int64
+	rosterDue bool
 
 	// events wait for the writer; eventsDropped counts those that never
 	// left. closing is set by Bye, after which the link does not dial
@@ -966,7 +973,7 @@ func (l *Link) caps(snap grantSnapshot) []string {
 	if l.opts.Tunnel != nil {
 		out = append(out, cloudwire.CapTun)
 	}
-	return out
+	return append(out, cloudwire.CapShard)
 }
 
 // tlsConfig is TLS 1.3 only, with the system roots unless a test or a dev
diff --git a/go/internal/cloudlink/link_test.go b/go/internal/cloudlink/link_test.go
index 0e352f62..d0493a93 100644
--- a/go/internal/cloudlink/link_test.go
+++ b/go/internal/cloudlink/link_test.go
@@ -181,6 +181,9 @@ type fakeService struct {
 	purge       bool
 	afterHello  func(conn *websocket.Conn, home string)
 	rosterSeqOK uint64
+	// wrongShard, when set, closes after hello with CloseWrongShard and
+	// this host, as an instance that does not hold the home.
+	wrongShard string
 
 	wmu   sync.Mutex
 	mu    sync.Mutex
@@ -194,10 +197,18 @@ type fakeService struct {
 }
 
 func newFakeService(t *testing.T) *fakeService {
+	return newFakeServiceCert(t, nil)
+}
+
+// newFakeServiceCert serves with cert, or httptest's own when nil.
+func newFakeServiceCert(t *testing.T, cert *tls.Certificate) *fakeService {
 	key, _ := ecdsa.GenerateKey(elliptic.P256(), rand.Reader)
 	s := &fakeService{t: t, key: key, ch: make(chan cloudwire.Envelope, 256)}
 	s.srv = httptest.NewUnstartedServer(http.HandlerFunc(s.handle))
 	s.srv.TLS = &tls.Config{MinVersion: tls.VersionTLS13}
+	if cert != nil {
+		s.srv.TLS.Certificates = []tls.Certificate{*cert}
+	}
 	s.srv.StartTLS()
 	t.Cleanup(s.srv.Close)
 	return s
@@ -262,7 +273,13 @@ func (s *fakeService) handle(w http.ResponseWriter, r *http.Request) {
 	}
 	s.mu.Lock()
 	s.home, s.pub, s.hello = home, pub, msg
+	shard := s.wrongShard
 	s.mu.Unlock()
+	if shard != "" {
+		closeWith(conn, cloudwire.CloseWrongShard, shard)
+		_, _, _ = conn.ReadMessage()
+		return
+	}
 
 	okEKM := ekm
 	if s.badEKM {
@@ -376,11 +393,20 @@ func newHarness(t *testing.T, svc *fakeService) *harness {
 // newHarnessKeys is a box that trusts keys; nil means the keys this build
 // carries.
 func newHarnessKeys(t *testing.T, svc *fakeService, keys []*ecdsa.PublicKey) *harness {
+	return newHarnessWith(t, svc, keys, nil, nil)
+}
+
+// newHarnessWith also takes the box's state store, and tweak may change the
+// options before the link starts.
+func newHarnessWith(t *testing.T, svc *fakeService, keys []*ecdsa.PublicKey, kv *memKV, tweak func(*Options)) *harness {
 	t.Helper()
-	h := &harness{svc: svc, site: &fakeSite{grid: 1200, state: protocol.SourceLive}, kv: &memKV{},
+	if kv == nil {
+		kv = &memKV{}
+	}
+	h := &harness{svc: svc, site: &fakeSite{grid: 1200, state: protocol.SourceLive}, kv: kv,
 		keyPath: filepath.Join(t.TempDir(), "cloud.key"), pushed: make(chan string, 8),
 		roster: &fakeRosterState{}, tunnel: &fakeTunnel{ch: make(chan cloudwire.Tun, 64)}}
-	link, err := New(Options{
+	opts := Options{
 		Site: h.site, Roster: fakeRoster{state: h.roster}, KV: h.kv, KeyPath: h.keyPath, Build: "test",
 		Started: time.Now().Add(-time.Minute), ClockSynced: func() bool { return true },
 		Notify:   func(kind string) { h.pushed <- kind },
@@ -388,7 +414,11 @@ func newHarnessKeys(t *testing.T, svc *fakeService, keys []*ecdsa.PublicKey) *ha
 		Endpoint: svc.endpoint(), ServiceKeys: keys,
 		RootCAs: svc.roots(), Tick: 20 * time.Millisecond,
 		Logger: slog.New(slog.NewTextHandler(io.Discard, nil)),
-	})
+	}
+	if tweak != nil {
+		tweak(&opts)
+	}
+	link, err := New(opts)
 	if err != nil {
 		t.Fatal(err)
 	}
diff --git a/go/internal/cloudlink/shard_test.go b/go/internal/cloudlink/shard_test.go
new file mode 100644
index 00000000..20978fa4
--- /dev/null
+++ b/go/internal/cloudlink/shard_test.go
@@ -0,0 +1,268 @@
+package cloudlink
+
+import (
+	"context"
+	"crypto/ecdsa"
+	"crypto/elliptic"
+	"crypto/rand"
+	"crypto/tls"
+	"crypto/x509"
+	"crypto/x509/pkix"
+	"errors"
+	"math/big"
+	"net"
+	"strings"
+	"sync"
+	"testing"
+	"time"
+
+	"github.com/gorilla/websocket"
+	"github.com/srcfl/ftw/go/protocol/cloudwire"
+)
+
+// closeWith ends a connection as the service does, with a close code and a
+// reason. The caller's read loop takes the box's answer.
+func closeWith(conn *websocket.Conn, code int, reason string) {
+	_ = conn.WriteControl(websocket.CloseMessage, websocket.FormatCloseMessage(code, reason), time.Now().Add(time.Second))
+}
+
+// shardCert is a certificate for cloud.test and s2.cloud.test that is its
+// own root.
+func shardCert(t *testing.T) (tls.Certificate, *x509.CertPool) {
+	t.Helper()
+	key, err := ecdsa.GenerateKey(elliptic.P256(), rand.Reader)
+	if err != nil {
+		t.Fatal(err)
+	}
+	tmpl := &x509.Certificate{
+		SerialNumber: big.NewInt(1), Subject: pkix.Name{CommonName: "cloud.test"},
+		DNSNames:  []string{"cloud.test", "s2.cloud.test"},
+		NotBefore: time.Now().Add(-time.Hour), NotAfter: time.Now().Add(time.Hour),
+		IsCA: true, BasicConstraintsValid: true,
+		KeyUsage:    x509.KeyUsageDigitalSignature | x509.KeyUsageCertSign,
+		ExtKeyUsage: []x509.ExtKeyUsage{x509.ExtKeyUsageServerAuth},
+	}
+	der, err := x509.CreateCertificate(rand.Reader, tmpl, tmpl, &key.PublicKey, key)
+	if err != nil {
+		t.Fatal(err)
+	}
+	leaf, _ := x509.ParseCertificate(der)
+	pool := x509.NewCertPool()
+	pool.AddCert(leaf)
+	return tls.Certificate{Certificate: [][]byte{der}, PrivateKey: key, Leaf: leaf}, pool
+}
+
+// shardNet dials each name at the fake service that plays it, whatever the
+// port, and counts the dials per name. A name it does not know fails.
+type shardNet struct {
+	mu    sync.Mutex
+	at    map[string]string
+	dials map[string]int
+}
+
+func (n *shardNet) dial(ctx context.Context, network, addr string) (net.Conn, error) {
+	host, _, err := net.SplitHostPort(addr)
+	if err != nil {
+		return nil, err
+	}
+	n.mu.Lock()
+	n.dials[host]++
+	to, ok := n.at[host]
+	n.mu.Unlock()
+	if !ok {
+		return nil, errors.New("no such host")
+	}
+	return (&net.Dialer{}).DialContext(ctx, network, to)
+}
+
+func (n *shardNet) count(host string) int {
+	n.mu.Lock()
+	defer n.mu.Unlock()
+	return n.dials[host]
+}
+
+// twoShards is cloud.test, which does not hold the home and names
+// s2.cloud.test, and s2.cloud.test, which does. Both carry the same service
+// key, as every instance does.
+func twoShards(t *testing.T) (s1, s2 *fakeService, nw *shardNet, pool *x509.CertPool) {
+	cert, pool := shardCert(t)
+	s1, s2 = newFakeServiceCert(t, &cert), newFakeServiceCert(t, &cert)
+	s2.key = s1.key
+	s1.wrongShard = "s2.cloud.test"
+	nw = &shardNet{at: map[string]string{
+		"cloud.test":    s1.srv.Listener.Addr().String(),
+		"s2.cloud.test": s2.srv.Listener.Addr().String(),
+	}, dials: map[string]int{}}
+	return s1, s2, nw, pool
+}
+
+// stop ends a harness's link before its test does; the cleanup still finds
+// the link's result.
+func stop(h *harness) {
+	h.cancel()
+	err := <-h.done
+	h.done <- err
+}
+
+func shardBox(t *testing.T, svc *fakeService, nw *shardNet, pool *x509.CertPool, kv *memKV, keyPath string) *harness {
+	return newHarnessWith(t, svc, []*ecdsa.PublicKey{&svc.key.PublicKey}, kv, func(o *Options) {
+		o.Endpoint, o.RootCAs, o.NetDial = "wss://cloud.test:443/v1/box", pool, nw.dial
+		if keyPath != "" {
+			o.KeyPath = keyPath
+		}
+	})
+}
+
+// An instance that does not hold the home names the one that does. The box
+// follows once, keeps the host, and dials it first after a restart.
+func TestTheBoxFollowsAWrongShardCloseOnceAndKeepsTheHost(t *testing.T) {
+	s1, s2, nw, pool := twoShards(t)
+	h := shardBox(t, s1, nw, pool, nil, "")
+	h.setRoster(func(r *fakeRosterState) { r.tun = true })
+	s2.waitFor(cloudwire.MsgRoster, 5*time.Second)
+	if n := nw.count("cloud.test"); n != 1 {
+		t.Fatalf("the default host was dialled %d times, want 1", n)
+	}
+	h.link.mu.Lock()
+	kept := h.link.link.Host
+	h.link.mu.Unlock()
+	if kept != "s2.cloud.test" {
+		t.Fatalf("kept host %q", kept)
+	}
+	if rec, _ := loadJSON[linkRecord](h.kv, linkKey); rec == nil || rec.Host != "s2.cloud.test" {
+		t.Fatalf("stored link %+v", rec)
+	}
+
+	// The same box after a restart goes straight to its shard.
+	keyPath := h.link.opts.KeyPath
+	stop(h)
+	before := nw.count("cloud.test")
+	h2 := shardBox(t, s1, nw, pool, h.kv, keyPath)
+	h2.setRoster(func(r *fakeRosterState) { r.tun = true })
+	s2.waitFor(cloudwire.MsgRoster, 5*time.Second)
+	if n := nw.count("cloud.test"); n != before {
+		t.Fatalf("a restarted box dialled the default host %d more times", n-before)
+	}
+}
+
+// A redirect off the service's domain is not followed: the box stays with
+// the default host and backs off.
+func TestTheBoxRefusesAShardHostOffTheServicesDomain(t *testing.T) {
+	s1, _, nw, pool := twoShards(t)
+	s1.wrongShard = "evil.example"
+	nw.at["evil.example"] = nw.at["s2.cloud.test"]
+	h := shardBox(t, s1, nw, pool, nil, "")
+	h.setRoster(func(r *fakeRosterState) { r.tun = true })
+	deadline := time.Now().Add(5 * time.Second)
+	for nw.count("cloud.test") < 2 && time.Now().Before(deadline) {
+		time.Sleep(20 * time.Millisecond)
+	}
+	if nw.count("cloud.test") < 2 || nw.count("evil.example") != 0 {
+		t.Fatalf("dials %v", nw.dials)
+	}
+	if rec, _ := loadJSON[linkRecord](h.kv, linkKey); rec == nil || rec.Host != "" {
+		t.Fatalf("stored link %+v", rec)
+	}
+}
+
+// Two instances that each name the other do not make the box loop: the
+// second wrong-shard close sends it back to the default host after backoff.
+func TestTwoWrongShardClosesInARowGoBackToTheDefaultHost(t *testing.T) {
+	s1, s2, nw, pool := twoShards(t)
+	s2.wrongShard = "cloud.test"
+	h := shardBox(t, s1, nw, pool, nil, "")
+	h.setRoster(func(r *fakeRosterState) { r.tun = true })
+	time.Sleep(1500 * time.Millisecond)
+	// Backoff from 0.5 s with full jitter allows a few rounds in 1.5 s,
+	// never the hundreds a loop without it would make.
+	if n := nw.count("cloud.test") + nw.count("s2.cloud.test"); n < 3 || n > 12 {
+		t.Fatalf("%d dials in 1.5 s", n)
+	}
+	if nw.count("cloud.test") < 2 {
+		t.Fatalf("the box never went back to the default host: %v", nw.dials)
+	}
+}
+
+// A kept host that stops answering is forgotten after
+// cloudwire.ShardHostFailures dials, and the default host names the shard
+// again.
+func TestAKeptHostThatStopsAnsweringIsForgotten(t *testing.T) {
+	s1, s2, nw, pool := twoShards(t)
+	kv := &memKV{}
+	h := shardBox(t, s1, nw, pool, kv, "")
+	h.setRoster(func(r *fakeRosterState) { r.tun = true })
+	s2.waitFor(cloudwire.MsgRoster, 5*time.Second)
+	keyPath := h.link.opts.KeyPath
+	stop(h)
+
+	// s2 is gone; s1 now holds the home.
+	nw.mu.Lock()
+	delete(nw.at, "s2.cloud.test")
+	nw.mu.Unlock()
+	s1.mu.Lock()
+	s1.wrongShard = ""
+	s1.mu.Unlock()
+	h2 := shardBox(t, s1, nw, pool, kv, keyPath)
+	h2.setRoster(func(r *fakeRosterState) { r.tun = true })
+	s1.waitFor(cloudwire.MsgRoster, 10*time.Second)
+	if n := nw.count("s2.cloud.test"); n != 1+cloudwire.ShardHostFailures {
+		t.Fatalf("the gone host was dialled %d times, want %d", n, 1+cloudwire.ShardHostFailures)
+	}
+	if rec, _ := loadJSON[linkRecord](kv, linkKey); rec == nil || rec.Host != "" {
+		t.Fatalf("stored link %+v", rec)
+	}
+}
+
+// A service about to stop closes with CloseRestart and a spread. The box
+// redials within the spread and does not count the close as a failure, so
+// its backoff does not grow.
+func TestARestartCloseRedialsWithinTheSpreadWithoutBackoff(t *testing.T) {
+	svc := newFakeService(t)
+	var mu sync.Mutex
+	restarts := 0
+	svc.afterHello = func(conn *websocket.Conn, _ string) {
+		mu.Lock()
+		restarts++
+		n := restarts
+		mu.Unlock()
+		if n <= 5 {
+			closeWith(conn, cloudwire.CloseRestart, "50")
+		}
+	}
+	h := newHarness(t, svc)
+	h.setRoster(func(r *fakeRosterState) { r.tun = true })
+	deadline := time.Now().Add(3 * time.Second)
+	for {
+		svc.mu.Lock()
+		n := svc.conns
+		svc.mu.Unlock()
+		if n >= 6 {
+			break
+		}
+		if time.Now().After(deadline) {
+			// Five failures in a row back off for up to 0.5+1+2+4+8 s.
+			t.Fatalf("%d connections in 3 s after five restarts", n)
+		}
+		time.Sleep(10 * time.Millisecond)
+	}
+	h.link.mu.Lock()
+	attempt := h.link.attempt
+	h.link.mu.Unlock()
+	if attempt != 0 {
+		t.Fatalf("attempt %d after restarts", attempt)
+	}
+}
+
+func TestSpreadStaysWithinItsMilliseconds(t *testing.T) {
+	for range 200 {
+		if d := spread(40); d < 0 || d >= 40*time.Millisecond {
+			t.Fatalf("spread(40) = %s", d)
+		}
+	}
+	if spread(0) != 0 {
+		t.Fatal("spread(0) waits")
+	}
+	if !strings.HasPrefix(cloudwire.DefaultHost, "cloud.") {
+		t.Fatal(cloudwire.DefaultHost)
+	}
+}
diff --git a/go/internal/cloudlink/tunnel_test.go b/go/internal/cloudlink/tunnel_test.go
index ccae9688..e644573b 100644
--- a/go/internal/cloudlink/tunnel_test.go
+++ b/go/internal/cloudlink/tunnel_test.go
@@ -41,7 +41,7 @@ func TestTheBoxLinksForItsPhonesAndSendsNoReadings(t *testing.T) {
 	if err := cloudwire.Unmarshal(env.B, &hello); err != nil {
 		t.Fatal(err)
 	}
-	if len(hello.GrantID) != 0 || !slices.Equal(hello.Caps, []string{cloudwire.MsgRoster, cloudwire.CapTun}) {
+	if len(hello.GrantID) != 0 || !slices.Equal(hello.Caps, []string{cloudwire.MsgRoster, cloudwire.CapTun, cloudwire.CapShard}) {
 		t.Fatalf("hello names grant %x and caps %v", hello.GrantID, hello.Caps)
 	}
 
diff --git a/go/protocol/cloudwire/shard.go b/go/protocol/cloudwire/shard.go
new file mode 100644
index 00000000..d07b4e1a
--- /dev/null
+++ b/go/protocol/cloudwire/shard.go
@@ -0,0 +1,89 @@
+package cloudwire
+
+import (
+	"strconv"
+	"strings"
+)
+
+// Shards and restarts.
+//
+// Every home lives on exactly one instance of the service, its shard, and its
+// box, phones, watch and extensions all meet there. DefaultHost is where a
+// client starts; with one shard it is the only host. An instance that gets a
+// connection or a request for a home it does not hold names the host that
+// does. The client follows that once, keeps the host for the home, and goes
+// back to DefaultHost when the kept host stops answering.
+//
+// An instance that is about to stop, for a deploy or a move, closes every
+// socket with CloseRestart. The reason says over how many milliseconds to
+// spread the redial, so the service sets the spread to its own load and no
+// client release is needed when it changes.
+
+// DefaultHost is the service's default host.
+const DefaultHost = "cloud.ftw.energy"
+
+const (
+	// CloseWrongShard closes /v1/box or /v1/app: this instance does not
+	// hold the home. The close reason is the host that does. On /v1/box it
+	// comes after hello, because the home is the key that signed it; on
+	// /v1/app it comes before message 1 is read.
+	CloseWrongShard = 4421
+	// CloseRestart closes /v1/box or /v1/app when the service is about to
+	// stop. The close reason is the spread in milliseconds, in decimal. The
+	// client redials at a uniformly random moment within the spread and
+	// does not count the close as a failed attempt.
+	CloseRestart = 4503
+	// StatusWrongShard answers /v1/app/once and /v1/now for a home this
+	// instance does not hold: 421 Misdirected Request, with the host that
+	// does in ShardHeader and an empty body.
+	StatusWrongShard = 421
+	// ShardHeader names the host on a StatusWrongShard answer.
+	ShardHeader = "Ftw-Shard"
+	// CapShard in hello.caps says the box follows CloseWrongShard and
+	// spreads its redial after CloseRestart. A second shard holds homes
+	// only once their boxes list it.
+	CapShard = "shard"
+
+	// RestartSpreadDefaultMs is the spread when a CloseRestart reason is
+	// missing or not a number; RestartSpreadMaxMs caps any spread.
+	RestartSpreadDefaultMs = 5_000
+	RestartSpreadMaxMs     = 60_000
+	// ShardHostFailures is how many dials in a row to a kept host may fail
+	// before the client forgets it and starts again at DefaultHost.
+	ShardHostFailures = 3
+)
+
+// RestartSpreadMs reads a CloseRestart reason: a whole number of
+// milliseconds, at most RestartSpreadMaxMs. Anything else is the default.
+func RestartSpreadMs(reason string) int64 {
+	ms, err := strconv.ParseInt(strings.TrimSpace(reason), 10, 64)
+	if err != nil || ms < 0 {
+		return RestartSpreadDefaultMs
+	}
+	return min(ms, RestartSpreadMaxMs)
+}
+
+// ShardHostOK reports whether a client of the service at base may follow a
+// wrong-shard answer to host: base itself, or one DNS label of lowercase
+// letters, digits and hyphens directly under it, such as s2.cloud.ftw.energy
+// under cloud.ftw.energy. Anything else is refused, so a redirect never sends
+// a box or a phone off the service's own domain. Neither name carries a port.
+func ShardHostOK(base, host string) bool {
+	if base == "" || host == "" || strings.ContainsAny(base, ":/") {
+		return false
+	}
+	if host == base {
+		return true
+	}
+	label, ok := strings.CutSuffix(host, "."+base)
+	if !ok || label == "" || len(label) > 63 || label[0] == '-' || label[len(label)-1] == '-' {
+		return false
+	}
+	for i := 0; i < len(label); i++ {
+		c := label[i]
+		if !(c >= 'a' && c <= 'z' || c >= '0' && c <= '9' || c == '-') {
+			return false
+		}
+	}
+	return true
+}
diff --git a/go/protocol/cloudwire/shard_test.go b/go/protocol/cloudwire/shard_test.go
new file mode 100644
index 00000000..8afd2ada
--- /dev/null
+++ b/go/protocol/cloudwire/shard_test.go
@@ -0,0 +1,47 @@
+package cloudwire
+
+import "testing"
+
+func TestShardHostOKStaysOnTheServicesDomain(t *testing.T) {
+	for _, c := range []struct {
+		base, host string
+		ok         bool
+	}{
+		{"cloud.ftw.energy", "cloud.ftw.energy", true},
+		{"cloud.ftw.energy", "s2.cloud.ftw.energy", true},
+		{"cloud.ftw.energy", "eu-2.cloud.ftw.energy", true},
+		{"cloud.ftw.energy", "a.b.cloud.ftw.energy", false},
+		{"cloud.ftw.energy", "S2.cloud.ftw.energy", false},
+		{"cloud.ftw.energy", "-s2.cloud.ftw.energy", false},
+		{"cloud.ftw.energy", "s2-.cloud.ftw.energy", false},
+		{"cloud.ftw.energy", ".cloud.ftw.energy", false},
+		{"cloud.ftw.energy", "s2cloud.ftw.energy", false},
+		{"cloud.ftw.energy", "evil.example", false},
+		{"cloud.ftw.energy", "cloud.ftw.energy.evil.example", false},
+		{"cloud.ftw.energy", "s2.cloud.ftw.energy:8443", false},
+		{"cloud.ftw.energy", "s2.cloud.ftw.energy/v1/box", false},
+		{"cloud.ftw.energy", "", false},
+		{"127.0.0.1:8443", "127.0.0.1:8443", false},
+		{"", "cloud.ftw.energy", false},
+	} {
+		if got := ShardHostOK(c.base, c.host); got != c.ok {
+			t.Errorf("ShardHostOK(%q, %q) = %v, want %v", c.base, c.host, got, c.ok)
+		}
+	}
+}
+
+func TestRestartSpreadReadsTheReason(t *testing.T) {
+	for reason, want := range map[string]int64{
+		"2500":   2500,
+		" 800 ":  800,
+		"0":      0,
+		"":       RestartSpreadDefaultMs,
+		"soon":   RestartSpreadDefaultMs,
+		"-1":     RestartSpreadDefaultMs,
+		"999999": RestartSpreadMaxMs,
+	} {
+		if got := RestartSpreadMs(reason); got != want {
+			t.Errorf("RestartSpreadMs(%q) = %d, want %d", reason, got, want)
+		}
+	}
+}

From ad8d1a9841a0e397b6fa1f333d629c0b45d1f507 Mon Sep 17 00:00:00 2001
From: Fredrik Ahlgren <fredrik@sourceful-labs.com>
Date: Thu, 15 Oct 2026 19:59:19 +0200
Subject: [PATCH 2/5] test: room for lucky jitter in the redirect loop test

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---
 go/internal/cloudlink/shard_test.go | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/go/internal/cloudlink/shard_test.go b/go/internal/cloudlink/shard_test.go
index 20978fa4..5dbfe567 100644
--- a/go/internal/cloudlink/shard_test.go
+++ b/go/internal/cloudlink/shard_test.go
@@ -175,7 +175,7 @@ func TestTwoWrongShardClosesInARowGoBackToTheDefaultHost(t *testing.T) {
 	time.Sleep(1500 * time.Millisecond)
 	// Backoff from 0.5 s with full jitter allows a few rounds in 1.5 s,
 	// never the hundreds a loop without it would make.
-	if n := nw.count("cloud.test") + nw.count("s2.cloud.test"); n < 3 || n > 12 {
+	if n := nw.count("cloud.test") + nw.count("s2.cloud.test"); n < 3 || n > 20 {
 		t.Fatalf("%d dials in 1.5 s", n)
 	}
 	if nw.count("cloud.test") < 2 {

From 4969254808131a40704ff9d4215ee3e7d4ede319 Mon Sep 17 00:00:00 2001
From: Fredrik Ahlgren <fredrik@sourceful-labs.com>
Date: Thu, 15 Oct 2026 20:15:05 +0200
Subject: [PATCH 3/5] fix(cloud): a forgotten host starts a new redirect chain,
 and a kept host must sit under the endpoint

- Forgetting the kept host resets the hop count, so the default host's
  next redirect is followed at once instead of costing another backoff
  (up to 8 s). The test fails without the fix.
- endpoint() dials a kept host only if it is the endpoint's own host or one
  label under it, so a host kept under an earlier build's endpoint is not
  dialled.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---
 go/internal/cloudlink/conn.go       | 12 +++++--
 go/internal/cloudlink/shard_test.go | 55 +++++++++++++++++++++++++++++
 2 files changed, 64 insertions(+), 3 deletions(-)

diff --git a/go/internal/cloudlink/conn.go b/go/internal/cloudlink/conn.go
index bbc6cdad..e694fe0a 100644
--- a/go/internal/cloudlink/conn.go
+++ b/go/internal/cloudlink/conn.go
@@ -112,7 +112,8 @@ func spread(ms int64) time.Duration {
 }
 
 // endpoint is the uplink at the host the service last named for this home,
-// or at Endpoint's own host.
+// or at Endpoint's own host. A kept host that is not under Endpoint's, as
+// after a build moved the service, is not dialled.
 func (l *Link) endpoint() string {
 	l.mu.Lock()
 	host := ""
@@ -121,7 +122,7 @@ func (l *Link) endpoint() string {
 	}
 	l.mu.Unlock()
 	u, err := url.Parse(l.opts.Endpoint)
-	if host == "" || err != nil {
+	if host == "" || err != nil || !cloudwire.ShardHostOK(u.Hostname(), host) {
 		return l.opts.Endpoint
 	}
 	if port := u.Port(); port != "" {
@@ -172,9 +173,14 @@ func (l *Link) hostFailed() {
 	}
 }
 
-// keepHostLocked stores the host the uplink dials. The caller holds l.mu.
+// keepHostLocked stores the host the uplink dials. Forgetting it starts a
+// new chain at Endpoint's host, where one redirect may be followed again.
+// The caller holds l.mu.
 func (l *Link) keepHostLocked(host string) {
 	l.hostFails = 0
+	if host == "" {
+		l.shardHops = 0
+	}
 	if l.link.Host == host {
 		return
 	}
diff --git a/go/internal/cloudlink/shard_test.go b/go/internal/cloudlink/shard_test.go
index 5dbfe567..fdd2994d 100644
--- a/go/internal/cloudlink/shard_test.go
+++ b/go/internal/cloudlink/shard_test.go
@@ -58,6 +58,14 @@ type shardNet struct {
 	mu    sync.Mutex
 	at    map[string]string
 	dials map[string]int
+	seq   []string
+}
+
+// order is every name dialled, in order.
+func (n *shardNet) order() []string {
+	n.mu.Lock()
+	defer n.mu.Unlock()
+	return append([]string(nil), n.seq...)
 }
 
 func (n *shardNet) dial(ctx context.Context, network, addr string) (net.Conn, error) {
@@ -67,6 +75,7 @@ func (n *shardNet) dial(ctx context.Context, network, addr string) (net.Conn, er
 	}
 	n.mu.Lock()
 	n.dials[host]++
+	n.seq = append(n.seq, host)
 	to, ok := n.at[host]
 	n.mu.Unlock()
 	if !ok {
@@ -208,6 +217,7 @@ func TestAKeptHostThatStopsAnsweringIsForgotten(t *testing.T) {
 	if n := nw.count("s2.cloud.test"); n != 1+cloudwire.ShardHostFailures {
 		t.Fatalf("the gone host was dialled %d times, want %d", n, 1+cloudwire.ShardHostFailures)
 	}
+
 	if rec, _ := loadJSON[linkRecord](kv, linkKey); rec == nil || rec.Host != "" {
 		t.Fatalf("stored link %+v", rec)
 	}
@@ -266,3 +276,48 @@ func TestSpreadStaysWithinItsMilliseconds(t *testing.T) {
 		t.Fatal(cloudwire.DefaultHost)
 	}
 }
+
+// A host kept under another Endpoint, as after a build moved the service, is
+// not dialled.
+func TestAKeptHostOffTheEndpointsDomainIsNotDialled(t *testing.T) {
+	s1, _, nw, pool := twoShards(t)
+	s1.wrongShard = ""
+	kv := &memKV{}
+	h := shardBox(t, s1, nw, pool, kv, "")
+	h.setRoster(func(r *fakeRosterState) { r.tun = true })
+	s1.waitFor(cloudwire.MsgRoster, 5*time.Second)
+	h.link.mu.Lock()
+	rec := *h.link.link
+	rec.Host = "s2.old-cloud.example"
+	h.link.link = &rec
+	h.link.mu.Unlock()
+	if got := h.link.endpoint(); got != "wss://cloud.test:443/v1/box" {
+		t.Fatalf("dials %s", got)
+	}
+}
+
+// After a host the box followed to is forgotten, the default host's next
+// redirect is followed at once, not refused as a second hop.
+func TestAfterForgettingAHostTheNextRedirectIsFollowed(t *testing.T) {
+	s1, _, nw, pool := twoShards(t)
+	// s2 never answers.
+	nw.mu.Lock()
+	delete(nw.at, "s2.cloud.test")
+	nw.mu.Unlock()
+	h := shardBox(t, s1, nw, pool, nil, "")
+	h.setRoster(func(r *fakeRosterState) { r.tun = true })
+	// cloud.test, then s2 until it is forgotten, then cloud.test again,
+	// and the dial after that.
+	want := 1 + cloudwire.ShardHostFailures + 2
+	deadline := time.Now().Add(15 * time.Second)
+	for len(nw.order()) < want && time.Now().Before(deadline) {
+		time.Sleep(20 * time.Millisecond)
+	}
+	got := nw.order()
+	if len(got) < want {
+		t.Fatalf("dials %v", got)
+	}
+	if got[want-2] != "cloud.test" || got[want-1] != "s2.cloud.test" {
+		t.Fatalf("after forgetting s2 the box dialled %v; want the redirect followed at once", got[:want])
+	}
+}

From 26426a60b2e722ea0eee0788572163a044054fa5 Mon Sep 17 00:00:00 2001
From: Fredrik Ahlgren <fredrik@sourceful-labs.com>
Date: Thu, 15 Oct 2026 22:35:37 +0200
Subject: [PATCH 4/5] fix(cloud): a 1 s floor on the restart spread, a 30 s
 window before a box forgets its shard, and shard vectors for every client

From the owner's review of #1645:
- RestartSpreadMs is 1 s to 60 s; a reason of 0 no longer gives a tight
  loop.
- A 4503 before hello_ok is a failed attempt: the backoff grows instead of
  resetting, so a service that turns every box away cannot hold them in a
  loop.
- A kept host is forgotten after 3 failed dials over at least 30 s
  (ShardHostFailWindowMs), which outlasts a shard's restart; three failures
  took about 3.5 s before.
- testdata/cloud-shard-vectors.json: which hosts a client may follow (no
  port, userinfo, uppercase, trailing dot or missing dot) and how it reads
  a spread, for Swift, Kotlin and TypeScript to replay.
- docs: a command in flight at a drain is unknown, not failed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---
 go/internal/cloudlink/conn.go                 | 21 ++++-
 go/internal/cloudlink/link.go                 | 18 ++--
 go/internal/cloudlink/link_test.go            | 12 ++-
 go/internal/cloudlink/shard_test.go           | 79 +++++++++++++++++-
 go/protocol/cloudwire/shard.go                | 18 ++--
 go/protocol/cloudwire/shard_test.go           | 83 ++++++++++---------
 .../testdata/cloud-shard-vectors.json         | 48 +++++++++++
 7 files changed, 222 insertions(+), 57 deletions(-)
 create mode 100644 go/protocol/cloudwire/testdata/cloud-shard-vectors.json

diff --git a/go/internal/cloudlink/conn.go b/go/internal/cloudlink/conn.go
index e694fe0a..1f50eae5 100644
--- a/go/internal/cloudlink/conn.go
+++ b/go/internal/cloudlink/conn.go
@@ -87,10 +87,16 @@ func (l *Link) once(ctx context.Context, snap grantSnapshot) (time.Duration, err
 				return 0, nil
 			}
 		case cloudwire.CloseRestart:
+			wait := spread(cloudwire.RestartSpreadMs(closed.Text))
+			if !up {
+				// A service that sends every box away before hello_ok is
+				// not restarting once; the backoff grows as for a failure.
+				return max(wait, l.backoff()), nil
+			}
 			l.mu.Lock()
 			l.attempt = 0
 			l.mu.Unlock()
-			return spread(cloudwire.RestartSpreadMs(closed.Text)), nil
+			return wait, nil
 		}
 	}
 	if !up {
@@ -158,16 +164,23 @@ func (l *Link) followShard(host string) bool {
 }
 
 // hostFailed counts a dial or handshake that failed at a kept host. After
-// cloudwire.ShardHostFailures in a row the box forgets it, and Endpoint's
-// host names the right one again.
+// cloudwire.ShardHostFailures in a row over at least
+// cloudwire.ShardHostFailWindowMs the box forgets it, and Endpoint's host
+// names the right one again. The window outlasts a shard's restart, so a
+// box does not leave its shard for the default host because of one.
 func (l *Link) hostFailed() {
 	l.mu.Lock()
 	defer l.mu.Unlock()
 	if l.link == nil || l.link.Host == "" {
 		return
 	}
+	now := l.now()
+	if l.hostFails == 0 {
+		l.hostFailedSince = now
+	}
 	l.hostFails++
-	if l.hostFails >= cloudwire.ShardHostFailures {
+	window := time.Duration(cloudwire.ShardHostFailWindowMs) * time.Millisecond
+	if l.hostFails >= cloudwire.ShardHostFailures && now.Sub(l.hostFailedSince) >= window {
 		l.log.Info("FTW Cloud: the kept host stopped answering; starting again at the default host")
 		l.keepHostLocked("")
 	}
diff --git a/go/internal/cloudlink/link.go b/go/internal/cloudlink/link.go
index 94dd0b17..6e974786 100644
--- a/go/internal/cloudlink/link.go
+++ b/go/internal/cloudlink/link.go
@@ -171,6 +171,7 @@ type Options struct {
 	RootCAs     *x509.CertPool
 	Tick        time.Duration
 	NetDial     func(ctx context.Context, network, addr string) (net.Conn, error)
+	Now         func() time.Time
 }
 
 // Link is the box's one uplink to FTW Cloud.
@@ -211,11 +212,12 @@ type Link struct {
 	attempt    int
 	// shardHops counts wrong-shard closes since the box was last
 	// connected, and hostFails dials in a row that failed at the kept
-	// host (linkRecord.Host).
-	shardHops int
-	hostFails int
-	purgedAt  int64
-	rosterDue bool
+	// host (linkRecord.Host), since hostFailedSince.
+	shardHops       int
+	hostFails       int
+	hostFailedSince time.Time
+	purgedAt        int64
+	rosterDue       bool
 
 	// events wait for the writer; eventsDropped counts those that never
 	// left. closing is set by Bye, after which the link does not dial
@@ -263,6 +265,9 @@ func New(opts Options) (*Link, error) {
 	if opts.Started.IsZero() {
 		opts.Started = time.Now()
 	}
+	if opts.Now == nil {
+		opts.Now = time.Now
+	}
 	if opts.Tick <= 0 {
 		opts.Tick = 250 * time.Millisecond
 	}
@@ -981,3 +986,6 @@ func (l *Link) caps(snap grantSnapshot) []string {
 func (l *Link) tlsConfig() *tls.Config {
 	return &tls.Config{MinVersion: tls.VersionTLS13, RootCAs: l.opts.RootCAs}
 }
+
+// now is the wall clock, or a test's.
+func (l *Link) now() time.Time { return l.opts.Now() }
diff --git a/go/internal/cloudlink/link_test.go b/go/internal/cloudlink/link_test.go
index d0493a93..3a5f0296 100644
--- a/go/internal/cloudlink/link_test.go
+++ b/go/internal/cloudlink/link_test.go
@@ -182,8 +182,11 @@ type fakeService struct {
 	afterHello  func(conn *websocket.Conn, home string)
 	rosterSeqOK uint64
 	// wrongShard, when set, closes after hello with CloseWrongShard and
-	// this host, as an instance that does not hold the home.
-	wrongShard string
+	// this host, as an instance that does not hold the home;
+	// restartBeforeOK closes after hello with CloseRestart, as a service
+	// that is stopping.
+	wrongShard      string
+	restartBeforeOK bool
 
 	wmu   sync.Mutex
 	mu    sync.Mutex
@@ -280,6 +283,11 @@ func (s *fakeService) handle(w http.ResponseWriter, r *http.Request) {
 		_, _, _ = conn.ReadMessage()
 		return
 	}
+	if s.restartBeforeOK {
+		closeWith(conn, cloudwire.CloseRestart, "0")
+		_, _, _ = conn.ReadMessage()
+		return
+	}
 
 	okEKM := ekm
 	if s.badEKM {
diff --git a/go/internal/cloudlink/shard_test.go b/go/internal/cloudlink/shard_test.go
index fdd2994d..3c1e4098 100644
--- a/go/internal/cloudlink/shard_test.go
+++ b/go/internal/cloudlink/shard_test.go
@@ -119,9 +119,24 @@ func shardBox(t *testing.T, svc *fakeService, nw *shardNet, pool *x509.CertPool,
 		if keyPath != "" {
 			o.KeyPath = keyPath
 		}
+		o.Now = fastClock()
 	})
 }
 
+// fastClock moves 16 s each time it is read, so three failed dials span
+// the 30 s window that lets a box forget its kept host, without the test
+// waiting for it.
+func fastClock() func() time.Time {
+	var mu sync.Mutex
+	t := time.Now()
+	return func() time.Time {
+		mu.Lock()
+		defer mu.Unlock()
+		t = t.Add(16 * time.Second)
+		return t
+	}
+}
+
 // An instance that does not hold the home names the one that does. The box
 // follows once, keeps the host, and dials it first after a restart.
 func TestTheBoxFollowsAWrongShardCloseOnceAndKeepsTheHost(t *testing.T) {
@@ -241,7 +256,9 @@ func TestARestartCloseRedialsWithinTheSpreadWithoutBackoff(t *testing.T) {
 	}
 	h := newHarness(t, svc)
 	h.setRoster(func(r *fakeRosterState) { r.tun = true })
-	deadline := time.Now().Add(3 * time.Second)
+	// Each spread is at least cloudwire.RestartSpreadMinMs, so five take
+	// up to 5 s; five failures in a row would back off up to 15.5 s.
+	deadline := time.Now().Add(7 * time.Second)
 	for {
 		svc.mu.Lock()
 		n := svc.conns
@@ -250,8 +267,7 @@ func TestARestartCloseRedialsWithinTheSpreadWithoutBackoff(t *testing.T) {
 			break
 		}
 		if time.Now().After(deadline) {
-			// Five failures in a row back off for up to 0.5+1+2+4+8 s.
-			t.Fatalf("%d connections in 3 s after five restarts", n)
+			t.Fatalf("%d connections in 7 s after five restarts", n)
 		}
 		time.Sleep(10 * time.Millisecond)
 	}
@@ -269,6 +285,9 @@ func TestSpreadStaysWithinItsMilliseconds(t *testing.T) {
 			t.Fatalf("spread(40) = %s", d)
 		}
 	}
+	if cloudwire.RestartSpreadMs("0") < cloudwire.RestartSpreadMinMs {
+		t.Fatal("a reason of 0 gives no spread")
+	}
 	if spread(0) != 0 {
 		t.Fatal("spread(0) waits")
 	}
@@ -321,3 +340,57 @@ func TestAfterForgettingAHostTheNextRedirectIsFollowed(t *testing.T) {
 		t.Fatalf("after forgetting s2 the box dialled %v; want the redirect followed at once", got[:want])
 	}
 }
+
+// Three failed dials within a few seconds, as during a shard's restart, do
+// not send the box back to the default host: the window is 30 s.
+func TestAShardRestartDoesNotMakeTheBoxForgetItsHost(t *testing.T) {
+	s1, s2, nw, pool := twoShards(t)
+	kv := &memKV{}
+	h := shardBox(t, s1, nw, pool, kv, "")
+	h.setRoster(func(r *fakeRosterState) { r.tun = true })
+	s2.waitFor(cloudwire.MsgRoster, 5*time.Second)
+	keyPath := h.link.opts.KeyPath
+	stop(h)
+	nw.mu.Lock()
+	addr := nw.at["s2.cloud.test"]
+	delete(nw.at, "s2.cloud.test") // s2 restarts
+	nw.mu.Unlock()
+	h2 := newHarnessWith(t, s1, []*ecdsa.PublicKey{&s1.key.PublicKey}, kv, func(o *Options) {
+		o.Endpoint, o.RootCAs, o.NetDial, o.KeyPath = "wss://cloud.test:443/v1/box", pool, nw.dial, keyPath
+	})
+	h2.setRoster(func(r *fakeRosterState) { r.tun = true })
+	deadline := time.Now().Add(10 * time.Second)
+	for nw.count("s2.cloud.test") < 1+cloudwire.ShardHostFailures+1 && time.Now().Before(deadline) {
+		time.Sleep(20 * time.Millisecond)
+	}
+	before := nw.count("cloud.test")
+	nw.mu.Lock()
+	nw.at["s2.cloud.test"] = addr // s2 is back
+	nw.mu.Unlock()
+	s2.waitFor(cloudwire.MsgRoster, 20*time.Second)
+	if n := nw.count("cloud.test") - before; n != 0 {
+		t.Fatalf("the box went to the default host %d times during its shard's restart", n)
+	}
+}
+
+// A 4503 before hello_ok, again and again, makes the backoff grow: only a
+// box that was connected treats it as a restart.
+func TestARestartBeforeHelloOKKeepsTheBackoff(t *testing.T) {
+	svc := newFakeService(t)
+	svc.restartBeforeOK = true
+	h := newHarness(t, svc)
+	h.setRoster(func(r *fakeRosterState) { r.tun = true })
+	deadline := time.Now().Add(8 * time.Second)
+	for {
+		h.link.mu.Lock()
+		attempt := h.link.attempt
+		h.link.mu.Unlock()
+		if attempt >= 3 {
+			break
+		}
+		if time.Now().After(deadline) {
+			t.Fatalf("attempt %d after 8 s of closes before hello_ok", attempt)
+		}
+		time.Sleep(20 * time.Millisecond)
+	}
+}
diff --git a/go/protocol/cloudwire/shard.go b/go/protocol/cloudwire/shard.go
index d07b4e1a..4c4967c3 100644
--- a/go/protocol/cloudwire/shard.go
+++ b/go/protocol/cloudwire/shard.go
@@ -45,22 +45,28 @@ const (
 	CapShard = "shard"
 
 	// RestartSpreadDefaultMs is the spread when a CloseRestart reason is
-	// missing or not a number; RestartSpreadMaxMs caps any spread.
+	// missing or not a number. RestartSpreadMinMs and RestartSpreadMaxMs
+	// bound any spread: a reason of 0 must not make every client redial in
+	// a tight loop.
 	RestartSpreadDefaultMs = 5_000
+	RestartSpreadMinMs     = 1_000
 	RestartSpreadMaxMs     = 60_000
-	// ShardHostFailures is how many dials in a row to a kept host may fail
-	// before the client forgets it and starts again at DefaultHost.
-	ShardHostFailures = 3
+	// A client forgets a kept host when ShardHostFailures dials in a row
+	// to it have failed over at least ShardHostFailWindowMs, and starts
+	// again at DefaultHost. The window outlasts a shard's restart.
+	ShardHostFailures     = 3
+	ShardHostFailWindowMs = 30_000
 )
 
 // RestartSpreadMs reads a CloseRestart reason: a whole number of
-// milliseconds, at most RestartSpreadMaxMs. Anything else is the default.
+// milliseconds, from RestartSpreadMinMs to RestartSpreadMaxMs. Anything else
+// is the default.
 func RestartSpreadMs(reason string) int64 {
 	ms, err := strconv.ParseInt(strings.TrimSpace(reason), 10, 64)
 	if err != nil || ms < 0 {
 		return RestartSpreadDefaultMs
 	}
-	return min(ms, RestartSpreadMaxMs)
+	return min(max(ms, RestartSpreadMinMs), RestartSpreadMaxMs)
 }
 
 // ShardHostOK reports whether a client of the service at base may follow a
diff --git a/go/protocol/cloudwire/shard_test.go b/go/protocol/cloudwire/shard_test.go
index 8afd2ada..1da498c0 100644
--- a/go/protocol/cloudwire/shard_test.go
+++ b/go/protocol/cloudwire/shard_test.go
@@ -1,47 +1,56 @@
 package cloudwire
 
-import "testing"
+import (
+	"encoding/json"
+	"os"
+	"testing"
+)
 
-func TestShardHostOKStaysOnTheServicesDomain(t *testing.T) {
-	for _, c := range []struct {
-		base, host string
-		ok         bool
-	}{
-		{"cloud.ftw.energy", "cloud.ftw.energy", true},
-		{"cloud.ftw.energy", "s2.cloud.ftw.energy", true},
-		{"cloud.ftw.energy", "eu-2.cloud.ftw.energy", true},
-		{"cloud.ftw.energy", "a.b.cloud.ftw.energy", false},
-		{"cloud.ftw.energy", "S2.cloud.ftw.energy", false},
-		{"cloud.ftw.energy", "-s2.cloud.ftw.energy", false},
-		{"cloud.ftw.energy", "s2-.cloud.ftw.energy", false},
-		{"cloud.ftw.energy", ".cloud.ftw.energy", false},
-		{"cloud.ftw.energy", "s2cloud.ftw.energy", false},
-		{"cloud.ftw.energy", "evil.example", false},
-		{"cloud.ftw.energy", "cloud.ftw.energy.evil.example", false},
-		{"cloud.ftw.energy", "s2.cloud.ftw.energy:8443", false},
-		{"cloud.ftw.energy", "s2.cloud.ftw.energy/v1/box", false},
-		{"cloud.ftw.energy", "", false},
-		{"127.0.0.1:8443", "127.0.0.1:8443", false},
-		{"", "cloud.ftw.energy", false},
-	} {
-		if got := ShardHostOK(c.base, c.host); got != c.ok {
-			t.Errorf("ShardHostOK(%q, %q) = %v, want %v", c.base, c.host, got, c.ok)
+// testdata/cloud-shard-vectors.json is what every client must decide about a
+// wrong-shard host and a restart's spread. A phone that follows Ftw-Shard on
+// /v1/now carries its bearer token there, so the clients must agree to the
+// byte.
+const shardVectorsPath = "testdata/cloud-shard-vectors.json"
+
+type shardVectors struct {
+	ShardHost []struct {
+		Name, Base, Host string
+		OK               bool `json:"ok"`
+	} `json:"shardHost"`
+	RestartSpread []struct {
+		Name, Reason string
+		MS           int64 `json:"ms"`
+	} `json:"restartSpread"`
+}
+
+func loadShardVectors(t *testing.T) shardVectors {
+	t.Helper()
+	raw, err := os.ReadFile(shardVectorsPath)
+	if err != nil {
+		t.Fatal(err)
+	}
+	var v shardVectors
+	if err := json.Unmarshal(raw, &v); err != nil {
+		t.Fatal(err)
+	}
+	if len(v.ShardHost) == 0 || len(v.RestartSpread) == 0 {
+		t.Fatal("empty vectors")
+	}
+	return v
+}
+
+func TestShardHostVectors(t *testing.T) {
+	for _, c := range loadShardVectors(t).ShardHost {
+		if got := ShardHostOK(c.Base, c.Host); got != c.OK {
+			t.Errorf("%s: ShardHostOK(%q, %q) = %v, want %v", c.Name, c.Base, c.Host, got, c.OK)
 		}
 	}
 }
 
-func TestRestartSpreadReadsTheReason(t *testing.T) {
-	for reason, want := range map[string]int64{
-		"2500":   2500,
-		" 800 ":  800,
-		"0":      0,
-		"":       RestartSpreadDefaultMs,
-		"soon":   RestartSpreadDefaultMs,
-		"-1":     RestartSpreadDefaultMs,
-		"999999": RestartSpreadMaxMs,
-	} {
-		if got := RestartSpreadMs(reason); got != want {
-			t.Errorf("RestartSpreadMs(%q) = %d, want %d", reason, got, want)
+func TestRestartSpreadVectors(t *testing.T) {
+	for _, c := range loadShardVectors(t).RestartSpread {
+		if got := RestartSpreadMs(c.Reason); got != c.MS {
+			t.Errorf("%s: RestartSpreadMs(%q) = %d, want %d", c.Name, c.Reason, got, c.MS)
 		}
 	}
 }
diff --git a/go/protocol/cloudwire/testdata/cloud-shard-vectors.json b/go/protocol/cloudwire/testdata/cloud-shard-vectors.json
new file mode 100644
index 00000000..c437dbc0
--- /dev/null
+++ b/go/protocol/cloudwire/testdata/cloud-shard-vectors.json
@@ -0,0 +1,48 @@
+{
+  "comment": "ShardHostOK(base, host) and RestartSpreadMs(reason), as the box, the phone, the watch and the web app must decide them. A client follows a wrong-shard answer only where ok is true. Go: shard_test.go; Swift, Kotlin and TypeScript replay this file.",
+  "shardHost": [
+    {"name": "the default host itself", "base": "cloud.ftw.energy", "host": "cloud.ftw.energy", "ok": true},
+    {"name": "one label under it", "base": "cloud.ftw.energy", "host": "s2.cloud.ftw.energy", "ok": true},
+    {"name": "a label with a hyphen and digits", "base": "cloud.ftw.energy", "host": "eu-2.cloud.ftw.energy", "ok": true},
+    {"name": "a label of 63 characters", "base": "cloud.ftw.energy", "host": "a23456789012345678901234567890123456789012345678901234567890123.cloud.ftw.energy", "ok": true},
+    {"name": "a label of 64 characters", "base": "cloud.ftw.energy", "host": "a234567890123456789012345678901234567890123456789012345678901234.cloud.ftw.energy", "ok": false},
+    {"name": "two labels under it", "base": "cloud.ftw.energy", "host": "a.b.cloud.ftw.energy", "ok": false},
+    {"name": "no dot before the suffix", "base": "cloud.ftw.energy", "host": "s2cloud.ftw.energy", "ok": false},
+    {"name": "an empty label", "base": "cloud.ftw.energy", "host": ".cloud.ftw.energy", "ok": false},
+    {"name": "uppercase in the label", "base": "cloud.ftw.energy", "host": "S2.cloud.ftw.energy", "ok": false},
+    {"name": "uppercase in the base part", "base": "cloud.ftw.energy", "host": "s2.CLOUD.ftw.energy", "ok": false},
+    {"name": "a trailing dot", "base": "cloud.ftw.energy", "host": "s2.cloud.ftw.energy.", "ok": false},
+    {"name": "the default host with a trailing dot", "base": "cloud.ftw.energy", "host": "cloud.ftw.energy.", "ok": false},
+    {"name": "a port", "base": "cloud.ftw.energy", "host": "s2.cloud.ftw.energy:443", "ok": false},
+    {"name": "userinfo before the host", "base": "cloud.ftw.energy", "host": "user@s2.cloud.ftw.energy", "ok": false},
+    {"name": "the host as userinfo of another", "base": "cloud.ftw.energy", "host": "s2.cloud.ftw.energy@evil.example", "ok": false},
+    {"name": "a path", "base": "cloud.ftw.energy", "host": "s2.cloud.ftw.energy/v1/box", "ok": false},
+    {"name": "a scheme", "base": "cloud.ftw.energy", "host": "wss://s2.cloud.ftw.energy", "ok": false},
+    {"name": "a leading hyphen", "base": "cloud.ftw.energy", "host": "-s2.cloud.ftw.energy", "ok": false},
+    {"name": "a trailing hyphen", "base": "cloud.ftw.energy", "host": "s2-.cloud.ftw.energy", "ok": false},
+    {"name": "an underscore", "base": "cloud.ftw.energy", "host": "s_2.cloud.ftw.energy", "ok": false},
+    {"name": "percent encoding", "base": "cloud.ftw.energy", "host": "s2%2ecloud.ftw.energy", "ok": false},
+    {"name": "a non-ASCII letter", "base": "cloud.ftw.energy", "host": "sö.cloud.ftw.energy", "ok": false},
+    {"name": "whitespace", "base": "cloud.ftw.energy", "host": " s2.cloud.ftw.energy", "ok": false},
+    {"name": "another domain", "base": "cloud.ftw.energy", "host": "evil.example", "ok": false},
+    {"name": "the base as a prefix of another domain", "base": "cloud.ftw.energy", "host": "cloud.ftw.energy.evil.example", "ok": false},
+    {"name": "an IP address", "base": "cloud.ftw.energy", "host": "13.50.84.149", "ok": false},
+    {"name": "empty", "base": "cloud.ftw.energy", "host": "", "ok": false},
+    {"name": "a base with a port", "base": "cloud.ftw.energy:443", "host": "cloud.ftw.energy:443", "ok": false},
+    {"name": "no base", "base": "", "host": "cloud.ftw.energy", "ok": false}
+  ],
+  "restartSpread": [
+    {"name": "a spread", "reason": "2500", "ms": 2500},
+    {"name": "spaces around it", "reason": " 800 ", "ms": 1000},
+    {"name": "zero is the floor", "reason": "0", "ms": 1000},
+    {"name": "under the floor", "reason": "999", "ms": 1000},
+    {"name": "the floor", "reason": "1000", "ms": 1000},
+    {"name": "the cap", "reason": "60000", "ms": 60000},
+    {"name": "over the cap", "reason": "999999", "ms": 60000},
+    {"name": "empty is the default", "reason": "", "ms": 5000},
+    {"name": "words are the default", "reason": "soon", "ms": 5000},
+    {"name": "negative is the default", "reason": "-1", "ms": 5000},
+    {"name": "a fraction is the default", "reason": "1500.5", "ms": 5000},
+    {"name": "a plus sign", "reason": "+2000", "ms": 2000}
+  ]
+}

From 7f40a0eccf7ded52ed71fbfc7a6341b5cf036861 Mon Sep 17 00:00:00 2001
From: Fredrik Ahlgren <fredrik@sourceful-labs.com>
Date: Thu, 15 Oct 2026 22:36:09 +0200
Subject: [PATCH 5/5] docs(cloud): the shard rules the last commit left out,
 and MCP stays on the default host

- A kept host is forgotten after 3 failed dials over at least 30 s; the
  spread is 1 s to 60 s; a 4503 before hello_ok or message 2 grows the
  backoff; cloud-shard-vectors.json is named with the other vectors.
- A command in flight at a drain is unknown to the phone, not failed.
- /mcp and /oauth stay on cloud.ftw.energy, which forwards to the home's
  shard once there is a second one: MCP and OAuth clients follow no 4421
  or 421.
- Moving homes needs a tool that comes with the second shard.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---
 .changeset/cloud-shard-redirect.md |  2 +-
 docs/protocol-cloud.md             | 49 +++++++++++++++++++++---------
 2 files changed, 36 insertions(+), 15 deletions(-)

diff --git a/.changeset/cloud-shard-redirect.md b/.changeset/cloud-shard-redirect.md
index 559f57b1..51f0610b 100644
--- a/.changeset/cloud-shard-redirect.md
+++ b/.changeset/cloud-shard-redirect.md
@@ -2,4 +2,4 @@
 "ftw": patch
 ---
 
-The box follows FTW Cloud to the instance that holds its home. When the service closes the uplink with 4421, the box dials the host named in the close reason once, only if it is `cloud.ftw.energy` or one label under it, and keeps that host with its link so a restart goes there first. It goes back to `cloud.ftw.energy` after a second redirect in a row or three failed dials. When the service closes with 4503 before a deploy, the box redials at a random moment within the spread the service names, instead of all boxes coming back within half a second. The box lists `shard` in `hello.caps`. The service still proves itself with the TLS certificate and the service key the box was built with, so a redirect grants nothing.
+The box follows FTW Cloud to the instance that holds its home. When the service closes the uplink with 4421, the box dials the host named in the close reason once, only if it is `cloud.ftw.energy` or one label under it, and keeps that host with its link so a restart goes there first. It goes back to `cloud.ftw.energy` after a second redirect in a row, or after three failed dials over at least 30 s, so a shard's restart does not send it away. When the service closes with 4503 before a deploy, the box redials at a random moment within the spread the service names, at least 1 s, instead of all boxes coming back within half a second; a 4503 before `hello_ok` counts as a failure and grows the backoff. The box lists `shard` in `hello.caps`. The service still proves itself with the TLS certificate and the service key the box was built with, so a redirect grants nothing.
diff --git a/docs/protocol-cloud.md b/docs/protocol-cloud.md
index 2f7676c0..f37dd19c 100644
--- a/docs/protocol-cloud.md
+++ b/docs/protocol-cloud.md
@@ -6,7 +6,9 @@ or the phone. The bytes live in `go/protocol/cloudwire`, with golden vectors
 in its `testdata`: `cloud-vectors.json` for the uplink,
 `cloud-app-vectors.json` for one whole app session, its
 [tunnel](#tunnel) messages and a pairing proof, and
-`cloud-once-vectors.json` for one [one-shot read](#one-shot-read).
+`cloud-once-vectors.json` for one [one-shot read](#one-shot-read), and
+`cloud-shard-vectors.json` for which hosts a client may follow and how it
+reads a restart's spread ([Shards and restarts](#shards-and-restarts)).
 `srcfl/ftw-cloud`, the phone and the watch replay them. `cloudwire.View` is the phone's
 [freshness](#freshness) rules in Go, and the cases listed there are its tests.
 
@@ -1117,7 +1119,10 @@ it does not hold names the host that does:
 
 A client follows a wrong-shard answer only to `cloud.ftw.energy` itself or
 one DNS label of lowercase letters, digits and hyphens directly under it,
-such as `s2.cloud.ftw.energy` (`cloudwire.ShardHostOK`). The redirect carries
+such as `s2.cloud.ftw.energy` (`cloudwire.ShardHostOK`). No port, userinfo,
+uppercase or trailing dot; `cloud-shard-vectors.json` holds the cases every
+client must decide alike. A phone that follows `Ftw-Shard` on `/v1/now`
+carries its bearer token there. The redirect carries
 no trust: the box still checks the TLS certificate for that host and
 `hello_ok` against the service keys it was built with, and the phone still
 runs Noise IK to the service Noise key it carries.
@@ -1127,8 +1132,10 @@ runs Noise IK to the service Noise key it carries.
 - It follows once. A second wrong-shard answer before it is connected again
   sends it back to `cloud.ftw.energy` after backoff, where it may follow once
   more.
-- After 3 dials in a row that fail at a kept host, the client forgets it and
-  starts again at `cloud.ftw.energy`.
+- After 3 dials in a row that fail at a kept host over at least 30 s, the
+  client forgets it and starts again at `cloud.ftw.energy`. The window
+  outlasts a shard's restart, so a restart does not send its clients to
+  the default host.
 - A box lists `shard` in `hello.caps` when it follows 4421 and spreads its
   redial after 4503. A second shard holds homes only once their boxes list
   it.
@@ -1136,19 +1143,33 @@ runs Noise IK to the service Noise key it carries.
 **Directory.** Each instance holds the same directory: the shards' hosts, a
 shard for each of 256 buckets of homes, and single homes moved on their own.
 A home's bucket is the first byte of SHA-256 of `ftw.cloud.shard\0` and the
-home id. Moving homes means copying their registry rows to the new shard,
-then updating the directory on every instance. Their clients follow the
-redirect at their next connection, and extension tokens keep working,
-because the rows hold their hashes.
+home id. Moving homes copies their registry rows to the new shard, then
+updates the directory on every instance; the tool for it comes with the
+second shard. Their clients follow the redirect at their next connection,
+and extension tokens keep working, because the rows hold their hashes and
+each token names its home.
+
+Assistants' MCP and OAuth clients belong to other parties and follow neither
+4421 nor 421. `/mcp` and `/oauth/*` therefore stay on `cloud.ftw.energy`.
+Once there is a second shard, that instance forwards each call to the shard
+that holds the home; the client never sees another host.
 
 **Restart.** An instance that is about to stop, for a deploy or a move, stops
 accepting connections and closes every socket with 4503. The reason is a
 spread in milliseconds, in decimal, which the service sets from how many
 sockets it holds and how fast the next instance can take them. The client
-redials at a uniformly random moment within the spread, at most 60 s, and
-does not count the close as a failed attempt. A missing or unreadable reason
-means 5 s. The service sends no `box.unreachable` for boxes it closed this
-way.
+redials at a uniformly random moment within the spread, from 1 s to 60 s,
+and does not count the close as a failed attempt. A missing or unreadable
+reason means 5 s. A 4503 that comes before `hello_ok`, or before message 2
+on `/v1/app`, is a failed attempt: the backoff grows, so a service that
+sends every client away at once cannot hold them in a loop. The service
+sends no `box.unreachable` for boxes it closed this way.
+
+A command in flight when the service stops may be lost with the socket,
+before or after the box acted on it. A phone treats a command that got no
+answer as unknown, not failed: it shows the box's state once the session is
+back, and sends the command again only if the state says it was not
+applied.
 
 ## Service state
 
@@ -1203,8 +1224,8 @@ the registry's roles would remove that copy.
 | What | Value |
 |---|---|
 | Reconnect backoff, box and phone | 0.5 s to 60 s, full jitter |
-| Redial after 4503 | Uniform within the reason's spread; 5 s without one; at most 60 s |
-| Wrong-shard redirects followed | One per connection; a kept host is forgotten after 3 failed dials |
+| Redial after 4503 | Uniform within the reason's spread, 1 s to 60 s; 5 s without one; backoff when it comes before `hello_ok` or message 2 |
+| Wrong-shard redirects followed | One per connection; a kept host is forgotten after 3 failed dials over at least 30 s |
 | Shard buckets | 256 |
 | Phone retry after 4403 | Half to all of min(15 s × 2ⁿ, 5 min) |
 | Ping, read deadline, write deadline | 15 s, 45 s, 10 s, on both sockets |