diff --git a/CLAUDE.md b/CLAUDE.md index 9f0971f..0634c78 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -19,7 +19,11 @@ maintains it. - **The home id comes from the key that signed hello** on this connection, never from a body field. Every later box message must verify with that key. - **No readings on disk.** Memory holds readings; SQLite holds only the home - id, the box key, the signed roster and tombstones. Nothing logs addresses. + id, the box key, the signed roster, owners' suspensions and tombstones. + Nothing logs addresses. +- **A phone can only narrow.** On `/v1/app` it can say whether it is looking, + and an owner can suspend a roster device. The service admits a phone by the + key its Noise handshake proved, and only if the box's roster lists it. - **TLS ends in `ftwcloudd`.** The handshake is bound to the TLS session; a terminating proxy breaks it on purpose. A TCP load balancer is fine. - **Forward the box's bytes unchanged.** The service cannot sign for a box, diff --git a/README.md b/README.md index e00d613..46a327f 100644 --- a/README.md +++ b/README.md @@ -12,7 +12,7 @@ both in `srcfl/ftw`. Both are proposed. The service runs at `cloud.ftw.energy` ([deploy/README.md](deploy/README.md)), but no box sends to it yet. -## What works today (C2) +## What works today (C2, C3, C4) - A box with FTW Cloud on keeps one WebSocket at `/v1/box`. The service checks that the box signed `hello` over this TLS session's exporter value, @@ -21,35 +21,59 @@ to it yet. - Each home's newest message of each class lives in memory, as the exact bytes the box sent. Readings are dropped an hour after the box goes quiet and are never written to disk. -- The registry (SQLite) holds the home id, the box key and the signed roster. - A box that turns FTW Cloud off gets its home deleted and a signed receipt. +- The registry (SQLite) holds the home id, the box key, the signed roster and + owners' suspensions. A box that turns FTW Cloud off gets its home deleted + and a signed receipt. - `/source` serves the source of the running build, as AGPL-3.0 asks, and names its commit and image digest. +- A phone reads its home at `/v1/app?home=`: Noise IK to the service's + Noise key, with the home id in the prologue. The service admits only a + phone whose key the box listed in its signed roster, and no owner has + suspended. Handshake message 2 carries the snapshot from memory; after it + each new box envelope the phone may read follows on the same socket, + unchanged. A roster change, a suspension, an expired roster or a purge + closes the sessions it touches at once. +- While a phone looks, the service asks the box for one-second updates and + asks again every 50 s; when nobody looks, it stops. +- Limits per home: 16 sessions, one per device, 256 envelopes queued per + phone, and a small message rate from each phone. +- **Alerts (C4).** A box's signed `event` goes to its phones through APNs, + wrapped unchanged: the phone checks the signature, writes its own words and + picks the buttons from the verified kind. Every alert has the same aps, + priority and push type and no collapse id. Ten minutes without the box's + socket sends `box.unreachable`, unless the box said `bye` for an update, + which holds it back for an hour. A phone registers its token over its app + session with `push.register`. Tokens are bound to roster entries that may + read `ftw.events.read`, get nothing while an owner has suspended the + device, and go when the roster drops them or Apple says they are gone. + Apple's key comes from Parameter Store + ([deploy/apns.md](deploy/apns.md)). Not built yet, with a marked place in the code: | Step | What | Where | |---|---|---| -| C3 | The app's Noise session, `/v1/app` | `internal/server`, `store.For` with a `Device` principal | -| C4 | Alerts over APNs, `box.unreachable` | not started | | C5 | Widget token reads, `/v1/now`, tiles | `internal/server` | | C7–C8 | Statistics | not started | ## How it holds a home ``` -box ──wss, TLS 1.3, signed frames──▶ /v1/box ──▶ store (memory) ◀── store.For(principal) - └────▶ registry (SQLite: key, roster, tombstones) +box ──wss, TLS 1.3, signed frames──▶ /v1/box ──▶ store (memory) ◀── store.For(principal) ◀── /v1/app ◀──Noise IK── phone + └────▶ registry (SQLite: key, roster, tombstones, suspensions) ``` - **The home id is a hash of the box's key.** It comes from the key that signed `hello`, never from a field anyone sends. A second box cannot claim a home that is not its own. - **One way in.** `store.For(principal)` is the only access to a home: a - `Box` principal may write its own home, and a `Device` principal (C3) may + `Box` principal may write its own home, and a `Device` principal may read the classes its role and the owner's grant both allow. There is no listing and no principal for staff; `TestNoRouteReadsAHome` checks every route. +- **A phone can narrow and nothing else.** It can say whether it is looking, + and an owner can suspend a device in the roster. Commands go from the app + to the box over the relay, never through here. - **The service sends five things.** `challenge`, `hello_ok`, `watch`, `resync` and `purged` are the whole downlink, and none asks the box to act. The box closes the uplink on anything else. @@ -61,9 +85,10 @@ box ──wss, TLS 1.3, signed frames──▶ /v1/box ──▶ store (memory) ```bash go build -tags ftwclouddev -o ftwcloudd ./cmd/ftwcloudd -./ftwcloudd keygen -out service.key # prints the public key +./ftwcloudd keygen -out service.key # prints the public key, for Core +./ftwcloudd keygen -noise -out noise.key # prints the Noise public key, for the app ./ftwcloudd -listen 127.0.0.1:18443 -state ./state -service-key service.key \ - -tls-cert leaf.pem -tls-key leaf.key + -noise-key noise.key -tls-cert leaf.pem -tls-key leaf.key ``` Build Core from `srcfl/ftw` against it. A release build names no service, so @@ -85,15 +110,30 @@ curl -X POST http://127.0.0.1:8080/api/cloud/grant \ curl http://127.0.0.1:8080/api/cloud/preview ``` +Without `-noise-key` (or with the file missing) the service still serves +boxes, and `/v1/app` answers 501: phones read through the relay until a +Noise key is installed. A host that should serve phones needs the key as a +second secret beside the service key. + `-tags ftwclouddev` adds `/dev/held?home=` on loopback, which shows what -the service holds so a test can compare it with the preview. Release builds -do not have it. +the service holds so a test can compare it with the preview, and +`-dev-push-listen 127.0.0.1:18444`, where `POST /dev/push/register` with +`{home, deviceId, apnsToken (hex), environment}` registers a token for a +device in the home's roster, as the app session does; `POST /dev/push/raw` +with `{home, payload}` sends a payload as it is, so a test can put a forged +one through Apple. `-dev-apns-capture ` writes alerts to files instead +of sending them. `-apns-dir` names a +directory holding `key.p8`, `key-id` and `team-id`; without it nothing is +sent. Release builds have neither dev route. ## Develop ```bash go vet ./... && go test -race ./... go test -run XXX -fuzz FuzzBoxMessage -fuzztime 30s ./internal/uplink +go test -run XXX -fuzz FuzzPhoneMessage -fuzztime 30s ./internal/appsession +go test -run TestTheSnapshotArrivesFast -v ./internal/appsession # loopback p50/p95 +go test -run XXX -bench BenchmarkSnapshot ./internal/appsession bash scripts/check-imports.sh ``` diff --git a/cmd/ftwcloudd/devpush_dev.go b/cmd/ftwcloudd/devpush_dev.go new file mode 100644 index 0000000..f97ff97 --- /dev/null +++ b/cmd/ftwcloudd/devpush_dev.go @@ -0,0 +1,169 @@ +//go:build ftwclouddev + +package main + +import ( + "context" + "encoding/hex" + "encoding/json" + "errors" + "fmt" + "log/slog" + "net" + "net/http" + "os" + "path/filepath" + "slices" + "sync" + "time" + + "github.com/srcfl/ftw-cloud/internal/apns" + "github.com/srcfl/ftw-cloud/internal/push" + "github.com/srcfl/ftw-cloud/internal/registry" + "github.com/srcfl/ftw/go/protocol/cloudwire" +) + +// serveDevPush listens on loopback for two routes. Release builds have +// neither. +// +// - POST /dev/push/register registers an APNs token for a device in a +// home's signed roster, as the app's session will, so an end-to-end test +// can run before the app session can. +// - POST /dev/push/raw {home, payload} sends a payload as it is to the +// home's registered phones, so a test can put a forged one through Apple +// and see the phone refuse it. +func serveDevPush(ctx context.Context, addr string, reg *registry.Registry, svc *push.Service, sender push.Sender, log *slog.Logger) error { + host, _, err := net.SplitHostPort(addr) + if err != nil || !net.ParseIP(host).IsLoopback() { + return errors.New("-dev-push-listen must be a loopback address") + } + mux := http.NewServeMux() + mux.HandleFunc("POST /dev/push/register", func(w http.ResponseWriter, r *http.Request) { + var req struct { + Home string `json:"home"` + DeviceID string `json:"deviceId"` + Token string `json:"apnsToken"` + Environment string `json:"environment"` + } + if json.NewDecoder(r.Body).Decode(&req) != nil { + http.Error(w, "bad request", http.StatusBadRequest) + return + } + token, err := hex.DecodeString(req.Token) + if err != nil { + http.Error(w, "apnsToken is hex", http.StatusBadRequest) + return + } + dev, scopes, ok := rosterDevice(reg, req.Home, req.DeviceID) + if !ok { + http.Error(w, "no such device in the home's roster", http.StatusNotFound) + return + } + if err := svc.Register(req.Home, dev, scopes, cloudwire.PushRegister{ + APNsToken: token, Environment: req.Environment, Kind: cloudwire.PushKindAlert}); err != nil { + http.Error(w, err.Error(), http.StatusForbidden) + return + } + w.WriteHeader(http.StatusNoContent) + }) + mux.HandleFunc("POST /dev/push/raw", func(w http.ResponseWriter, r *http.Request) { + var req struct { + Home string `json:"home"` + Payload json.RawMessage `json:"payload"` + } + if json.NewDecoder(r.Body).Decode(&req) != nil || len(req.Payload) == 0 { + http.Error(w, "bad request", http.StatusBadRequest) + return + } + if sender == nil { + http.Error(w, "no APNs key", http.StatusServiceUnavailable) + return + } + regs, err := reg.PushRegs(req.Home, cloudwire.PushKindAlert) + if err != nil || len(regs) == 0 { + http.Error(w, "no phone registered for that home", http.StatusNotFound) + return + } + var out []string + for _, x := range regs { + res, err := sender.Send(r.Context(), apns.Notification{Environment: x.Environment, DeviceToken: x.Token, + Payload: req.Payload, Expiration: time.Now().Add(10 * time.Minute)}) + out = append(out, fmt.Sprintf("%d %s %v", res.Status, res.Reason, err)) + } + _ = json.NewEncoder(w).Encode(out) + }) + srv := &http.Server{Addr: addr, Handler: mux, ReadHeaderTimeout: 5 * time.Second} + go func() { + <-ctx.Done() + _ = srv.Close() + }() + go func() { + if err := srv.ListenAndServe(); err != nil && !errors.Is(err, http.ErrServerClosed) { + log.Warn("dev push route stopped", "err", err) + } + }() + log.Warn("development build: tokens can be registered by hand", "addr", addr) + return nil +} + +// captureSender writes each alert to dir as .json, {environment, +// deviceToken, payload}, and tells the service Apple took it. A test hands +// the payload to a simulator with xcrun simctl push. +func captureSender(dir string) (push.Sender, error) { + if err := os.MkdirAll(dir, 0o700); err != nil { + return nil, err + } + return &capture{dir: dir}, nil +} + +type capture struct { + mu sync.Mutex + dir string + n int +} + +func (c *capture) Send(_ context.Context, n apns.Notification) (apns.Result, error) { + c.mu.Lock() + defer c.mu.Unlock() + c.n++ + raw, err := json.Marshal(map[string]any{"environment": n.Environment, + "deviceToken": hex.EncodeToString(n.DeviceToken), "payload": json.RawMessage(n.Payload)}) + if err != nil { + return apns.Result{}, err + } + name := filepath.Join(c.dir, fmt.Sprintf("%04d.json", c.n)) + if err := os.WriteFile(name+".tmp", raw, 0o600); err != nil { + return apns.Result{}, err + } + return apns.Result{Status: http.StatusOK}, os.Rename(name+".tmp", name) +} + +// rosterDevice finds a device in the roster the box last signed, with its +// effective scope. +func rosterDevice(reg *registry.Registry, home, id string) (cloudwire.Device, []string, bool) { + h, ok, err := reg.Get(home) + if err != nil || !ok || h.Roster == nil { + return cloudwire.Device{}, nil, false + } + env, err := cloudwire.DecodeEnvelope(h.Roster) + if err != nil { + return cloudwire.Device{}, nil, false + } + var r cloudwire.Roster + if cloudwire.Unmarshal(env.B, &r) != nil { + return cloudwire.Device{}, nil, false + } + for _, d := range r.Devices { + if d.ID != id { + continue + } + var scopes []string + for _, s := range d.Scopes { + if slices.Contains(r.Grant.Scopes, s) { + scopes = append(scopes, s) + } + } + return d, scopes, true + } + return cloudwire.Device{}, nil, false +} diff --git a/cmd/ftwcloudd/devpush_other.go b/cmd/ftwcloudd/devpush_other.go new file mode 100644 index 0000000..68afd55 --- /dev/null +++ b/cmd/ftwcloudd/devpush_other.go @@ -0,0 +1,22 @@ +//go:build !ftwclouddev + +package main + +import ( + "context" + "errors" + "log/slog" + + "github.com/srcfl/ftw-cloud/internal/push" + "github.com/srcfl/ftw-cloud/internal/registry" +) + +// captureSender exists only in development builds. +func captureSender(string) (push.Sender, error) { + return nil, errors.New("-dev-apns-capture needs a build with -tags ftwclouddev") +} + +// serveDevPush exists only in development builds. +func serveDevPush(context.Context, string, *registry.Registry, *push.Service, push.Sender, *slog.Logger) error { + return errors.New("-dev-push-listen needs a build with -tags ftwclouddev") +} diff --git a/cmd/ftwcloudd/main.go b/cmd/ftwcloudd/main.go index e758bd8..f0d89f7 100644 --- a/cmd/ftwcloudd/main.go +++ b/cmd/ftwcloudd/main.go @@ -2,12 +2,14 @@ // home that turned it on, in memory, and serves it to that home's phones. It // reads and never controls. See README.md. // -// ftwcloudd keygen -out service.key make the service key; prints its public half -// ftwcloudd [flags] serve +// ftwcloudd keygen -out service.key make the service key; prints its public half +// ftwcloudd keygen -noise -out noise.key make the Noise key phones dial; prints its public half +// ftwcloudd [flags] serve package main import ( "context" + "crypto/ecdh" "crypto/ecdsa" "crypto/elliptic" "crypto/rand" @@ -25,13 +27,18 @@ import ( "os" "os/signal" "path/filepath" + "strings" "syscall" "time" + "github.com/srcfl/ftw-cloud/internal/apns" + "github.com/srcfl/ftw-cloud/internal/appsession" + "github.com/srcfl/ftw-cloud/internal/push" "github.com/srcfl/ftw-cloud/internal/registry" "github.com/srcfl/ftw-cloud/internal/server" "github.com/srcfl/ftw-cloud/internal/store" "github.com/srcfl/ftw-cloud/internal/uplink" + "github.com/srcfl/ftw/go/protocol/appwire" "github.com/srcfl/ftw/go/protocol/cloudwire" "golang.org/x/crypto/acme/autocert" ) @@ -56,9 +63,14 @@ func serve(args []string) error { domain := fs.String("domain", "cloud.ftw.energy", "name to get a certificate for") stateDir := fs.String("state", "/state", "directory for the registry and certificates") keyPath := fs.String("service-key", "/run/secrets/ftw-cloud-service.key", "service key, PEM") + noisePath := fs.String("noise-key", "/run/secrets/ftw-cloud-noise.key", "Noise key phones dial, PEM; without it /v1/app is off") certFile := fs.String("tls-cert", "", "certificate file, instead of ACME (local tests)") keyFile := fs.String("tls-key", "", "certificate key file, with -tls-cert") email := fs.String("acme-email", "", "contact for the certificate authority") + apnsDir := fs.String("apns-dir", "/run/secrets/apns", "directory with Apple's key.p8, key-id and team-id; without it no alert is sent") + apnsTopic := fs.String("apns-topic", "energy.ftw.app", "the app's bundle id") + devPush := fs.String("dev-push-listen", "", "loopback address for registering a token by hand (development builds only)") + devCapture := fs.String("dev-apns-capture", "", "write each alert to this directory instead of sending it to Apple (development builds only)") _ = fs.Parse(args) harden() @@ -67,6 +79,15 @@ func serve(args []string) error { if err != nil { return err } + // Phones read through /v1/app only with a Noise key. A host without one + // still serves boxes, and the app reads through the relay meanwhile. + noise, err := loadNoiseKey(*noisePath) + noiseOn := err == nil + if errors.Is(err, os.ErrNotExist) { + log.Warn("no Noise key: phones cannot read from FTW Cloud until one is installed", "path", *noisePath) + } else if err != nil { + return err + } reg, err := registry.Open(filepath.Join(*stateDir, "registry.db")) if err != nil { return err @@ -74,7 +95,44 @@ func serve(args []string) error { defer reg.Close() st := store.New(nil) - srv := &server.Server{Store: st, Registry: reg, Uplink: uplink.New(st, reg, key, log), Log: log} + st.UseSuspensions(reg.Suspended) + up := uplink.New(st, reg, key, log) + srv := &server.Server{Store: st, Registry: reg, Uplink: up, Log: log} + + // Alerts: the box's signed events to its phones through Apple, and + // box.unreachable when a box is away. Without a key the service keeps + // registrations and sends nothing. + var sender push.Sender + switch client, err := loadAPNs(*apnsDir, *apnsTopic); { + case err != nil: + return err + case *devCapture != "": + if sender, err = captureSender(*devCapture); err != nil { + return err + } + case client == nil: + log.Warn("APNs off: no key in " + *apnsDir) + default: + sender = client + } + pushSvc := push.New(reg, sender, log) + up.OnEvent = pushSvc.Event + up.OnRoster = pushSvc.Roster + up.OnGone = pushSvc.BoxGone + + // Phones register their tokens over their app session, so without a + // Noise key only the development route registers one. + if noiseOn { + srv.App = appsession.New(st, reg, up, noise, log) + srv.App.Push = pushSvc + } + app := srv.App + up.OnConnect = func(home string) { + if app != nil { + app.BoxConnected(home) + } + pushSvc.BoxConnected(home) + } // TLS ends here and nowhere else: each box binds its handshake to this // TLS session, so a terminating proxy in front would break it. A plain @@ -111,6 +169,12 @@ func serve(args []string) error { ctx, stop := signal.NotifyContext(context.Background(), syscall.SIGINT, syscall.SIGTERM) defer stop() go srv.Upkeep(ctx, time.Minute) + go pushSvc.Run(ctx) + if *devPush != "" { + if err := serveDevPush(ctx, *devPush, reg, pushSvc, sender, log); err != nil { + return err + } + } go func() { <-ctx.Done() shutdown, cancel := context.WithTimeout(context.Background(), 5*time.Second) @@ -119,7 +183,8 @@ func serve(args []string) error { }() pub, _ := cloudwire.PublicKeyBytesOf(&key.PublicKey) - log.Info("ftwcloudd listening", "addr", *listen, "service_key", base64.RawURLEncoding.EncodeToString(pub)) + log.Info("ftwcloudd listening", "addr", *listen, "service_key", base64.RawURLEncoding.EncodeToString(pub), + "noise_key", base64.RawURLEncoding.EncodeToString(noise.Public())) if err := httpSrv.ListenAndServeTLS("", ""); err != nil && !errors.Is(err, http.ErrServerClosed) { return err } @@ -128,6 +193,32 @@ func serve(args []string) error { func log0() *log.Logger { return log.New(io.Discard, "", 0) } +// loadAPNs reads Apple's key from dir: key.p8, key-id and team-id, as the +// host writes them from SSM Parameter Store at boot. No directory or no key +// is no sender; a key that is there and wrong is an error. +func loadAPNs(dir, topic string) (*apns.Client, error) { + p8, err := os.ReadFile(filepath.Join(dir, "key.p8")) + if errors.Is(err, os.ErrNotExist) { + return nil, nil + } + if err != nil { + return nil, fmt.Errorf("apns key: %w", err) + } + read := func(name string) (string, error) { + b, err := os.ReadFile(filepath.Join(dir, name)) + return strings.TrimSpace(string(b)), err + } + keyID, err := read("key-id") + if err != nil { + return nil, fmt.Errorf("apns key id: %w", err) + } + teamID, err := read("team-id") + if err != nil { + return nil, fmt.Errorf("apns team id: %w", err) + } + return apns.New(apns.Config{Key: p8, KeyID: keyID, TeamID: teamID, Topic: topic}) +} + func loadServiceKey(path string) (*ecdsa.PrivateKey, error) { raw, err := os.ReadFile(path) if err != nil { @@ -147,15 +238,53 @@ func loadServiceKey(path string) (*ecdsa.PrivateKey, error) { return key, nil } -// keygen writes a new service key and prints its public half, which Core -// carries in cloudlink.ServiceKeys. +// loadNoiseKey reads the X25519 key phones run Noise IK against, PKCS #8 PEM. +func loadNoiseKey(path string) (appwire.KeyPair, error) { + raw, err := os.ReadFile(path) + if err != nil { + return appwire.KeyPair{}, fmt.Errorf("noise key: %w", err) + } + block, _ := pem.Decode(raw) + if block == nil || block.Type != "PRIVATE KEY" { + return appwire.KeyPair{}, fmt.Errorf("noise key: %s is not a PKCS #8 private key", path) + } + parsed, err := x509.ParsePKCS8PrivateKey(block.Bytes) + if err != nil { + return appwire.KeyPair{}, fmt.Errorf("noise key: %w", err) + } + x, ok := parsed.(*ecdh.PrivateKey) + if !ok || x.Curve() != ecdh.X25519() { + return appwire.KeyPair{}, fmt.Errorf("noise key: %s is not an X25519 key", path) + } + return appwire.KeyPairFromSecret(x.Bytes()) +} + +// keygen writes a new key and prints its public half. The service key's goes +// into Core's cloudlink.ServiceKeys; with -noise, the Noise key's goes into +// the app. func keygen(args []string) error { fs := flag.NewFlagSet("keygen", flag.ExitOnError) out := fs.String("out", "", "file to write the key to") + noise := fs.Bool("noise", false, "make the X25519 key phones dial instead of the P-256 service key") _ = fs.Parse(args) if *out == "" { return errors.New("keygen: -out is required") } + if *noise { + x, err := ecdh.X25519().GenerateKey(rand.Reader) + if err != nil { + return err + } + der, err := x509.MarshalPKCS8PrivateKey(x) + if err != nil { + return err + } + if err := writePEM(*out, "PRIVATE KEY", der); err != nil { + return err + } + fmt.Println(base64.RawURLEncoding.EncodeToString(x.PublicKey().Bytes())) + return nil + } key, err := ecdsa.GenerateKey(elliptic.P256(), rand.Reader) if err != nil { return err @@ -164,15 +293,7 @@ func keygen(args []string) error { if err != nil { return err } - f, err := os.OpenFile(*out, os.O_WRONLY|os.O_CREATE|os.O_EXCL, 0o600) - if err != nil { - return err - } - if err := pem.Encode(f, &pem.Block{Type: "EC PRIVATE KEY", Bytes: der}); err != nil { - _ = f.Close() - return err - } - if err := f.Close(); err != nil { + if err := writePEM(*out, "EC PRIVATE KEY", der); err != nil { return err } pub, err := cloudwire.PublicKeyBytesOf(&key.PublicKey) @@ -182,3 +303,17 @@ func keygen(args []string) error { fmt.Println(base64.RawURLEncoding.EncodeToString(pub)) return nil } + +// writePEM writes a new key file that only its owner can read, and never over +// an existing one. +func writePEM(path, kind string, der []byte) error { + f, err := os.OpenFile(path, os.O_WRONLY|os.O_CREATE|os.O_EXCL, 0o600) + if err != nil { + return err + } + if err := pem.Encode(f, &pem.Block{Type: kind, Bytes: der}); err != nil { + _ = f.Close() + return err + } + return f.Close() +} diff --git a/deploy/apns.md b/deploy/apns.md new file mode 100644 index 0000000..2ec8968 --- /dev/null +++ b/deploy/apns.md @@ -0,0 +1,66 @@ +# Apple's push key + +`ftwcloudd` sends alerts through APNs with a token key (`.p8`) from Sourceful's +Apple team. Only the owner handles the key: he makes it at Apple and puts it +in AWS Systems Manager Parameter Store himself. The host reads it from there +into tmpfs at every boot. It is never in the repository, the image, the volume +or a log. + +## 1. Make the key (owner, once) + +1. [developer.apple.com/account](https://developer.apple.com/account) → + Certificates, Identifiers & Profiles → **Keys** → **+**. +2. Name it `FTW Cloud APNs`, tick **Apple Push Notifications service (APNs)**, + and choose **Configure**: environment **Sandbox & Production**; key + restriction **Topic Specific** with `energy.ftw.app`, or **Team Scoped**. + Save, then **Continue** → **Register**. +3. **Download** `AuthKey_.p8`. Apple lets you download it once. Note the + 10-character **Key ID**. The Team ID is `L66M368PZ6`. +4. Under **Identifiers** → `energy.ftw.app`, check that **Push Notifications** + and **Time Sensitive Notifications** are on. Xcode turns them on the first + time it signs the app with the new entitlements. + +## 2. Put it in Parameter Store (owner, once) + +In a terminal signed in to the FTW Cloud account (976075258475), with the +file where Safari saved it: + +```bash +aws ssm put-parameter --region eu-north-1 --name /ftw-cloud/apns/key --type SecureString --value "file://$HOME/Downloads/AuthKey_KEYID.p8" +aws ssm put-parameter --region eu-north-1 --name /ftw-cloud/apns/key-id --type String --value KEYID +aws ssm put-parameter --region eu-north-1 --name /ftw-cloud/apns/team-id --type String --value L66M368PZ6 +``` + +Then keep the `.p8` in the password manager or delete it. A new key replaces +an old one with `--overwrite`; revoke the old one at Apple afterwards. + +## 3. Give it to the host (once per host) + +`deploy/apns.sh` installs `ftw-cloud-apns.service`, which reads the three +parameters into `/run/ftw-cloud/apns/` (tmpfs, readable only by the +container's user) before Docker starts. `compose.yml` mounts that directory +read-only. Run it through SSM, then redeploy so a new container mounts the +key: + +```bash +aws ssm send-command --region eu-north-1 --instance-ids i-0ecac4cbc6b5f8adc \ + --document-name AWS-RunShellScript \ + --parameters 'commands=["bash /opt/ftw-cloud/deploy/apns.sh","/usr/local/sbin/ftw-cloud-deploy"]' +``` + +The log then has no `APNs off` line. Without the parameters the service runs +and sends no alerts. + +## Access + +The host role needs nothing new. `AmazonSSMManagedInstanceCore` allows +`ssm:GetParameter` on every parameter, and the account's `aws/ssm` key allows +decrypting through SSM for every principal in the account. Checked on +2026-10-13 with `iam simulate-principal-policy` (`ssm:GetParameter` allowed) +and the key policy of `alias/aws/ssm` in eu-north-1. Secrets Manager would +need a policy change; Parameter Store does not. + +srcfl/ftw-cloud#3 moves the service key to Secrets Manager with an inline +policy for that one secret. The APNs key could follow it there: one more ARN +in `deploy/host-policy.json`, applied by the owner, and `get-secret-value` +in `apns.sh`. Nothing else changes. diff --git a/deploy/apns.sh b/deploy/apns.sh new file mode 100755 index 0000000..ea8b554 --- /dev/null +++ b/deploy/apns.sh @@ -0,0 +1,74 @@ +#!/bin/bash +# Give ftwcloudd Apple's push key. Run as root on the FTW Cloud host through +# SSM, after the owner has put the key in Parameter Store (deploy/apns.md). +# Safe to re-run. +# +# The key rests only in Parameter Store, as a SecureString under the +# account's aws/ssm key. At every boot, before Docker starts, a unit reads it +# into tmpfs, where compose.yml mounts it into the container. It is never +# written to the volume or the image, and nothing prints it. +# +# The host role reads it with what it already has: AmazonSSMManagedInstanceCore +# allows ssm:GetParameter, and the aws/ssm key's policy allows decrypting +# through SSM for every principal in the account. No IAM change is needed. +set -euo pipefail + +cat > /usr/local/sbin/ftw-cloud-apns <<'FETCH' +#!/bin/bash +# Read Apple's push key from Parameter Store into tmpfs for ftwcloudd. +# Without the parameters ftwcloudd runs and sends no alerts. +set -euo pipefail +export AWS_PAGER="" +umask 077 +dir=/run/ftw-cloud/apns +install -d -m 0700 /run/ftw-cloud +new=$(mktemp -d /run/ftw-cloud/apns.XXXXXX) +trap 'rm -rf "$new"' EXIT +# get NAME FILE: 0 when read, 1 when the parameter does not exist. At boot +# the network and the instance's credentials may lag a little. +get() { + local err + for try in $(seq 30); do + if err=$(aws ssm get-parameter --region eu-north-1 --name "$1" --with-decryption \ + --query Parameter.Value --output text 2>&1 >"$new/$2"); then + return 0 + fi + case "$err" in *ParameterNotFound*) return 1 ;; esac + [ "$try" -lt 30 ] || { echo "$err" >&2; exit 1; } + sleep 2 + done +} +if ! get /ftw-cloud/apns/key key.p8 || ! get /ftw-cloud/apns/key-id key-id || ! get /ftw-cloud/apns/team-id team-id; then + echo "no APNs key in Parameter Store; ftwcloudd will send no alerts" + exit 0 +fi +grep -q '^-----BEGIN PRIVATE KEY-----$' "$new/key.p8" +chown -R 65532:65532 "$new" +chmod 0400 "$new"/* +chmod 0500 "$new" +rm -rf "$dir" +mv "$new" "$dir" +trap - EXIT +echo "APNs key in place" +FETCH +chmod 0755 /usr/local/sbin/ftw-cloud-apns + +cat > /etc/systemd/system/ftw-cloud-apns.service <<'UNIT' +[Unit] +Description=FTW Cloud APNs key into tmpfs +Before=docker.service +Wants=network-online.target +After=network-online.target + +[Service] +Type=oneshot +RemainAfterExit=yes +ExecStart=/usr/local/sbin/ftw-cloud-apns + +[Install] +WantedBy=docker.service +UNIT + +systemctl daemon-reload +systemctl enable ftw-cloud-apns.service +systemctl restart ftw-cloud-apns.service diff --git a/deploy/compose.yml b/deploy/compose.yml index 9cc519f..50d3e3b 100644 --- a/deploy/compose.yml +++ b/deploy/compose.yml @@ -26,6 +26,9 @@ services: volumes: # The registry and certificates. No readings are ever written here. - ftw-cloud-state:/state + # Apple's push key, read from Parameter Store into tmpfs at boot by + # deploy/apns.sh. Empty means no alerts, and the service runs. + - /run/ftw-cloud/apns:/run/secrets/apns:ro secrets: - ftw-cloud-service.key read_only: true diff --git a/go.mod b/go.mod index 2c89de0..f23c894 100644 --- a/go.mod +++ b/go.mod @@ -5,7 +5,7 @@ go 1.26.0 require ( github.com/fxamacker/cbor/v2 v2.9.4 github.com/gorilla/websocket v1.5.3 - github.com/srcfl/ftw/go/protocol v0.0.0-20261012183740-8a030e2e4805 + github.com/srcfl/ftw/go/protocol v0.0.0-20261013075909-a9e6f66f1b89 golang.org/x/crypto v0.57.0 golang.org/x/sys v0.48.0 modernc.org/sqlite v1.59.0 diff --git a/go.sum b/go.sum index 70f63f7..aeb676e 100644 --- a/go.sum +++ b/go.sum @@ -16,8 +16,8 @@ github.com/ncruces/go-strftime v1.0.0 h1:HMFp8mLCTPp341M/ZnA4qaf7ZlsbTc+miZjCLOF github.com/ncruces/go-strftime v1.0.0/go.mod h1:Fwc5htZGVVkseilnfgOVb9mKy6w1naJmn9CehxcKcls= github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec h1:W09IVJc94icq4NjY3clb7Lk8O1qJ8BdBEF8z0ibU0rE= github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec/go.mod h1:qqbHyh8v60DhA7CoWK5oRCqLrMHRGoxYCSS9EjAz6Eo= -github.com/srcfl/ftw/go/protocol v0.0.0-20261012183740-8a030e2e4805 h1:DmeVlx+LhctSDD4sQS1ltcN8QD2EdaKiHtqU1Qn3K1U= -github.com/srcfl/ftw/go/protocol v0.0.0-20261012183740-8a030e2e4805/go.mod h1:VEjr/o0yZijJ1nApuJK+nak6FxJ/fubAUBNOunqNrUU= +github.com/srcfl/ftw/go/protocol v0.0.0-20261013075909-a9e6f66f1b89 h1:QXz3pQKjju/Kaa9NaR9I8Mut5r1Hkv+sohz+scB9Vns= +github.com/srcfl/ftw/go/protocol v0.0.0-20261013075909-a9e6f66f1b89/go.mod h1:VEjr/o0yZijJ1nApuJK+nak6FxJ/fubAUBNOunqNrUU= github.com/x448/float16 v0.8.4 h1:qLwI1I70+NjRFUR3zs1JPUCgaCXSh3SW62uAKT1mSBM= github.com/x448/float16 v0.8.4/go.mod h1:14CWIYCyZA/cWjXOioeEpHeN/83MdbZDRQHoFcYsOfg= golang.org/x/crypto v0.57.0 h1:3ZVCjf8Ggz7zneR/EHRVx68Ctf+2pmIMP2UFhh9cC6M= diff --git a/internal/apns/apns.go b/internal/apns/apns.go new file mode 100644 index 0000000..d16b37a --- /dev/null +++ b/internal/apns/apns.go @@ -0,0 +1,266 @@ +// Package apns sends alerts to Apple Push Notification service: HTTP/2 to +// Apple's provider API, authenticated with a token (a JWT signed with ES256 +// by the team's .p8 key) that is renewed every 40 minutes. +// +// It knows nothing about homes or events. The caller hands it a device +// token and the payload bytes, and gets back what Apple said, including +// whether the token is gone for good. +package apns + +import ( + "bytes" + "context" + "crypto/ecdsa" + "crypto/rand" + "crypto/sha256" + "crypto/tls" + "crypto/x509" + "encoding/base64" + "encoding/hex" + "encoding/json" + "encoding/pem" + "errors" + "fmt" + "io" + "net/http" + "strconv" + "sync" + "time" +) + +// Apple's hosts. Sandbox serves development builds of the app; production +// serves TestFlight and the App Store. +const ( + ProductionURL = "https://api.push.apple.com" + SandboxURL = "https://api.sandbox.push.apple.com" +) + +// Environments, as a phone names them when it registers. +const ( + Sandbox = "sandbox" + Production = "production" +) + +// TokenLife is how long one provider token is used. Apple refuses a token +// older than an hour and one renewed more often than every 20 minutes. +const TokenLife = 40 * time.Minute + +// requestTimeout bounds one request. +const requestTimeout = 10 * time.Second + +// Config is one team key and the app it sends for. +type Config struct { + // Key is the .p8 file Apple issued: a PKCS #8 P-256 key in PEM. + Key []byte + // KeyID is the key's 10-character id; TeamID the team's. + KeyID string + TeamID string + // Topic is the app's bundle id. + Topic string + + // Seams for tests. Production leaves them zero. + SandboxURL string + ProductionURL string + HTTPClient *http.Client + Now func() time.Time +} + +// Client sends to Apple. It is safe for concurrent use and keeps one HTTP/2 +// connection per host. +type Client struct { + cfg Config + key *ecdsa.PrivateKey + http *http.Client + now func() time.Time + + mu sync.Mutex + token string + tokenAt time.Time +} + +// New parses the key and returns a client. +func New(cfg Config) (*Client, error) { + key, err := ParseKey(cfg.Key) + if err != nil { + return nil, err + } + if len(cfg.KeyID) != 10 || len(cfg.TeamID) != 10 || cfg.Topic == "" { + return nil, errors.New("apns: a 10-character key id, a 10-character team id and a topic are required") + } + if cfg.SandboxURL == "" { + cfg.SandboxURL = SandboxURL + } + if cfg.ProductionURL == "" { + cfg.ProductionURL = ProductionURL + } + if cfg.Now == nil { + cfg.Now = time.Now + } + client := cfg.HTTPClient + if client == nil { + client = &http.Client{Transport: &http.Transport{ + TLSClientConfig: &tls.Config{MinVersion: tls.VersionTLS12}, + ForceAttemptHTTP2: true, + IdleConnTimeout: time.Hour, + }} + } + return &Client{cfg: cfg, key: key, http: client, now: cfg.Now}, nil +} + +// ParseKey reads a .p8: PEM around a PKCS #8 P-256 key. +func ParseKey(p8 []byte) (*ecdsa.PrivateKey, error) { + block, _ := pem.Decode(p8) + if block == nil || block.Type != "PRIVATE KEY" { + return nil, errors.New("apns: the key is not a PEM private key") + } + parsed, err := x509.ParsePKCS8PrivateKey(block.Bytes) + if err != nil { + return nil, fmt.Errorf("apns: %w", err) + } + key, ok := parsed.(*ecdsa.PrivateKey) + if !ok || key.Curve.Params().Name != "P-256" { + return nil, errors.New("apns: the key is not a P-256 key") + } + return key, nil +} + +// Notification is one push. +type Notification struct { + // Environment is Sandbox or Production. + Environment string + // DeviceToken is the token the phone got from iOS. + DeviceToken []byte + Payload []byte + // Expiration is when Apple may stop trying to deliver. Zero means + // deliver now or not at all. + Expiration time.Time +} + +// Result is Apple's answer. +type Result struct { + // Status is the HTTP status; 200 is delivered to Apple. + Status int + // Reason is Apple's reason string when Status is not 200. + Reason string + // Gone means the token no longer names a phone for this app: forget it. + Gone bool + // GoneSince is when Apple last knew the token was valid, for a 410. + GoneSince time.Time +} + +// OK reports whether Apple took the push. +func (r Result) OK() bool { return r.Status == http.StatusOK } + +// Send posts one alert. Every alert has the same priority and push type, +// and no collapse id, so nothing in the request says what kind it is. +func (c *Client) Send(ctx context.Context, n Notification) (Result, error) { + base := c.cfg.ProductionURL + switch n.Environment { + case Production: + case Sandbox: + base = c.cfg.SandboxURL + default: + return Result{}, fmt.Errorf("apns: unknown environment %q", n.Environment) + } + if len(n.DeviceToken) == 0 { + return Result{}, errors.New("apns: no device token") + } + res, err := c.post(ctx, base, n, false) + if err == nil && (res.Reason == "ExpiredProviderToken" || res.Reason == "InvalidProviderToken") { + // A token Apple no longer takes: make a new one and try once more. + res, err = c.post(ctx, base, n, true) + } + return res, err +} + +func (c *Client) post(ctx context.Context, base string, n Notification, renew bool) (Result, error) { + token, err := c.providerToken(renew) + if err != nil { + return Result{}, err + } + ctx, cancel := context.WithTimeout(ctx, requestTimeout) + defer cancel() + req, err := http.NewRequestWithContext(ctx, http.MethodPost, base+"/3/device/"+hex.EncodeToString(n.DeviceToken), + bytes.NewReader(n.Payload)) + if err != nil { + return Result{}, err + } + req.Header.Set("authorization", "bearer "+token) + req.Header.Set("apns-topic", c.cfg.Topic) + req.Header.Set("apns-push-type", "alert") + req.Header.Set("apns-priority", "10") + exp := int64(0) + if !n.Expiration.IsZero() { + exp = n.Expiration.Unix() + } + req.Header.Set("apns-expiration", strconv.FormatInt(exp, 10)) + req.Header.Set("content-type", "application/json") + + resp, err := c.http.Do(req) + if err != nil { + return Result{}, fmt.Errorf("apns: %w", err) + } + defer resp.Body.Close() + res := Result{Status: resp.StatusCode} + if resp.StatusCode == http.StatusOK { + _, _ = io.Copy(io.Discard, resp.Body) + return res, nil + } + var body struct { + Reason string `json:"reason"` + Timestamp int64 `json:"timestamp"` + } + _ = json.NewDecoder(io.LimitReader(resp.Body, 4096)).Decode(&body) + res.Reason = body.Reason + switch { + case resp.StatusCode == http.StatusGone: + res.Gone = true + if body.Timestamp > 0 { + res.GoneSince = time.UnixMilli(body.Timestamp) + } + case body.Reason == "BadDeviceToken" || body.Reason == "DeviceTokenNotForTopic": + res.Gone = true + } + return res, nil +} + +// providerToken is the JWT Apple authenticates the provider by, renewed +// after TokenLife or when renew asks. +func (c *Client) providerToken(renew bool) (string, error) { + c.mu.Lock() + defer c.mu.Unlock() + now := c.now() + if !renew && c.token != "" && now.Sub(c.tokenAt) < TokenLife { + return c.token, nil + } + token, err := signToken(c.key, c.cfg.KeyID, c.cfg.TeamID, now) + if err != nil { + return "", err + } + c.token, c.tokenAt = token, now + return token, nil +} + +// signToken makes Apple's provider token: {alg: ES256, kid} . {iss, iat}, +// signed with ES256 as r ‖ s. +func signToken(key *ecdsa.PrivateKey, keyID, teamID string, now time.Time) (string, error) { + enc := base64.RawURLEncoding + header, err := json.Marshal(map[string]string{"alg": "ES256", "kid": keyID}) + if err != nil { + return "", err + } + claims, err := json.Marshal(map[string]any{"iss": teamID, "iat": now.Unix()}) + if err != nil { + return "", err + } + signing := enc.EncodeToString(header) + "." + enc.EncodeToString(claims) + digest := sha256.Sum256([]byte(signing)) + r, s, err := ecdsa.Sign(rand.Reader, key, digest[:]) + if err != nil { + return "", err + } + sig := make([]byte, 64) + r.FillBytes(sig[:32]) + s.FillBytes(sig[32:]) + return signing + "." + enc.EncodeToString(sig), nil +} diff --git a/internal/apns/apns_test.go b/internal/apns/apns_test.go new file mode 100644 index 0000000..42fe07a --- /dev/null +++ b/internal/apns/apns_test.go @@ -0,0 +1,236 @@ +package apns + +import ( + "context" + "crypto/ecdsa" + "crypto/elliptic" + "crypto/rand" + "crypto/sha256" + "crypto/x509" + "encoding/base64" + "encoding/json" + "encoding/pem" + "io" + "math/big" + "net/http" + "net/http/httptest" + "strings" + "sync" + "testing" + "time" +) + +func testP8(t *testing.T) ([]byte, *ecdsa.PrivateKey) { + t.Helper() + key, err := ecdsa.GenerateKey(elliptic.P256(), rand.Reader) + if err != nil { + t.Fatal(err) + } + der, err := x509.MarshalPKCS8PrivateKey(key) + if err != nil { + t.Fatal(err) + } + return pem.EncodeToMemory(&pem.Block{Type: "PRIVATE KEY", Bytes: der}), key +} + +// fakeApple is Apple's provider API, over HTTP/2 with TLS. +type fakeApple struct { + t *testing.T + pub *ecdsa.PublicKey + srv *httptest.Server + + mu sync.Mutex + reqs []*http.Request + bodies [][]byte + tokens []string + respond func(r *http.Request) (int, string) +} + +func newFakeApple(t *testing.T, pub *ecdsa.PublicKey) *fakeApple { + f := &fakeApple{t: t, pub: pub} + f.srv = httptest.NewUnstartedServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + body, _ := io.ReadAll(r.Body) + f.mu.Lock() + f.reqs = append(f.reqs, r) + f.bodies = append(f.bodies, body) + f.tokens = append(f.tokens, strings.TrimPrefix(r.Header.Get("authorization"), "bearer ")) + respond := f.respond + f.mu.Unlock() + status, reason := http.StatusOK, "" + if respond != nil { + status, reason = respond(r) + } + w.Header().Set("apns-id", "00000000-0000-0000-0000-000000000000") + w.WriteHeader(status) + if reason != "" { + body := map[string]any{"reason": reason} + if status == http.StatusGone { + body["timestamp"] = int64(1_791_820_800_000) + } + _ = json.NewEncoder(w).Encode(body) + } + })) + f.srv.EnableHTTP2 = true + f.srv.StartTLS() + t.Cleanup(f.srv.Close) + return f +} + +func (f *fakeApple) client(t *testing.T, p8 []byte, now func() time.Time) *Client { + t.Helper() + c, err := New(Config{Key: p8, KeyID: "ABC123DEFG", TeamID: "L66M368PZ6", Topic: "energy.ftw.app", + SandboxURL: f.srv.URL + "/sandbox", ProductionURL: f.srv.URL, HTTPClient: f.srv.Client(), Now: now}) + if err != nil { + t.Fatal(err) + } + return c +} + +// checkToken verifies a provider token the way Apple does. +func (f *fakeApple) checkToken(token string) (header map[string]string, claims map[string]any) { + f.t.Helper() + parts := strings.Split(token, ".") + if len(parts) != 3 { + f.t.Fatalf("token %q", token) + } + dec := base64.RawURLEncoding + h, _ := dec.DecodeString(parts[0]) + c, _ := dec.DecodeString(parts[1]) + sig, _ := dec.DecodeString(parts[2]) + if json.Unmarshal(h, &header) != nil || json.Unmarshal(c, &claims) != nil || len(sig) != 64 { + f.t.Fatalf("token %q does not parse", token) + } + digest := sha256.Sum256([]byte(parts[0] + "." + parts[1])) + if !ecdsa.Verify(f.pub, digest[:], new(big.Int).SetBytes(sig[:32]), new(big.Int).SetBytes(sig[32:])) { + f.t.Fatal("token signature does not verify") + } + return header, claims +} + +func TestAnAlertGoesOverHTTP2WithTheSameHeadersForEveryKind(t *testing.T) { + p8, key := testP8(t) + apple := newFakeApple(t, &key.PublicKey) + now := time.Unix(1_791_820_800, 0) + c := apple.client(t, p8, func() time.Time { return now }) + + payload := []byte(`{"aps":{"mutable-content":1},"ftw":{"v":1}}`) + res, err := c.Send(context.Background(), Notification{Environment: Production, + DeviceToken: []byte{0xab, 0xcd}, Payload: payload, Expiration: now.Add(time.Hour)}) + if err != nil || !res.OK() { + t.Fatalf("send: %+v %v", res, err) + } + r := apple.reqs[0] + if r.ProtoMajor != 2 { + t.Fatalf("Apple was reached over HTTP/%d", r.ProtoMajor) + } + if r.Method != http.MethodPost || r.URL.Path != "/3/device/abcd" { + t.Fatalf("%s %s", r.Method, r.URL.Path) + } + for k, want := range map[string]string{ + "apns-topic": "energy.ftw.app", "apns-push-type": "alert", "apns-priority": "10", + "apns-expiration": "1791824400", "content-type": "application/json", + } { + if got := r.Header.Get(k); got != want { + t.Errorf("%s = %q, want %q", k, got, want) + } + } + if r.Header.Get("apns-collapse-id") != "" { + t.Error("a collapse id says what kind an alert is") + } + if string(apple.bodies[0]) != string(payload) { + t.Fatalf("payload %s", apple.bodies[0]) + } + header, claims := apple.checkToken(apple.tokens[0]) + if header["alg"] != "ES256" || header["kid"] != "ABC123DEFG" || + claims["iss"] != "L66M368PZ6" || claims["iat"] != float64(now.Unix()) { + t.Fatalf("token %v %v", header, claims) + } +} + +func TestTheProviderTokenIsRenewedEveryFortyMinutes(t *testing.T) { + p8, key := testP8(t) + apple := newFakeApple(t, &key.PublicKey) + now := time.Unix(1_791_820_800, 0) + c := apple.client(t, p8, func() time.Time { return now }) + send := func() { + t.Helper() + if _, err := c.Send(context.Background(), Notification{Environment: Sandbox, DeviceToken: []byte{1}, Payload: []byte("{}")}); err != nil { + t.Fatal(err) + } + } + send() + now = now.Add(39 * time.Minute) + send() + now = now.Add(2 * time.Minute) + send() + if apple.tokens[0] != apple.tokens[1] { + t.Fatal("renewed inside 40 minutes") + } + if apple.tokens[2] == apple.tokens[1] { + t.Fatal("still the same token after 41 minutes") + } + if !strings.HasPrefix(apple.reqs[0].URL.Path, "/sandbox/3/device/") { + t.Fatalf("sandbox went to %s", apple.reqs[0].URL.Path) + } +} + +func TestAnExpiredProviderTokenIsRenewedAndTriedOnce(t *testing.T) { + p8, key := testP8(t) + apple := newFakeApple(t, &key.PublicKey) + calls := 0 + apple.respond = func(*http.Request) (int, string) { + calls++ + if calls == 1 { + return http.StatusForbidden, "ExpiredProviderToken" + } + return http.StatusOK, "" + } + c := apple.client(t, p8, nil) + res, err := c.Send(context.Background(), Notification{Environment: Production, DeviceToken: []byte{1}, Payload: []byte("{}")}) + if err != nil || !res.OK() || calls != 2 { + t.Fatalf("%+v %v after %d calls", res, err, calls) + } +} + +func TestAGoneTokenSaysSo(t *testing.T) { + p8, key := testP8(t) + apple := newFakeApple(t, &key.PublicKey) + for _, c := range []struct { + status int + reason string + gone bool + }{ + {http.StatusGone, "Unregistered", true}, + {http.StatusBadRequest, "BadDeviceToken", true}, + {http.StatusBadRequest, "DeviceTokenNotForTopic", true}, + {http.StatusTooManyRequests, "TooManyRequests", false}, + {http.StatusServiceUnavailable, "ServiceUnavailable", false}, + } { + apple.respond = func(*http.Request) (int, string) { return c.status, c.reason } + res, err := apple.client(t, p8, nil).Send(context.Background(), + Notification{Environment: Production, DeviceToken: []byte{1}, Payload: []byte("{}")}) + if err != nil || res.OK() || res.Gone != c.gone || res.Reason != c.reason { + t.Fatalf("%d %s: %+v %v", c.status, c.reason, res, err) + } + if c.status == http.StatusGone && res.GoneSince.UnixMilli() != 1_791_820_800_000 { + t.Fatalf("gone since %v", res.GoneSince) + } + } +} + +func TestOnlyAnAppleKeyIsTaken(t *testing.T) { + if _, err := New(Config{Key: []byte("not a key"), KeyID: "ABC123DEFG", TeamID: "L66M368PZ6", Topic: "x"}); err == nil { + t.Fatal("took garbage") + } + p8, _ := testP8(t) + if _, err := New(Config{Key: p8, KeyID: "short", TeamID: "L66M368PZ6", Topic: "x"}); err == nil { + t.Fatal("took a key id Apple never issues") + } + c, err := New(Config{Key: p8, KeyID: "ABC123DEFG", TeamID: "L66M368PZ6", Topic: "x"}) + if err != nil { + t.Fatal(err) + } + if _, err := c.Send(context.Background(), Notification{Environment: "staging", DeviceToken: []byte{1}}); err == nil { + t.Fatal("sent to an environment that does not exist") + } +} diff --git a/internal/appsession/appsession.go b/internal/appsession/appsession.go new file mode 100644 index 0000000..8835240 --- /dev/null +++ b/internal/appsession/appsession.go @@ -0,0 +1,505 @@ +// Package appsession serves /v1/app, where a phone reads its home. +// docs/protocol-cloud.md in srcfl/ftw is the wire format ("App session"); +// go/protocol/cloudwire holds its bytes. +// +// The phone runs Noise IK as initiator with the service's Noise key as the +// responder's static key and the home id in the prologue. The service admits +// the phone only through store.For with the key the handshake authenticated, +// so a phone reaches only a home whose box listed it in a signed roster. +// Message 2 carries the snapshot; after it every envelope the box sends that +// the device may read goes out unchanged, one frame per Noise message. +// +// A phone may ask for faster updates, and an owner may suspend a device. That +// is all it can do here: commands go to the box over the relay, never +// through FTW Cloud. +package appsession + +import ( + "crypto/tls" + "errors" + "log/slog" + "net/http" + "sync" + "time" + + "github.com/gorilla/websocket" + "github.com/srcfl/ftw-cloud/internal/push" + "github.com/srcfl/ftw-cloud/internal/registry" + "github.com/srcfl/ftw-cloud/internal/store" + "github.com/srcfl/ftw/go/protocol/appwire" + "github.com/srcfl/ftw/go/protocol/cloudwire" +) + +const ( + // maxPhoneMessage bounds one message from a phone. Message 1 with now is + // about 120 bytes and nothing a phone sends later is larger. + maxPhoneMessage = 4 << 10 + // handshakeTimeout bounds message 1. + handshakeTimeout = 10 * time.Second + // The same ping, read and write deadlines as the uplink. + pingEvery = 15 * time.Second + readTimeout = 45 * time.Second + writeTimeout = 10 * time.Second + // A phone sends a message when it looks away or back, or when an owner + // suspends a device. Past this burst at this rate the session closes. + rateBurst = 20 + ratePerSec = 2 + // maxNoiseMessage is the largest Noise message, handshake or transport. + maxNoiseMessage = 0xffff + // WatchRenew is how often the service asks the box again for one-second + // updates while a phone looks. The box's watch lasts 60 s, so the ask + // comes before it lapses. + WatchRenew = 50 * time.Second + // roleOwner is the registry's owner role, as the roster names it. + // go/protocol does not generate role names yet. + roleOwner = "owner" +) + +// Watcher asks a home's box for one-second updates. The uplink is one. +type Watcher interface { + Watch(home string) +} + +// Server is the /v1/app handler. +type Server struct { + store *store.Store + registry *registry.Registry + watcher Watcher + key appwire.KeyPair + log *slog.Logger + now func() time.Time + // renewEvery is WatchRenew; tests shorten it. + renewEvery time.Duration + + // Push keeps the phones' APNs tokens. Nil ignores push.register and + // push.unregister. Set before serving. + Push Pusher + + mu sync.Mutex + looking map[string]*look +} + +// Pusher keeps a device's APNs tokens. push.Service is one. dev and scopes +// come from store.For, never from the phone's message. +type Pusher interface { + Register(home string, dev cloudwire.Device, scopes []string, r cloudwire.PushRegister) error + Unregister(home string, dev cloudwire.Device, r cloudwire.PushUnregister) error +} + +// look is one home's phones that can see it, and the loop that keeps the +// box's watch in force while there is at least one. +type look struct { + n int + stop chan struct{} +} + +// New builds the handler. key is the service's Noise static key; the app +// carries its public half. +func New(st *store.Store, reg *registry.Registry, w Watcher, key appwire.KeyPair, log *slog.Logger) *Server { + if log == nil { + log = slog.Default() + } + return &Server{store: st, registry: reg, watcher: w, key: key, log: log, now: time.Now, + renewEvery: WatchRenew, looking: map[string]*look{}} +} + +var upgrader = websocket.Upgrader{ + EnableCompression: false, + // The app sends no Origin. A browser always does, and has no business + // here. + CheckOrigin: func(r *http.Request) bool { return r.Header.Get("Origin") == "" }, +} + +func (s *Server) ServeHTTP(w http.ResponseWriter, r *http.Request) { + if r.TLS == nil || r.TLS.Version != tls.VersionTLS13 { + http.Error(w, "TLS 1.3 only", http.StatusUpgradeRequired) + return + } + home := r.URL.Query().Get(cloudwire.AppHomeParam) + if !cloudwire.ValidHome(home) { + http.Error(w, "name a home", http.StatusBadRequest) + return + } + conn, err := upgrader.Upgrade(w, r, nil) + if err != nil { + return + } + defer conn.Close() + conn.SetReadLimit(maxPhoneMessage) + code, err := s.serve(conn, home) + if err != nil { + s.log.Debug("app session closed", "err", err) + } + if code != 0 { + _ = conn.WriteControl(websocket.CloseMessage, websocket.FormatCloseMessage(code, ""), time.Now().Add(time.Second)) + } +} + +var errProtocol = errors.New("appsession: the phone broke the protocol") + +// serve runs one session and returns the close code to end it with. +func (s *Server) serve(conn *websocket.Conn, home string) (int, error) { + _ = conn.SetReadDeadline(time.Now().Add(handshakeTimeout)) + kind, msg1, err := conn.ReadMessage() + if err != nil { + return 0, err + } + if kind != websocket.BinaryMessage { + return cloudwire.CloseProtocol, errProtocol + } + hs, err := appwire.NewResponder(appwire.HandshakeOptions{StaticKey: s.key, Prologue: cloudwire.AppPrologue(home)}) + if err != nil { + return websocket.CloseInternalServerErr, err + } + payload, err := hs.ReadMessage(msg1) + if err != nil { + // Another service key, or another home in the prologue than in the + // query. Neither says anything about the home. + return cloudwire.CloseDenied, err + } + hz, err := parseNow(payload) + if err != nil { + return cloudwire.CloseProtocol, err + } + + dev, err := s.store.For(store.Principal{Kind: store.Device, Home: home, DeviceKey: hs.RemoteStatic()}) + if err != nil { + return cloudwire.CloseDenied, err + } + snap, sess, err := dev.Open() + switch { + case errors.Is(err, store.ErrBusy): + return cloudwire.CloseBusy, err + case err != nil: + return cloudwire.CloseDenied, err + } + defer sess.Close() + + msg2Payload, rest, err := snapshotPayload(snap) + if err != nil { + return websocket.CloseInternalServerErr, err + } + msg2, err := hs.WriteMessage(msg2Payload) + if err != nil { + return websocket.CloseInternalServerErr, err + } + c := &phoneConn{s: s, conn: conn, dev: dev, home: home, tokens: rateBurst, refilled: s.now()} + if err := c.write(msg2); err != nil { + return 0, err + } + keys, err := hs.Split() + if err != nil { + return websocket.CloseInternalServerErr, err + } + c.tr = appwire.NewTransport(keys) + for _, env := range rest { + if err := c.forward(env); err != nil { + return 0, err + } + } + + c.setLooking(hz >= cloudwire.HzLooking) + defer c.stopLooking() + return c.run(sess) +} + +// parseNow reads message 1's payload. +func parseNow(payload []byte) (int, error) { + env, err := cloudwire.DecodeEnvelope(payload) + if err != nil || env.T != cloudwire.MsgNow { + return 0, errProtocol + } + var now cloudwire.Now + if err := cloudwire.Unmarshal(env.B, &now); err != nil { + return 0, errProtocol + } + return now.Hz, nil +} + +// snapshotPayload is message 2. Envelopes that do not fit in one Noise +// message come back as rest, to follow as frames. +func snapshotPayload(snap store.Snapshot) ([]byte, [][]byte, error) { + envs := snap.Envelopes + var rest [][]byte + for { + payload, err := cloudwire.SnapshotPayload(envs, snap.LastSeenMs, snap.Online, len(rest)) + if err != nil { + return nil, nil, err + } + if len(payload)+appwire.Message2Overhead <= maxNoiseMessage || len(envs) == 0 { + return payload, rest, nil + } + rest = append([][]byte{envs[len(envs)-1]}, rest...) + envs = envs[:len(envs)-1] + } +} + +// setLooking keeps the box's watch in force while at least one phone of the +// home can see it. +func (s *Server) setLooking(home string, delta int) { + s.mu.Lock() + defer s.mu.Unlock() + l := s.looking[home] + if l == nil { + l = &look{} + s.looking[home] = l + } + l.n += delta + switch { + case l.n > 0 && l.stop == nil: + l.stop = make(chan struct{}) + go s.renew(home, l.stop) + case l.n <= 0: + if l.stop != nil { + close(l.stop) + } + delete(s.looking, home) + } +} + +func (s *Server) renew(home string, stop <-chan struct{}) { + s.watcher.Watch(home) + t := time.NewTicker(s.renewEvery) + defer t.Stop() + for { + select { + case <-stop: + return + case <-t.C: + s.watcher.Watch(home) + } + } +} + +// BoxConnected asks a box that just connected for one-second updates when a +// phone is already looking. The uplink calls it after hello_ok. +func (s *Server) BoxConnected(home string) { + s.mu.Lock() + looking := s.looking[home] != nil + s.mu.Unlock() + if looking { + s.watcher.Watch(home) + } +} + +// phoneConn is one phone's session after the handshake. +type phoneConn struct { + s *Server + conn *websocket.Conn + tr *appwire.Transport + dev *store.Handle + home string + + // looking is touched by the read loop and by the session's end. + lookMu sync.Mutex + looking bool + ended bool + + tokens float64 + refilled time.Time +} + +func (c *phoneConn) setLooking(on bool) { + c.lookMu.Lock() + defer c.lookMu.Unlock() + if c.ended || on == c.looking { + return + } + c.looking = on + delta := -1 + if on { + delta = 1 + } + c.s.setLooking(c.home, delta) +} + +// stopLooking ends this phone's part in the watch for good, whatever the +// read loop still delivers. +func (c *phoneConn) stopLooking() { + c.setLooking(false) + c.lookMu.Lock() + c.ended = true + c.lookMu.Unlock() +} + +// write sends one WebSocket message. Only the session's own goroutine +// writes; pings go through WriteControl, which may run beside it. +func (c *phoneConn) write(msg []byte) error { + if err := c.conn.SetWriteDeadline(time.Now().Add(writeTimeout)); err != nil { + return err + } + return c.conn.WriteMessage(websocket.BinaryMessage, msg) +} + +// forward sends one box envelope as the box sent it. An envelope too large +// for a Noise message cannot reach a phone; it is dropped, and the phone sees +// the gap through the next hb. +func (c *phoneConn) forward(env []byte) error { + frame, err := cloudwire.EncodeFrame(env) + if err != nil || len(frame)+appwire.TransportOverhead > maxNoiseMessage { + return nil + } + msg, err := c.tr.Encrypt(frame) + if err != nil { + return err + } + return c.write(msg) +} + +// run forwards until the store ends the session, the phone leaves or breaks +// the protocol. +func (c *phoneConn) run(sess *store.Session) (int, error) { + type result struct { + code int + err error + } + read := make(chan result, 1) + go func() { + code, err := c.readLoop() + read <- result{code, err} + }() + ping := time.NewTicker(pingEvery) + defer ping.Stop() + for { + select { + case env := <-sess.C(): + if err := c.forward(env); err != nil { + return 0, err + } + case <-sess.Done(): + return sess.Code(), nil + case r := <-read: + // The read loop's own work (a suspension) may have ended this + // session; the store's reason wins over a plain close. + select { + case <-sess.Done(): + return sess.Code(), r.err + default: + } + return r.code, r.err + case <-ping.C: + if err := c.conn.WriteControl(websocket.PingMessage, nil, time.Now().Add(writeTimeout)); err != nil { + return 0, err + } + } + } +} + +func (c *phoneConn) readLoop() (int, error) { + _ = c.conn.SetReadDeadline(time.Now().Add(readTimeout)) + c.conn.SetPongHandler(func(string) error { + return c.conn.SetReadDeadline(time.Now().Add(readTimeout)) + }) + for { + kind, msg, err := c.conn.ReadMessage() + if err != nil { + return 0, err + } + _ = c.conn.SetReadDeadline(time.Now().Add(readTimeout)) + if !c.allow() { + return cloudwire.CloseProtocol, errProtocol + } + if kind != websocket.BinaryMessage { + return cloudwire.CloseProtocol, errProtocol + } + frame, err := c.tr.Decrypt(msg) + if err != nil { + return cloudwire.CloseProtocol, err + } + if code, err := c.apply(frame); err != nil { + return code, err + } + } +} + +// apply acts on one message from the phone. Types it does not know are +// ignored, so a newer app works with an older service. +func (c *phoneConn) apply(frame []byte) (int, error) { + payload, err := cloudwire.DecodeFrame(frame) + if err != nil { + return cloudwire.CloseProtocol, err + } + env, err := cloudwire.DecodeEnvelope(payload) + if err != nil { + return cloudwire.CloseProtocol, err + } + switch env.T { + case cloudwire.MsgWatch: + var w cloudwire.Watch + if err := cloudwire.Unmarshal(env.B, &w); err != nil { + return cloudwire.CloseProtocol, err + } + c.setLooking(w.Hz >= cloudwire.HzLooking) + case cloudwire.MsgSuspend, cloudwire.MsgUnsuspend: + var sus cloudwire.Suspend + if err := cloudwire.Unmarshal(env.B, &sus); err != nil { + return cloudwire.CloseProtocol, err + } + return 0, c.suspend(sus.DeviceID, env.T == cloudwire.MsgSuspend) + case cloudwire.MsgPushRegister: + var r cloudwire.PushRegister + if err := cloudwire.Unmarshal(env.B, &r); err != nil { + return cloudwire.CloseProtocol, err + } + if c.s.Push != nil { + return 0, pushed(c.s.Push.Register(c.home, c.dev.Device(), c.dev.Scopes(), r)) + } + case cloudwire.MsgPushUnregister: + var r cloudwire.PushUnregister + if err := cloudwire.Unmarshal(env.B, &r); err != nil { + return cloudwire.CloseProtocol, err + } + if c.s.Push != nil { + return 0, pushed(c.s.Push.Unregister(c.home, c.dev.Device(), r)) + } + } + return 0, nil +} + +// pushed lets the session go on after a registration the device may not +// make, such as a viewer's: it gets no alerts, and nothing else changes. +func pushed(err error) error { + if errors.Is(err, push.ErrNotForThisDevice) { + return nil + } + return err +} + +// suspend applies an owner's suspension or lifts one. It only narrows what +// the roster allows: the device must be in the roster, and lifting a +// suspension gives back no more than the roster says. +func (c *phoneConn) suspend(deviceID string, on bool) error { + if c.dev.Device().Role != roleOwner { + return nil + } + r, _ := c.dev.Roster() + if r == nil { + return nil + } + for _, d := range r.Devices { + if d.ID != deviceID { + continue + } + if !on { + return c.s.registry.Unsuspend(c.home, d.ID) + } + if err := c.s.registry.Suspend(c.home, d.ID, d.Epoch, c.s.now()); err != nil { + return err + } + c.dev.Recheck() + return nil + } + return nil +} + +// allow is a token bucket over the phone's messages. +func (c *phoneConn) allow() bool { + now := c.s.now() + c.tokens = min(rateBurst, c.tokens+now.Sub(c.refilled).Seconds()*ratePerSec) + c.refilled = now + if c.tokens < 1 { + return false + } + c.tokens-- + return true +} diff --git a/internal/appsession/appsession_test.go b/internal/appsession/appsession_test.go new file mode 100644 index 0000000..408da70 --- /dev/null +++ b/internal/appsession/appsession_test.go @@ -0,0 +1,354 @@ +package appsession + +import ( + "bytes" + "crypto/ecdsa" + "testing" + "time" + + "github.com/gorilla/websocket" + "github.com/srcfl/ftw-cloud/internal/store" + "github.com/srcfl/ftw/go/protocol" + "github.com/srcfl/ftw/go/protocol/appwire" + "github.com/srcfl/ftw/go/protocol/cloudwire" +) + +// The snapshot carries the box's own bytes in rising seq, and every later +// envelope follows on the same connection, unchanged. +func TestSnapshotThenStreamCarryTheBoxsBytes(t *testing.T) { + h := newHarness(t) + full := h.live(true) + plan := h.plan() + delta := h.live(false) + hb := h.hb() + + p := h.mustDial(dialOpts{key: h.owner, hz: 1}) + want := [][]byte{full, plan, delta, hb} + if len(p.snap.Envelopes) != len(want) || !p.snap.Online || p.snap.More != 0 { + t.Fatalf("snapshot: %d envelopes, online %v, more %d", len(p.snap.Envelopes), p.snap.Online, p.snap.More) + } + pub := &h.boxKey.PublicKey + for i, e := range p.snap.Envelopes { + if !bytes.Equal(e, want[i]) { + t.Fatalf("snapshot entry %d is not the box's bytes", i) + } + verify(t, pub, h.home, e) + } + + next := h.live(false) + got, code := p.next(2 * time.Second) + if code != 0 || !bytes.Equal(got, next) { + t.Fatalf("the stream carried %x (code %d), not the box's envelope", got, code) + } + verify(t, pub, h.home, got) +} + +func verify(t *testing.T, pub *ecdsa.PublicKey, home string, raw []byte) { + t.Helper() + env, err := cloudwire.DecodeEnvelope(raw) + if err != nil { + t.Fatal(err) + } + if err := cloudwire.Verify(pub, home, env); err != nil { + t.Fatalf("%s does not verify with the box key: %v", env.T, err) + } +} + +// A viewer reads its effective scope, on the snapshot and on the stream. +func TestAViewerReadsOnlyItsScope(t *testing.T) { + h := newHarness(t) + full := h.live(true) + h.plan() + p := h.mustDial(dialOpts{key: h.viewer, hz: 1}) + if len(p.snap.Envelopes) != 1 || !bytes.Equal(p.snap.Envelopes[0], full) { + t.Fatalf("viewer snapshot: %d envelopes", len(p.snap.Envelopes)) + } + h.plan() + hb := h.hb() + got, _ := p.next(2 * time.Second) + if !bytes.Equal(got, hb) { + t.Fatal("the viewer was sent a plan, or not the hb") + } +} + +// Every refusal closes before message 2 with the one code that says nothing +// about which check failed. +func TestRefusalsSayNothing(t *testing.T) { + h := newHarness(t) + h.live(true) + stranger, _ := appwire.GenerateKeyPair() + otherService, _ := appwire.GenerateKeyPair() + otherHome := "aaaaaaaaaaaaaaaaaaaaaaaaaa" + for name, o := range map[string]dialOpts{ + "a key the roster lacks": {key: stranger}, + "another service key": {key: h.owner, service: otherService.Public()}, + "another home in prologue": {key: h.owner, prologue: otherHome}, + "a home the service lacks": {key: h.owner, home: otherHome}, + } { + _, code, err := h.dial(o) + if err != nil || code != cloudwire.CloseDenied { + t.Errorf("%s: code %d, %v", name, code, err) + } + } +} + +// The doc's C3 proof: a revoke cuts the stream within a second. +func TestARosterWithoutTheDeviceClosesItsSessionWithinASecond(t *testing.T) { + h := newHarness(t) + h.live(true) + viewer := h.mustDial(dialOpts{key: h.viewer, hz: 1}) + owner := h.mustDial(dialOpts{key: h.owner, hz: 1}) + + start := time.Now() + h.setRoster(h.devices(1, 1)[:1], protocol.ScopeLiveRead, protocol.ScopePlanRead) + if code := viewer.closed(time.Second); code != cloudwire.CloseDenied { + t.Fatalf("revoked viewer closed with %d", code) + } + if took := time.Since(start); took > time.Second { + t.Fatalf("revoke took %v", took) + } + // The owner's access did not change, so its stream goes on. + next := h.live(false) + if got, code := owner.next(time.Second); code != 0 || !bytes.Equal(got, next) { + t.Fatalf("the owner's stream ended with the viewer's: code %d", code) + } + if _, code, _ := h.dial(dialOpts{key: h.viewer}); code != cloudwire.CloseDenied { + t.Fatalf("a revoked device reconnected: code %d", code) + } +} + +// Narrower scopes or a new epoch end the session; the phone comes back and +// learns what it may read now. +func TestAChangedAccessEndsTheSession(t *testing.T) { + h := newHarness(t) + h.live(true) + owner := h.mustDial(dialOpts{key: h.owner}) + h.setRoster(h.devices(1, 1), protocol.ScopeLiveRead) // plan dropped from the grant + if code := owner.closed(time.Second); code != cloudwire.CloseChanged { + t.Fatalf("narrowed grant closed with %d", code) + } + viewer := h.mustDial(dialOpts{key: h.viewer}) + h.setRoster(h.devices(1, 2), protocol.ScopeLiveRead) // viewer paired again + if code := viewer.closed(time.Second); code != cloudwire.CloseChanged { + t.Fatalf("new epoch closed with %d", code) + } + h.mustDial(dialOpts{key: h.viewer}) +} + +func TestTurningFTWCloudOffClosesEverySession(t *testing.T) { + h := newHarness(t) + h.live(true) + p := h.mustDial(dialOpts{key: h.owner}) + h.box.Delete() + if code := p.closed(time.Second); code != cloudwire.CloseDenied { + t.Fatalf("purge closed with %d", code) + } +} + +// An owner's suspension only narrows: it closes the device's session and +// keeps it out, a viewer cannot suspend anyone, a device outside the roster +// cannot be named, and lifting it gives back what the roster allows. +func TestSuspensionOnlyNarrows(t *testing.T) { + h := newHarness(t) + h.live(true) + owner := h.mustDial(dialOpts{key: h.owner}) + viewer := h.mustDial(dialOpts{key: h.viewer}) + + viewer.send(cloudwire.MsgSuspend, cloudwire.Suspend{DeviceID: "owner"}) + owner.send(cloudwire.MsgSuspend, cloudwire.Suspend{DeviceID: "nobody"}) + next := h.live(false) + if got, code := owner.next(time.Second); code != 0 || !bytes.Equal(got, next) { + t.Fatalf("a viewer suspended the owner: code %d", code) + } + if h.reg.Suspended(h.home, "nobody", 0) { + t.Fatal("a device outside the roster was suspended") + } + + owner.send(cloudwire.MsgSuspend, cloudwire.Suspend{DeviceID: "viewer"}) + if code := viewer.closed(time.Second); code != cloudwire.CloseDenied { + t.Fatalf("suspended viewer closed with %d", code) + } + if _, code, _ := h.dial(dialOpts{key: h.viewer}); code != cloudwire.CloseDenied { + t.Fatalf("a suspended device reconnected: code %d", code) + } + + owner.send(cloudwire.MsgUnsuspend, cloudwire.Suspend{DeviceID: "viewer"}) + waitFor(t, "the suspension to lift", func() bool { return !h.reg.Suspended(h.home, "viewer", 1) }) + h.mustDial(dialOpts{key: h.viewer}) + + // Paired again on the box: a new epoch ends a suspension. + owner.send(cloudwire.MsgSuspend, cloudwire.Suspend{DeviceID: "viewer"}) + waitFor(t, "the suspension", func() bool { return h.reg.Suspended(h.home, "viewer", 1) }) + h.setRoster(h.devices(1, 2), protocol.ScopeLiveRead, protocol.ScopePlanRead) + h.mustDial(dialOpts{key: h.viewer}) +} + +// While a phone looks, the box is asked for one-second updates, and asked +// again before its watch lapses. When nobody looks, the asking stops. +func TestTheBoxIsWatchedWhileAPhoneLooks(t *testing.T) { + h := newHarness(t) + h.app.renewEvery = 40 * time.Millisecond + h.live(true) + + away := h.mustDial(dialOpts{key: h.viewer, hz: 0}) + time.Sleep(60 * time.Millisecond) + if n := h.watch.count(); n != 0 { + t.Fatalf("a phone that is not looking caused %d watches", n) + } + looking := h.mustDial(dialOpts{key: h.owner, hz: 1}) + waitFor(t, "renewals", func() bool { return h.watch.count() >= 3 }) + + // Two phones looking, one looks away: still watched. + away.send(cloudwire.MsgWatch, cloudwire.Watch{Hz: 1}) + looking.send(cloudwire.MsgWatch, cloudwire.Watch{Hz: 0}) + n := h.watch.count() + waitFor(t, "renewals with one phone left", func() bool { return h.watch.count() >= n+2 }) + + away.send(cloudwire.MsgWatch, cloudwire.Watch{Hz: 0}) + waitFor(t, "the watch to stop", func() bool { + h.app.mu.Lock() + defer h.app.mu.Unlock() + return len(h.app.looking) == 0 + }) + n = h.watch.count() + time.Sleep(120 * time.Millisecond) + if h.watch.count() != n { + t.Fatal("the service kept asking with nobody looking") + } + + // A closed session stops its share too. + away.send(cloudwire.MsgWatch, cloudwire.Watch{Hz: 1}) + waitFor(t, "a watch", func() bool { return h.watch.count() > n }) + away.conn.Close() + waitFor(t, "the watch to stop after close", func() bool { + h.app.mu.Lock() + defer h.app.mu.Unlock() + return len(h.app.looking) == 0 + }) +} + +func TestABoxThatComesBackIsAskedAtOnce(t *testing.T) { + h := newHarness(t) + h.app.renewEvery = time.Hour + h.live(true) + h.mustDial(dialOpts{key: h.owner, hz: 1}) + waitFor(t, "the first watch", func() bool { return h.watch.count() == 1 }) + h.app.BoxConnected(h.home) + h.app.BoxConnected("aaaaaaaaaaaaaaaaaaaaaaaaaa") + if n := h.watch.count(); n != 2 { + t.Fatalf("watches after the box came back: %d", n) + } +} + +func TestANewSessionFromTheSameDeviceReplacesTheOld(t *testing.T) { + h := newHarness(t) + h.live(true) + first := h.mustDial(dialOpts{key: h.owner}) + h.mustDial(dialOpts{key: h.owner}) + if code := first.closed(time.Second); code != cloudwire.CloseReplaced { + t.Fatalf("the older session closed with %d", code) + } +} + +func TestAHomeHoldsABoundedNumberOfSessions(t *testing.T) { + h := newHarness(t) + h.live(true) + var devices []cloudwire.Device + var keys []appwire.KeyPair + for i := 0; i <= store.MaxSessionsPerHome; i++ { + k, _ := appwire.GenerateKeyPair() + keys = append(keys, k) + devices = append(devices, cloudwire.Device{ID: string(rune('a' + i)), Key: k.Public(), Role: "viewer", + Scopes: []string{protocol.ScopeLiveRead}, Epoch: 1}) + } + h.setRoster(devices, protocol.ScopeLiveRead) + for _, k := range keys[:store.MaxSessionsPerHome] { + h.mustDial(dialOpts{key: k}) + } + if _, code, _ := h.dial(dialOpts{key: keys[store.MaxSessionsPerHome]}); code != cloudwire.CloseBusy { + t.Fatalf("one session over the limit: code %d", code) + } +} + +// A snapshot larger than one Noise message spills its newest envelopes into +// frames right after message 2, still in rising seq. +func TestALargeSnapshotSpillsIntoFrames(t *testing.T) { + h := newHarness(t) + h.live(true) + big := func(class string) []byte { + return h.put(class, func(hdr cloudwire.Header) any { + return struct { + cloudwire.Header + Pad []byte `cbor:"pad"` + }{hdr, bytes.Repeat([]byte{7}, 30_000)} + }, false) + } + plan := big(cloudwire.MsgPlan) + prices := big(cloudwire.MsgPrices) + lp := big(cloudwire.MsgLP) + p := h.mustDial(dialOpts{key: h.owner}) + if p.snap.More == 0 || len(p.snap.Envelopes)+p.snap.More != 4 { + t.Fatalf("snapshot held %d, more %d", len(p.snap.Envelopes), p.snap.More) + } + var rest [][]byte + for range p.snap.More { + env, code := p.next(time.Second) + if code != 0 { + t.Fatalf("closed with %d", code) + } + rest = append(rest, env) + } + var all [][]byte + for _, e := range p.snap.Envelopes { + all = append(all, e) + } + all = append(all, rest...) + for i, want := range [][]byte{plan, prices, lp} { + if !bytes.Equal(all[i+1], want) { + t.Fatalf("entry %d out of order", i+1) + } + } +} + +func TestAPhoneThatFloodsIsClosed(t *testing.T) { + h := newHarness(t) + h.live(true) + p := h.mustDial(dialOpts{key: h.owner}) + for range rateBurst + 5 { + env, _ := cloudwire.Unsigned("chatter", struct{}{}) + frame, _ := cloudwire.EncodeFrame(env) + msg, _ := p.tr.Encrypt(frame) + if p.conn.WriteMessage(websocket.BinaryMessage, msg) != nil { + break + } + } + if code := p.closed(2 * time.Second); code != cloudwire.CloseProtocol { + t.Fatalf("a flood closed with %d", code) + } +} + +func TestATamperedMessageClosesTheSession(t *testing.T) { + h := newHarness(t) + h.live(true) + p := h.mustDial(dialOpts{key: h.owner}) + env, _ := cloudwire.Unsigned(cloudwire.MsgWatch, cloudwire.Watch{Hz: 0}) + frame, _ := cloudwire.EncodeFrame(env) + msg, _ := p.tr.Encrypt(frame) + msg[len(msg)-1] ^= 1 + _ = p.conn.WriteMessage(websocket.BinaryMessage, msg) + if code := p.closed(2 * time.Second); code != cloudwire.CloseProtocol { + t.Fatalf("a tampered message closed with %d", code) + } +} + +func TestPlainHTTPAndABadHomeAreRefused(t *testing.T) { + h := newHarness(t) + resp, err := h.srv.Client().Get(h.srv.URL + "/v1/app?home=nothome") + if err != nil { + t.Fatal(err) + } + resp.Body.Close() + if resp.StatusCode != 400 { + t.Fatalf("a bad home: %d", resp.StatusCode) + } +} diff --git a/internal/appsession/fuzz_test.go b/internal/appsession/fuzz_test.go new file mode 100644 index 0000000..67cd253 --- /dev/null +++ b/internal/appsession/fuzz_test.go @@ -0,0 +1,78 @@ +package appsession + +import ( + "bytes" + "testing" + + "github.com/srcfl/ftw/go/protocol/appwire" + "github.com/srcfl/ftw/go/protocol/cloudwire" +) + +// Whatever a phone sends after the handshake, the service neither panics nor +// does more than the protocol allows: a phone can change whether it looks, +// and an owner can suspend a device in the roster. Nothing else changes. +func FuzzPhoneMessage(f *testing.F) { + h := newHarness(f) + h.live(true) + for _, seed := range [][]byte{ + mustFrame(cloudwire.MsgWatch, cloudwire.Watch{Hz: 1}), + mustFrame(cloudwire.MsgWatch, cloudwire.Watch{Hz: 0}), + mustFrame(cloudwire.MsgSuspend, cloudwire.Suspend{DeviceID: "viewer"}), + mustFrame(cloudwire.MsgUnsuspend, cloudwire.Suspend{DeviceID: "viewer"}), + mustFrame("token.get", struct{}{}), + {1, 1, 0, 0, 0, 1, 0xa0}, + {1, 0, 0, 0, 0, 0}, + {}, + } { + f.Add(seed) + } + f.Fuzz(func(t *testing.T, frame []byte) { + for _, key := range []appwire.KeyPair{h.viewer, h.owner} { + dev, err := h.store.For(storePrincipal(h, key)) + if err != nil { + t.Fatal(err) + } + c := &phoneConn{s: h.app, dev: dev, home: h.home, tokens: rateBurst, refilled: h.app.now()} + code, err := c.apply(frame) + if err != nil && code != cloudwire.CloseProtocol && code != 0 { + t.Fatalf("closed with %d", code) + } + c.stopLooking() + // Only the owner may suspend, and only the viewer exists to be + // suspended besides the owner itself. + if bytes.Equal(key.Public(), h.viewer.Public()) && h.reg.Suspended(h.home, "owner", 1) { + t.Fatal("a viewer suspended the owner") + } + _ = h.reg.Unsuspend(h.home, "viewer") + _ = h.reg.Unsuspend(h.home, "owner") + } + h.app.mu.Lock() + left := len(h.app.looking) + h.app.mu.Unlock() + if left != 0 { + t.Fatal("a session that ended is still counted as looking") + } + }) +} + +// Message 1's payload: anything at all is refused or read, never a panic. +func FuzzNow(f *testing.F) { + now, _ := cloudwire.Unsigned(cloudwire.MsgNow, cloudwire.Now{Hz: 1}) + f.Add(now) + f.Add([]byte{0xa0}) + f.Fuzz(func(t *testing.T, payload []byte) { + _, _ = parseNow(payload) + }) +} + +func mustFrame(t string, body any) []byte { + env, err := cloudwire.Unsigned(t, body) + if err != nil { + panic(err) + } + frame, err := cloudwire.EncodeFrame(env) + if err != nil { + panic(err) + } + return frame +} diff --git a/internal/appsession/harness_test.go b/internal/appsession/harness_test.go new file mode 100644 index 0000000..7386374 --- /dev/null +++ b/internal/appsession/harness_test.go @@ -0,0 +1,310 @@ +package appsession + +import ( + "bytes" + "crypto/ecdsa" + "crypto/elliptic" + "crypto/rand" + "crypto/tls" + "errors" + "io" + "log/slog" + "net/http" + "net/http/httptest" + "path/filepath" + "strings" + "sync" + "testing" + "time" + + "github.com/gorilla/websocket" + "github.com/srcfl/ftw-cloud/internal/registry" + "github.com/srcfl/ftw-cloud/internal/store" + "github.com/srcfl/ftw/go/protocol" + "github.com/srcfl/ftw/go/protocol/appwire" + "github.com/srcfl/ftw/go/protocol/cloudwire" +) + +// A home with a box key, a roster with an owner and a viewer, and a TLS +// server answering /v1/app the way ftwcloudd does. +type harness struct { + t testing.TB + store *store.Store + reg *registry.Registry + app *Server + srv *httptest.Server + watch *watchCount + noise appwire.KeyPair + boxKey *ecdsa.PrivateKey + home string + box *store.Handle + owner appwire.KeyPair + viewer appwire.KeyPair + seq uint64 + rosterN uint64 + boot []byte +} + +type watchCount struct { + mu sync.Mutex + homes []string +} + +func (w *watchCount) Watch(home string) { + w.mu.Lock() + defer w.mu.Unlock() + w.homes = append(w.homes, home) +} + +func (w *watchCount) count() int { + w.mu.Lock() + defer w.mu.Unlock() + return len(w.homes) +} + +func newHarness(t testing.TB) *harness { + t.Helper() + reg, err := registry.Open(filepath.Join(t.TempDir(), "registry.db")) + if err != nil { + t.Fatal(err) + } + t.Cleanup(func() { _ = reg.Close() }) + st := store.New(nil) + st.UseSuspensions(reg.Suspended) + noise, _ := appwire.GenerateKeyPair() + w := &watchCount{} + app := New(st, reg, w, noise, slog.New(slog.NewTextHandler(io.Discard, nil))) + srv := httptest.NewUnstartedServer(app) + srv.TLS = &tls.Config{MinVersion: tls.VersionTLS13} + srv.StartTLS() + t.Cleanup(srv.Close) + + boxKey, _ := ecdsa.GenerateKey(elliptic.P256(), rand.Reader) + pk, _ := cloudwire.PublicKeyBytesOf(&boxKey.PublicKey) + home := cloudwire.HomeID(pk) + box, err := st.For(store.Principal{Kind: store.Box, Home: home}) + if err != nil { + t.Fatal(err) + } + box.Connected() + if err := reg.Claim(home, pk, time.Now()); err != nil { + t.Fatal(err) + } + owner, _ := appwire.GenerateKeyPair() + viewer, _ := appwire.GenerateKeyPair() + h := &harness{t: t, store: st, reg: reg, app: app, srv: srv, watch: w, noise: noise, boxKey: boxKey, + home: home, box: box, owner: owner, viewer: viewer, boot: bytes.Repeat([]byte{0xb0}, cloudwire.BootBytes)} + h.setRoster(h.devices(1, 1), protocol.ScopeLiveRead, protocol.ScopePlanRead) + return h +} + +func (h *harness) devices(ownerEpoch, viewerEpoch uint64) []cloudwire.Device { + return []cloudwire.Device{ + {ID: "owner", Key: h.owner.Public(), Role: "owner", Scopes: protocol.CloudScopes, Epoch: ownerEpoch}, + {ID: "viewer", Key: h.viewer.Public(), Role: "viewer", Scopes: []string{protocol.ScopeLiveRead}, Epoch: viewerEpoch}, + } +} + +func (h *harness) setRoster(devices []cloudwire.Device, grant ...string) { + h.rosterN++ + r := &cloudwire.Roster{RosterSeq: h.rosterN, TTLS: 86_400, Devices: devices, + Grant: cloudwire.Grant{ID: make([]byte, 16), Scopes: grant, Purposes: []string{protocol.CloudPurposeServe}, Controller: "sourceful"}} + if !h.box.SetRoster(r) { + h.t.Fatal("roster refused") + } + epochs := map[string]uint64{} + for _, d := range devices { + epochs[d.ID] = d.Epoch + } + if err := h.reg.PruneSuspensions(h.home, epochs); err != nil { + h.t.Fatal(err) + } +} + +// put signs a body as the box and holds it, as the uplink does. +func (h *harness) put(t string, body func(cloudwire.Header) any, table bool) []byte { + h.t.Helper() + h.seq++ + hdr := cloudwire.Header{Seq: h.seq, Boot: h.boot, UptimeMs: int64(60_000 + h.seq*100)} + env, err := cloudwire.Seal(h.boxKey, h.home, t, body(hdr)) + if err != nil { + h.t.Fatal(err) + } + raw, err := env.Encode() + if err != nil { + h.t.Fatal(err) + } + if err := h.box.Put(t, raw, h.seq, table); err != nil { + h.t.Fatal(err) + } + return raw +} + +func (h *harness) live(table bool) []byte { + return h.put(cloudwire.MsgLive, func(hdr cloudwire.Header) any { + l := cloudwire.Live{Header: hdr, Fields: map[string]int64{"2": 1800}, SrcRev: 1, Ages: map[string]int64{"m": 400}} + if table { + l.Sources = map[string]cloudwire.Source{"m": {Kind: "driver", State: protocol.SourceLive, StaleAfterMs: 10_000}} + l.FieldSources = map[string][]string{"2": {"m"}} + } + return l + }, table) +} + +func (h *harness) plan() []byte { + return h.put(cloudwire.MsgPlan, func(hdr cloudwire.Header) any { + return struct { + cloudwire.Header + DecisionID string `cbor:"decisionId"` + }{hdr, "d1"} + }, false) +} + +func (h *harness) hb() []byte { + return h.put(cloudwire.MsgHB, func(hdr cloudwire.Header) any { + return cloudwire.HB{Header: hdr, Clock: cloudwire.ClockNTP, SrcRev: 1, LiveSeq: h.seq - 1} + }, false) +} + +// phone is a Go client of /v1/app. +type phone struct { + t testing.TB + conn *websocket.Conn + tr *appwire.Transport + snap cloudwire.Snapshot +} + +type dialOpts struct { + key appwire.KeyPair + service []byte // the service key the phone pins; nil uses the harness's + home string // query; "" uses the harness's + prologue string // home in the prologue; "" uses query + hz int +} + +// dial runs the handshake. It returns the close code when the service refuses +// before message 2. +func (h *harness) dial(o dialOpts) (*phone, int, error) { + h.t.Helper() + if o.service == nil { + o.service = h.noise.Public() + } + if o.home == "" { + o.home = h.home + } + if o.prologue == "" { + o.prologue = o.home + } + d := websocket.Dialer{TLSClientConfig: &tls.Config{InsecureSkipVerify: true, MinVersion: tls.VersionTLS13}} //nolint:gosec // the test's own server + url := strings.Replace(h.srv.URL, "https://", "wss://", 1) + "/v1/app?home=" + o.home + conn, _, err := d.Dial(url, nil) + if err != nil { + return nil, 0, err + } + init, err := appwire.NewInitiator(appwire.InitiatorOptions{StaticKey: o.key, RemoteStatic: o.service, Prologue: cloudwire.AppPrologue(o.prologue)}) + if err != nil { + h.t.Fatal(err) + } + now, _ := cloudwire.Unsigned(cloudwire.MsgNow, cloudwire.Now{Hz: o.hz}) + msg1, _ := init.WriteMessage(now) + if err := conn.WriteMessage(websocket.BinaryMessage, msg1); err != nil { + return nil, 0, err + } + _ = conn.SetReadDeadline(time.Now().Add(5 * time.Second)) + _, msg2, err := conn.ReadMessage() + if err != nil { + var ce *websocket.CloseError + if errors.As(err, &ce) { + conn.Close() + return nil, ce.Code, nil + } + return nil, 0, err + } + payload, err := init.ReadMessage(msg2) + if err != nil { + h.t.Fatalf("message 2: %v", err) + } + snap, err := cloudwire.ParseSnapshot(payload) + if err != nil { + h.t.Fatalf("snapshot: %v", err) + } + keys, _ := init.Split() + p := &phone{t: h.t, conn: conn, tr: appwire.NewTransport(keys), snap: snap} + h.t.Cleanup(func() { conn.Close() }) + return p, 0, nil +} + +func (h *harness) mustDial(o dialOpts) *phone { + h.t.Helper() + p, code, err := h.dial(o) + if err != nil || code != 0 { + h.t.Fatalf("dial: code %d, %v", code, err) + } + return p +} + +// next reads one forwarded envelope, or the close code. +func (p *phone) next(within time.Duration) ([]byte, int) { + p.t.Helper() + _ = p.conn.SetReadDeadline(time.Now().Add(within)) + _, msg, err := p.conn.ReadMessage() + if err != nil { + var ce *websocket.CloseError + if errors.As(err, &ce) { + return nil, ce.Code + } + p.t.Fatalf("read: %v", err) + } + frame, err := p.tr.Decrypt(msg) + if err != nil { + p.t.Fatalf("decrypt: %v", err) + } + env, err := cloudwire.DecodeFrame(frame) + if err != nil { + p.t.Fatalf("frame: %v", err) + } + return env, 0 +} + +// closed waits for the service to end the session and returns its code. +func (p *phone) closed(within time.Duration) int { + p.t.Helper() + deadline := time.Now().Add(within) + for time.Now().Before(deadline) { + if _, code := p.next(time.Until(deadline)); code != 0 { + return code + } + } + p.t.Fatal("the session stayed open") + return 0 +} + +func (p *phone) send(t string, body any) { + p.t.Helper() + env, _ := cloudwire.Unsigned(t, body) + frame, _ := cloudwire.EncodeFrame(env) + msg, err := p.tr.Encrypt(frame) + if err != nil { + p.t.Fatal(err) + } + if err := p.conn.WriteMessage(websocket.BinaryMessage, msg); err != nil { + p.t.Fatal(err) + } +} + +func waitFor(t *testing.T, what string, ok func() bool) { + t.Helper() + deadline := time.Now().Add(3 * time.Second) + for !ok() { + if time.Now().After(deadline) { + t.Fatalf("waited 3 s for %s", what) + } + time.Sleep(5 * time.Millisecond) + } +} + +var _ http.Handler = (*Server)(nil) + +func storePrincipal(h *harness, key appwire.KeyPair) store.Principal { + return store.Principal{Kind: store.Device, Home: h.home, DeviceKey: key.Public()} +} diff --git a/internal/appsession/perf_test.go b/internal/appsession/perf_test.go new file mode 100644 index 0000000..e203758 --- /dev/null +++ b/internal/appsession/perf_test.go @@ -0,0 +1,111 @@ +package appsession + +import ( + "slices" + "testing" + "time" + + "github.com/srcfl/ftw/go/protocol" + "github.com/srcfl/ftw/go/protocol/appwire" + "github.com/srcfl/ftw/go/protocol/cloudwire" +) + +// A home the size of a real one: the source table, a live, an lp with one +// car, a day and a half of quarter-hour prices and an hb. +func (h *harness) fillLikeAHouse() { + srcs := map[string]cloudwire.Source{} + ages := map[string]int64{} + for _, id := range []string{"ACfHt4DWL7g", "f-oHMDqV-mA", "DUYWWQmn-Xg", "j3HZORNCWHs"} { + srcs[id] = cloudwire.Source{Kind: "driver", State: protocol.SourceLive, LastOkMs: 59_000, StaleAfterMs: 10_000} + ages[id] = 900 + } + fields := map[string]int64{"1": 0, "2": 1840, "3": -2310, "4": 950, "5": 612, "6": 1100, "10": 2100} + h.put(cloudwire.MsgLive, func(hdr cloudwire.Header) any { + return cloudwire.Live{Header: hdr, Fields: fields, Sources: srcs, SrcRev: 1, Ages: ages, + FieldSources: map[string][]string{"2": {"ACfHt4DWL7g"}, "3": {"f-oHMDqV-mA"}, "10": {"DUYWWQmn-Xg"}}} + }, true) + h.put(cloudwire.MsgLive, func(hdr cloudwire.Header) any { + return cloudwire.Live{Header: hdr, Fields: fields, SrcRev: 1, Ages: ages} + }, false) + v := func(x int64) *int64 { return &x } + h.put(cloudwire.MsgLP, func(hdr cloudwire.Header) any { + return cloudwire.LP{Header: hdr, ControlRev: 812, Loadpoints: []cloudwire.Loadpoint{{ + ID: "j3HZORNCWHs", Src: "DUYWWQmn-Xg", Plugged: true, SocPermille: v(430), SocSource: "inferred", + TargetPermille: v(800), ReadyByMs: v(1_791_867_600_000), PowerW: 2100, SessionWh: 3400, Session: "s1", + CommandedW: v(2070), CommandedReason: "plan", NextStartMs: v(1_791_850_000_000)}}, + Feedback: []cloudwire.Feedback{{Kind: "ev", Src: "DUYWWQmn-Xg", Status: "following", Reason: "plan", + Severity: "ok", Evidence: "measured", Fresh: true, ActualW: v(2100), AgeMs: v(900)}}} + }, false) + var spot, total []*int64 + for i := range 144 { + spot, total = append(spot, v(int64(40+i%30))), append(total, v(int64(110+i%30))) + } + h.put(cloudwire.MsgPrices, func(hdr cloudwire.Header) any { + return cloudwire.Prices{Header: hdr, Zone: "SE3", Currency: "SEK", T0Ms: 1_791_820_800_000, SlotMs: 900_000, + SpotMinor: spot, TotalMinor: total} + }, false) + h.put(cloudwire.MsgHB, func(hdr cloudwire.Header) any { + return cloudwire.HB{Header: hdr, Clock: cloudwire.ClockNTP, SrcRev: 1, LiveSeq: 2, LpSeq: 3, Ages: ages} + }, false) +} + +// The doc's speed claim for the second layer: the cloud's copy, from memory, +// in well under 100 ms. Here the whole of it on loopback: TCP, TLS 1.3, the +// WebSocket upgrade, Noise IK and the snapshot, twenty times. +func TestTheSnapshotArrivesFast(t *testing.T) { + h := newHarness(t) + h.fillLikeAHouse() + var took []time.Duration + var size int + for range 20 { + start := time.Now() + p := h.mustDial(dialOpts{key: h.owner, hz: 1}) + took = append(took, time.Since(start)) + size = len(p.snap.Envelopes) + p.conn.Close() + } + slices.Sort(took) + p50, p95 := took[len(took)/2], took[len(took)*95/100] + t.Logf("handshake and snapshot of %d envelopes on loopback: p50 %v, p95 %v, max %v", size, p50, p95, took[len(took)-1]) + if p95 > 100*time.Millisecond { + t.Fatalf("p95 %v is over the 100 ms the snapshot has", p95) + } +} + +// What the service spends per phone between message 1 and message 2: the +// store's snapshot, the payload and the Noise work, with no network. +func BenchmarkSnapshot(b *testing.B) { + h := newHarness(b) + h.fillLikeAHouse() + now, _ := cloudwire.Unsigned(cloudwire.MsgNow, cloudwire.Now{Hz: 1}) + prologue := cloudwire.AppPrologue(h.home) + b.ReportAllocs() + var bytesOut int + for b.Loop() { + init, _ := appwire.NewInitiator(appwire.InitiatorOptions{StaticKey: h.owner, RemoteStatic: h.noise.Public(), Prologue: prologue}) + msg1, _ := init.WriteMessage(now) + resp, _ := appwire.NewResponder(appwire.HandshakeOptions{StaticKey: h.noise, Prologue: prologue}) + if _, err := resp.ReadMessage(msg1); err != nil { + b.Fatal(err) + } + dev, err := h.store.For(storePrincipal(h, h.owner)) + if err != nil { + b.Fatal(err) + } + snap, sess, err := dev.Open() + if err != nil { + b.Fatal(err) + } + payload, _, err := snapshotPayload(snap) + if err != nil { + b.Fatal(err) + } + msg2, err := resp.WriteMessage(payload) + if err != nil { + b.Fatal(err) + } + sess.Close() + bytesOut = len(msg2) + } + b.ReportMetric(float64(bytesOut), "msg2-bytes") +} diff --git a/internal/appsession/push_test.go b/internal/appsession/push_test.go new file mode 100644 index 0000000..55c3aeb --- /dev/null +++ b/internal/appsession/push_test.go @@ -0,0 +1,94 @@ +package appsession + +import ( + "bytes" + "sync/atomic" + "testing" + "time" + + "github.com/srcfl/ftw-cloud/internal/push" + "github.com/srcfl/ftw/go/protocol" + "github.com/srcfl/ftw/go/protocol/cloudwire" +) + +// countedPush is the push service, counting the calls the sessions make. +type countedPush struct { + *push.Service + calls atomic.Int32 +} + +func (p *countedPush) Register(home string, dev cloudwire.Device, scopes []string, r cloudwire.PushRegister) error { + defer p.calls.Add(1) + return p.Service.Register(home, dev, scopes, r) +} + +func (p *countedPush) Unregister(home string, dev cloudwire.Device, r cloudwire.PushUnregister) error { + defer p.calls.Add(1) + return p.Service.Unregister(home, dev, r) +} + +// A phone registers its APNs token over its own session. The roster entry +// and scope come from the session, never from the message: a viewer's +// registration registers nothing and its session goes on. +func TestAPhoneRegistersItsTokenOverItsSession(t *testing.T) { + h := newHarness(t) + h.setRoster(h.devices(1, 1), protocol.ScopeLiveRead, protocol.ScopeEventsRead) + p := &countedPush{Service: push.New(h.reg, nil, nil)} + h.app.Push = p + h.live(true) + owner := h.mustDial(dialOpts{key: h.owner}) + viewer := h.mustDial(dialOpts{key: h.viewer}) + register := func(token byte) cloudwire.PushRegister { + return cloudwire.PushRegister{APNsToken: bytes.Repeat([]byte{token}, 32), + Environment: cloudwire.PushSandbox, Kind: cloudwire.PushKindAlert} + } + regs := func() []string { + got, err := h.reg.PushRegs(h.home, cloudwire.PushKindAlert) + if err != nil { + t.Fatal(err) + } + var out []string + for _, r := range got { + out = append(out, r.DeviceID) + } + return out + } + + viewer.send(cloudwire.MsgPushRegister, register(0xa1)) + owner.send(cloudwire.MsgPushRegister, register(0xa2)) + waitFor(t, "both registrations", func() bool { return p.calls.Load() == 2 }) + if got := regs(); len(got) != 1 || got[0] != "owner" { + t.Fatalf("registered %v, want the owner alone", got) + } + next := h.live(false) + for name, ph := range map[string]*phone{"viewer": viewer, "owner": owner} { + if got, code := ph.next(time.Second); code != 0 || !bytes.Equal(got, next) { + t.Fatalf("the %s's session did not go on: code %d", name, code) + } + } + + owner.send(cloudwire.MsgPushUnregister, cloudwire.PushUnregister{APNsToken: register(0xa2).APNsToken}) + waitFor(t, "the unregistration", func() bool { return p.calls.Load() == 3 }) + if got := regs(); len(got) != 0 { + t.Fatalf("still registered: %v", got) + } + + // A registration the service cannot read is a protocol error. + owner.send(cloudwire.MsgPushRegister, "not a registration") + if code := owner.closed(2 * time.Second); code != cloudwire.CloseProtocol { + t.Fatalf("an unreadable registration closed with %d", code) + } +} + +// Without a push service, a registration is ignored and the session goes on. +func TestWithoutPushARegistrationIsIgnored(t *testing.T) { + h := newHarness(t) + h.live(true) + owner := h.mustDial(dialOpts{key: h.owner}) + owner.send(cloudwire.MsgPushRegister, cloudwire.PushRegister{APNsToken: []byte{1}, + Environment: cloudwire.PushSandbox, Kind: cloudwire.PushKindAlert}) + next := h.live(false) + if got, code := owner.next(time.Second); code != 0 || !bytes.Equal(got, next) { + t.Fatalf("the session did not go on: code %d", code) + } +} diff --git a/internal/push/push.go b/internal/push/push.go new file mode 100644 index 0000000..772f3ed --- /dev/null +++ b/internal/push/push.go @@ -0,0 +1,322 @@ +// Package push tells a home's phones what its box said while they were not +// looking, through Apple. docs/protocol-cloud.md in srcfl/ftw, "Push", is the +// wire format; go/protocol/cloudwire builds the payload. +// +// The service chooses nothing a phone shows. An alert carries the box's +// signed event exactly as it arrived; the phone checks the signature, writes +// its own words and picks the buttons from the verified kind. Every alert +// has the same aps, priority and push type, and no collapse id, so the +// request says nothing a kind would. The one alert the service writes is +// box.unreachable, after the box has been away for ten minutes without +// saying it was restarting on purpose. It carries no arguments and no +// buttons. +// +// Tokens are bound to roster entries. Only a device whose role and the +// owner's grant both allow ftw.events.read may register, and a roster that +// drops the device, moves its epoch or narrows its scope drops its tokens. +// A device an owner has suspended gets no alerts until the suspension ends. +// Nothing here logs a token or a home id. +package push + +import ( + "context" + "errors" + "log/slog" + "slices" + "sync" + "time" + + "github.com/srcfl/ftw-cloud/internal/apns" + "github.com/srcfl/ftw-cloud/internal/registry" + "github.com/srcfl/ftw/go/protocol" + "github.com/srcfl/ftw/go/protocol/cloudwire" +) + +const ( + // MaxAlertsPerHour bounds what one home can make its phones buzz. A box + // raises a few notifications a day; past this, something is wrong and + // the rest are dropped. + MaxAlertsPerHour = 20 + // eventExpiry is how long Apple may keep trying to deliver an event. + // Later than that the phone would show only the generic line. + eventExpiry = cloudwire.AlertMaxAgeMs * time.Millisecond + // unreachableExpiry is the same for box.unreachable, which stays true + // while the box is away. + unreachableExpiry = time.Hour + // queueSize bounds the alerts waiting to be sent. + queueSize = 256 + // expireEvery is how often registrations of long-silent homes go. + expireEvery = time.Hour +) + +// Sender is Apple. apns.Client is one. +type Sender interface { + Send(ctx context.Context, n apns.Notification) (apns.Result, error) +} + +// ErrNotForThisDevice refuses a registration the device may not make: a +// viewer, a grant without events, or a kind that does not exist yet. The +// app session goes on. +var ErrNotForThisDevice = errors.New("push: not a registration this device may make") + +// Service keeps registrations and sends alerts. +type Service struct { + reg *registry.Registry + sender Sender + log *slog.Logger + now func() time.Time + // after runs f after d and returns a function that stops it. Tests + // replace it. + after func(d time.Duration, f func()) (stop func() bool) + + work chan func(context.Context) + + mu sync.Mutex + homes map[string]*homeState +} + +// homeState is one home's box connection, as the dead man's switch sees it, +// and its recent alerts. +type homeState struct { + online bool + // gen moves on every connect and disconnect, so a timer set for one + // outage cannot fire into the next. + gen uint64 + stop func() bool + told bool + sent []time.Time +} + +// New builds the service. A nil sender keeps registrations and sends +// nothing, for a service without an APNs key. +func New(reg *registry.Registry, sender Sender, log *slog.Logger) *Service { + if log == nil { + log = slog.Default() + } + return &Service{ + reg: reg, sender: sender, log: log, now: time.Now, + after: func(d time.Duration, f func()) func() bool { return time.AfterFunc(d, f).Stop }, + work: make(chan func(context.Context), queueSize), + homes: map[string]*homeState{}, + } +} + +// Run sends queued alerts and drops long-silent registrations until ctx +// ends. +func (s *Service) Run(ctx context.Context) { + t := time.NewTicker(expireEvery) + defer t.Stop() + for { + select { + case <-ctx.Done(): + return + case job := <-s.work: + job(ctx) + case <-t.C: + if n, err := s.reg.PushExpire(s.now()); err != nil { + s.log.Warn("push upkeep failed", "err", err) + } else if n > 0 { + s.log.Info("push registrations expired with their silent homes", "count", n) + } + } + } +} + +// enqueue hands a job to Run without ever blocking the caller, which is a +// box's uplink. A full queue drops the job. +func (s *Service) enqueue(job func(context.Context)) { + select { + case s.work <- job: + default: + s.log.Warn("push queue full; an alert was dropped") + } +} + +// -------------------------------------------------------------------------- +// Registrations, from a phone's app session +// -------------------------------------------------------------------------- + +// Register keeps a device's APNs token. dev is the roster entry the phone's +// Noise key matched and scopes its effective scope, both from store.For. +func (s *Service) Register(home string, dev cloudwire.Device, scopes []string, r cloudwire.PushRegister) error { + if r.Kind != cloudwire.PushKindAlert || !slices.Contains(scopes, protocol.ScopeEventsRead) || + len(r.APNsToken) == 0 || len(r.APNsToken) > cloudwire.APNsTokenMaxBytes || + (r.Environment != cloudwire.PushSandbox && r.Environment != cloudwire.PushProduction) { + return ErrNotForThisDevice + } + return s.reg.PushRegister(registry.PushReg{Home: home, DeviceID: dev.ID, Epoch: dev.Epoch, + Token: r.APNsToken, Environment: r.Environment, Kind: r.Kind, At: s.now()}) +} + +// Unregister forgets one of the device's tokens. +func (s *Service) Unregister(home string, dev cloudwire.Device, r cloudwire.PushUnregister) error { + return s.reg.PushUnregister(home, dev.ID, r.APNsToken) +} + +// -------------------------------------------------------------------------- +// From a box's uplink. None of these blocks. +// -------------------------------------------------------------------------- + +// Roster drops the tokens of devices the box's roster no longer lets read +// events: gone, paired again under a new epoch, or narrowed. +func (s *Service) Roster(home string, r *cloudwire.Roster) { + if r == nil { + return + } + allowed := map[string]uint64{} + for _, d := range r.Devices { + if slices.Contains(d.Scopes, protocol.ScopeEventsRead) && slices.Contains(r.Grant.Scopes, protocol.ScopeEventsRead) { + allowed[d.ID] = d.Epoch + } + } + if err := s.reg.PushPrune(home, func(id string, epoch uint64) bool { + e, ok := allowed[id] + return ok && e == epoch + }); err != nil { + s.log.Warn("could not drop the tokens a roster dropped", "err", err) + } +} + +// Event sends a box-signed event envelope, as it arrived, to the home's +// registered phones. +func (s *Service) Event(home string, envelope []byte) { + payload, err := cloudwire.AlertPayload(cloudwire.Alert{Home: home, Event: envelope}) + if err != nil { + return + } + s.enqueue(func(ctx context.Context) { s.send(ctx, home, payload, eventExpiry) }) +} + +// BoxConnected stops the dead man's switch for a home whose box is back. +func (s *Service) BoxConnected(home string) { + s.mu.Lock() + defer s.mu.Unlock() + st := s.state(home) + st.gen++ + st.online, st.told = true, false + if st.stop != nil { + st.stop() + st.stop = nil + } +} + +// BoxGone starts the dead man's switch. why is the box's bye reason, empty +// when the socket just dropped. A bye for an update holds box.unreachable +// back until the box has been away for cloudwire.UpdateGraceMs; a revoke +// ends it, since the home is gone. +func (s *Service) BoxGone(home, why string) { + s.mu.Lock() + defer s.mu.Unlock() + if why == cloudwire.ByeRevoked { + if st := s.homes[home]; st != nil && st.stop != nil { + st.stop() + } + delete(s.homes, home) + return + } + st := s.state(home) + st.gen++ + st.online = false + if st.stop != nil { + st.stop() + } + wait := time.Duration(cloudwire.UnreachableAfterMs) * time.Millisecond + if why == cloudwire.ByeUpdate { + wait = time.Duration(cloudwire.UpdateGraceMs) * time.Millisecond + } + s.log.Info("a box went away", "bye", why, "out_of_reach_after", wait) + gen := st.gen + st.stop = s.after(wait, func() { s.unreachable(home, gen) }) +} + +func (s *Service) unreachable(home string, gen uint64) { + s.mu.Lock() + st := s.homes[home] + if st == nil || st.gen != gen || st.online || st.told { + s.mu.Unlock() + return + } + st.told, st.stop = true, nil + s.mu.Unlock() + s.log.Info("a box is out of reach; telling its phones") + payload, err := cloudwire.AlertPayload(cloudwire.Alert{Home: home, Kind: protocol.PushBoxUnreachable}) + if err != nil { + return + } + s.enqueue(func(ctx context.Context) { s.send(ctx, home, payload, unreachableExpiry) }) +} + +func (s *Service) state(home string) *homeState { + st := s.homes[home] + if st == nil { + st = &homeState{} + s.homes[home] = st + } + return st +} + +// allow counts one alert against the home's hourly cap. +func (s *Service) allow(home string) bool { + s.mu.Lock() + defer s.mu.Unlock() + st := s.state(home) + now := s.now() + keep := st.sent[:0] + for _, at := range st.sent { + if now.Sub(at) < time.Hour { + keep = append(keep, at) + } + } + st.sent = keep + if len(st.sent) >= MaxAlertsPerHour { + return false + } + st.sent = append(st.sent, now) + return true +} + +// send posts one payload to every phone of the home registered for alerts, +// and forgets the tokens Apple says are gone. +func (s *Service) send(ctx context.Context, home string, payload []byte, expiry time.Duration) { + if s.sender == nil { + return + } + regs, err := s.reg.PushRegs(home, cloudwire.PushKindAlert) + if err != nil { + s.log.Warn("could not read push registrations", "err", err) + return + } + // A device an owner has suspended keeps its tokens, so lifting the + // suspension brings its alerts back, but gets nothing meanwhile. + regs = slices.DeleteFunc(regs, func(r registry.PushReg) bool { + return s.reg.Suspended(home, r.DeviceID, r.Epoch) + }) + if len(regs) == 0 { + return + } + if !s.allow(home) { + s.log.Warn("a home passed its alert cap; alert dropped", "cap_per_hour", MaxAlertsPerHour) + return + } + var sent, gone, failed int + exp := s.now().Add(expiry) + for _, r := range regs { + res, err := s.sender.Send(ctx, apns.Notification{Environment: r.Environment, DeviceToken: r.Token, + Payload: payload, Expiration: exp}) + switch { + case err != nil: + failed++ + case res.OK(): + sent++ + case res.Gone: + gone++ + if err := s.reg.PushGone(home, r.Token, res.GoneSince); err != nil { + s.log.Warn("could not forget a gone token", "err", err) + } + default: + failed++ + s.log.Warn("APNs refused an alert", "status", res.Status, "reason", res.Reason) + } + } + s.log.Info("alert sent", "phones", sent, "gone", gone, "failed", failed) +} diff --git a/internal/push/push_test.go b/internal/push/push_test.go new file mode 100644 index 0000000..787ceab --- /dev/null +++ b/internal/push/push_test.go @@ -0,0 +1,402 @@ +package push + +import ( + "bytes" + "context" + "encoding/json" + "io" + "log/slog" + "path/filepath" + "sync" + "testing" + "time" + + "github.com/srcfl/ftw-cloud/internal/apns" + "github.com/srcfl/ftw-cloud/internal/registry" + "github.com/srcfl/ftw/go/protocol" + "github.com/srcfl/ftw/go/protocol/cloudwire" +) + +type fakeApple struct { + mu sync.Mutex + sent []apns.Notification + gone map[string]bool +} + +func (f *fakeApple) Send(_ context.Context, n apns.Notification) (apns.Result, error) { + f.mu.Lock() + defer f.mu.Unlock() + f.sent = append(f.sent, n) + if f.gone[string(n.DeviceToken)] { + return apns.Result{Status: 410, Reason: "Unregistered", Gone: true}, nil + } + return apns.Result{Status: 200}, nil +} + +func (f *fakeApple) notifications() []apns.Notification { + f.mu.Lock() + defer f.mu.Unlock() + return append([]apns.Notification(nil), f.sent...) +} + +// timers is a clock the test moves by hand. +type timers struct { + mu sync.Mutex + now time.Time + due []*timer +} + +type timer struct { + at time.Time + f func() + stopped bool +} + +func (c *timers) after(d time.Duration, f func()) func() bool { + c.mu.Lock() + defer c.mu.Unlock() + t := &timer{at: c.now.Add(d), f: f} + c.due = append(c.due, t) + return func() bool { + c.mu.Lock() + defer c.mu.Unlock() + was := !t.stopped + t.stopped = true + return was + } +} + +func (c *timers) advance(d time.Duration) { + c.mu.Lock() + c.now = c.now.Add(d) + var fire []func() + for _, t := range c.due { + if !t.stopped && !t.at.After(c.now) { + t.stopped = true + fire = append(fire, t.f) + } + } + c.mu.Unlock() + for _, f := range fire { + f() + } +} + +func (c *timers) time() time.Time { + c.mu.Lock() + defer c.mu.Unlock() + return c.now +} + +type rig struct { + svc *Service + reg *registry.Registry + apple *fakeApple + clock *timers + home string + owner cloudwire.Device + viewer cloudwire.Device + all []string +} + +func newRig(t *testing.T) *rig { + t.Helper() + reg, err := registry.Open(filepath.Join(t.TempDir(), "registry.db")) + if err != nil { + t.Fatal(err) + } + t.Cleanup(func() { _ = reg.Close() }) + r := &rig{reg: reg, apple: &fakeApple{gone: map[string]bool{}}, + clock: &timers{now: time.Unix(1_791_820_800, 0)}, home: "fjzyr67buzpyncqmjliwgezvde"} + r.svc = New(reg, r.apple, slog.New(slog.NewTextHandler(io.Discard, nil))) + r.svc.now, r.svc.after = r.clock.time, r.clock.after + if err := reg.Claim(r.home, []byte("pk"), r.clock.time()); err != nil { + t.Fatal(err) + } + r.all = []string{protocol.ScopeLiveRead, protocol.ScopeEventsRead} + r.owner = cloudwire.Device{ID: "owner-phone", Key: bytes.Repeat([]byte{5}, 32), Role: "owner", Scopes: r.all, Epoch: 1} + r.viewer = cloudwire.Device{ID: "viewer-phone", Key: bytes.Repeat([]byte{6}, 32), Role: "viewer", + Scopes: []string{protocol.ScopeLiveRead}, Epoch: 1} + return r +} + +// drain runs what the uplink queued, as Run would. +func (r *rig) drain() { + for { + select { + case job := <-r.svc.work: + job(context.Background()) + default: + return + } + } +} + +func (r *rig) register(t *testing.T, dev cloudwire.Device, scopes []string, token string) error { + t.Helper() + return r.svc.Register(r.home, dev, scopes, cloudwire.PushRegister{ + APNsToken: []byte(token), Environment: cloudwire.PushSandbox, Kind: cloudwire.PushKindAlert}) +} + +func TestAnEventGoesToRegisteredPhonesExactlyAsTheBoxSignedIt(t *testing.T) { + r := newRig(t) + if err := r.register(t, r.owner, r.all, "owner-token"); err != nil { + t.Fatal(err) + } + envelope := []byte{0xa3, 0x61, 0x62, 0x41, 0x00, 0x61, 0x74, 0x65, 'e', 'v', 'e', 'n', 't', 0x63, 's', 'i', 'g', 0x40} + r.svc.Event(r.home, envelope) + r.drain() + sent := r.apple.notifications() + if len(sent) != 1 { + t.Fatalf("sent %d", len(sent)) + } + n := sent[0] + want, _ := cloudwire.AlertPayload(cloudwire.Alert{Home: r.home, Event: envelope}) + if !bytes.Equal(n.Payload, want) || string(n.DeviceToken) != "owner-token" || n.Environment != apns.Sandbox { + t.Fatalf("notification %+v", n) + } + var p struct { + FTW cloudwire.Alert `json:"ftw"` + } + if json.Unmarshal(n.Payload, &p) != nil || !bytes.Equal(p.FTW.Event, envelope) { + t.Fatal("the box's envelope did not arrive unchanged") + } + if !n.Expiration.Equal(r.clock.time().Add(10 * time.Minute)) { + t.Fatalf("expires %v", n.Expiration) + } +} + +func TestOnlyADeviceThatMayReadEventsRegisters(t *testing.T) { + r := newRig(t) + if err := r.register(t, r.viewer, r.viewer.Scopes, "viewer-token"); err != ErrNotForThisDevice { + t.Fatalf("a viewer registered: %v", err) + } + // An owner under a grant that leaves events out has no events scope + // in its effective scope. + if err := r.register(t, r.owner, []string{protocol.ScopeLiveRead}, "owner-token"); err != ErrNotForThisDevice { + t.Fatalf("an owner without events registered: %v", err) + } + bad := []cloudwire.PushRegister{ + {APNsToken: []byte("t"), Environment: "staging", Kind: cloudwire.PushKindAlert}, + {APNsToken: []byte("t"), Environment: cloudwire.PushSandbox, Kind: "liveactivity"}, + {APNsToken: nil, Environment: cloudwire.PushSandbox, Kind: cloudwire.PushKindAlert}, + {APNsToken: make([]byte, 101), Environment: cloudwire.PushSandbox, Kind: cloudwire.PushKindAlert}, + } + for _, b := range bad { + if err := r.svc.Register(r.home, r.owner, r.all, b); err != ErrNotForThisDevice { + t.Fatalf("registered %+v: %v", b, err) + } + } + if regs, _ := r.reg.PushRegs(r.home, cloudwire.PushKindAlert); len(regs) != 0 { + t.Fatalf("kept %d", len(regs)) + } +} + +func TestASuspendedDeviceGetsNothingUntilTheSuspensionEnds(t *testing.T) { + r := newRig(t) + other := r.owner + other.ID, other.Key = "other-owner", bytes.Repeat([]byte{7}, 32) + if err := r.register(t, r.owner, r.all, "owner-token"); err != nil { + t.Fatal(err) + } + if err := r.register(t, other, r.all, "other-token"); err != nil { + t.Fatal(err) + } + if err := r.reg.Suspend(r.home, other.ID, other.Epoch, r.clock.time()); err != nil { + t.Fatal(err) + } + tokens := func() []string { + r.drain() + var out []string + for _, n := range r.apple.notifications() { + out = append(out, string(n.DeviceToken)) + } + r.apple.mu.Lock() + r.apple.sent = nil + r.apple.mu.Unlock() + return out + } + + r.svc.Event(r.home, []byte{0xa0}) + if got := tokens(); len(got) != 1 || got[0] != "owner-token" { + t.Fatalf("while suspended, sent to %v", got) + } + // box.unreachable takes the same road. + r.svc.BoxGone(r.home, "") + r.clock.advance(10 * time.Minute) + if got := tokens(); len(got) != 1 || got[0] != "owner-token" { + t.Fatalf("unreachable went to %v", got) + } + + if err := r.reg.Unsuspend(r.home, other.ID); err != nil { + t.Fatal(err) + } + r.svc.Event(r.home, []byte{0xa0}) + if got := tokens(); len(got) != 2 { + t.Fatalf("after the suspension ended, sent to %v", got) + } +} + +func TestARosterThatDropsOrNarrowsADeviceDropsItsTokens(t *testing.T) { + r := newRig(t) + other := r.owner + other.ID, other.Key = "second-owner", bytes.Repeat([]byte{7}, 32) + _ = r.register(t, r.owner, r.all, "a") + _ = r.register(t, other, r.all, "b") + grant := cloudwire.Grant{ID: bytes.Repeat([]byte{4}, 16), Scopes: r.all} + + // The second owner became a viewer: no events in its scopes. + narrowed := other + narrowed.Scopes, narrowed.Role = []string{protocol.ScopeLiveRead}, "viewer" + r.svc.Roster(r.home, &cloudwire.Roster{Devices: []cloudwire.Device{r.owner, narrowed}, Grant: grant}) + regs, _ := r.reg.PushRegs(r.home, cloudwire.PushKindAlert) + if len(regs) != 1 || regs[0].DeviceID != r.owner.ID { + t.Fatalf("after narrowing: %+v", regs) + } + // The owner paired again: a new epoch. + again := r.owner + again.Epoch = 2 + r.svc.Roster(r.home, &cloudwire.Roster{Devices: []cloudwire.Device{again}, Grant: grant}) + if regs, _ := r.reg.PushRegs(r.home, cloudwire.PushKindAlert); len(regs) != 0 { + t.Fatalf("after a new epoch: %+v", regs) + } + // And a grant that drops events drops every token. + _ = r.register(t, again, r.all, "c") + grant.Scopes = []string{protocol.ScopeLiveRead} + r.svc.Roster(r.home, &cloudwire.Roster{Devices: []cloudwire.Device{again}, Grant: grant}) + if regs, _ := r.reg.PushRegs(r.home, cloudwire.PushKindAlert); len(regs) != 0 { + t.Fatalf("after the grant dropped events: %+v", regs) + } +} + +func TestATokenAppleSaysIsGoneIsForgotten(t *testing.T) { + r := newRig(t) + _ = r.register(t, r.owner, r.all, "dead") + r.apple.gone["dead"] = true + r.svc.Event(r.home, []byte{0xa0}) + r.drain() + if regs, _ := r.reg.PushRegs(r.home, cloudwire.PushKindAlert); len(regs) != 0 { + t.Fatal("kept a token Apple called gone") + } +} + +func unreachableSent(t *testing.T, r *rig) int { + t.Helper() + r.drain() + n := 0 + for _, x := range r.apple.notifications() { + var p struct { + FTW cloudwire.Alert `json:"ftw"` + } + if json.Unmarshal(x.Payload, &p) == nil && p.FTW.Kind == protocol.PushBoxUnreachable { + if len(p.FTW.Event) != 0 || p.FTW.Home != r.home { + t.Fatalf("unreachable payload %s", x.Payload) + } + n++ + } + } + return n +} + +func TestBoxUnreachableAfterTenMinutesOncePerOutage(t *testing.T) { + r := newRig(t) + _ = r.register(t, r.owner, r.all, "tok") + r.svc.BoxConnected(r.home) + r.svc.BoxGone(r.home, "") + r.clock.advance(599 * time.Second) + if n := unreachableSent(t, r); n != 0 { + t.Fatal("before ten minutes") + } + r.clock.advance(2 * time.Second) + if n := unreachableSent(t, r); n != 1 { + t.Fatalf("sent %d at ten minutes", n) + } + r.clock.advance(time.Hour) + if n := unreachableSent(t, r); n != 1 { + t.Fatalf("sent %d over one outage", n) + } + // Back and away again is a new outage. + r.svc.BoxConnected(r.home) + r.svc.BoxGone(r.home, cloudwire.ByeShutdown) + r.clock.advance(601 * time.Second) + if n := unreachableSent(t, r); n != 2 { + t.Fatalf("sent %d over two outages", n) + } +} + +func TestABoxBackInTimeSaysNothing(t *testing.T) { + r := newRig(t) + _ = r.register(t, r.owner, r.all, "tok") + r.svc.BoxConnected(r.home) + r.svc.BoxGone(r.home, "") + r.clock.advance(5 * time.Minute) + r.svc.BoxConnected(r.home) + r.clock.advance(time.Hour) + if n := unreachableSent(t, r); n != 0 { + t.Fatalf("sent %d for a box that came back", n) + } +} + +// A planned update sends nothing, unless the box is still away after an +// hour. +func TestAPlannedUpdateHoldsItBackForAnHour(t *testing.T) { + r := newRig(t) + _ = r.register(t, r.owner, r.all, "tok") + r.svc.BoxConnected(r.home) + r.svc.BoxGone(r.home, cloudwire.ByeUpdate) + r.clock.advance(20 * time.Minute) + if n := unreachableSent(t, r); n != 0 { + t.Fatal("an update was reported as out of reach") + } + r.svc.BoxConnected(r.home) + r.clock.advance(2 * time.Hour) + if n := unreachableSent(t, r); n != 0 { + t.Fatal("an update that came back was reported") + } + + r.svc.BoxGone(r.home, cloudwire.ByeUpdate) + r.clock.advance(time.Hour + time.Second) + if n := unreachableSent(t, r); n != 1 { + t.Fatalf("an update still away after an hour: %d", n) + } +} + +func TestARevokedHomeHasNoDeadMan(t *testing.T) { + r := newRig(t) + _ = r.register(t, r.owner, r.all, "tok") + r.svc.BoxConnected(r.home) + r.svc.BoxGone(r.home, cloudwire.ByeRevoked) + r.clock.advance(2 * time.Hour) + if n := unreachableSent(t, r); n != 0 { + t.Fatal("a revoked home was reported out of reach") + } +} + +func TestAHomeCannotMakeItsPhonesBuzzWithoutEnd(t *testing.T) { + r := newRig(t) + _ = r.register(t, r.owner, r.all, "tok") + for range MaxAlertsPerHour + 5 { + r.svc.Event(r.home, []byte{0xa0}) + } + r.drain() + if n := len(r.apple.notifications()); n != MaxAlertsPerHour { + t.Fatalf("sent %d in an hour", n) + } + r.clock.advance(time.Hour) + r.svc.Event(r.home, []byte{0xa0}) + r.drain() + if n := len(r.apple.notifications()); n != MaxAlertsPerHour+1 { + t.Fatalf("the cap did not lift after an hour: %d", n) + } +} + +func TestWithoutAKeyNothingIsSent(t *testing.T) { + r := newRig(t) + r.svc.sender = nil + _ = r.register(t, r.owner, r.all, "tok") + r.svc.Event(r.home, []byte{0xa0}) + r.drain() + if regs, _ := r.reg.PushRegs(r.home, cloudwire.PushKindAlert); len(regs) != 1 { + t.Fatal("registrations need no key") + } +} diff --git a/internal/registry/push.go b/internal/registry/push.go new file mode 100644 index 0000000..1bc25ec --- /dev/null +++ b/internal/registry/push.go @@ -0,0 +1,135 @@ +package registry + +import ( + "database/sql" + "time" +) + +// APNs registrations. Each row binds one device token to one roster entry +// (device id and epoch) of one home. A row goes when its home goes (the +// foreign key cascades on purge and expiry), when a roster drops the device +// or moves its epoch, when the phone unregisters, when Apple says the token +// is gone, and after PushKeptWithoutBox without a roster from the box. + +// PushKeptWithoutBox is how long registrations outlive the box's last +// roster. The box renews its roster every six hours while it is connected. +const PushKeptWithoutBox = 30 * 24 * time.Hour + +// PushReg is one registration. +type PushReg struct { + Home string + DeviceID string + Epoch uint64 + Token []byte + Environment string + Kind string + At time.Time +} + +func createPushTable(db *sql.DB) error { + _, err := db.Exec(` +CREATE TABLE IF NOT EXISTS push ( + home TEXT NOT NULL REFERENCES homes(home) ON DELETE CASCADE, + device_id TEXT NOT NULL, + epoch INTEGER NOT NULL, + token BLOB NOT NULL, + environment TEXT NOT NULL, + kind TEXT NOT NULL, + registered_ms INTEGER NOT NULL, + PRIMARY KEY (home, token, kind) +);`) + return err +} + +// PushRegister keeps a device's token. A token already registered for the +// home moves to this device: a phone that paired again keeps its token. +func (r *Registry) PushRegister(reg PushReg) error { + _, err := r.db.Exec(`INSERT INTO push (home, device_id, epoch, token, environment, kind, registered_ms) + VALUES (?, ?, ?, ?, ?, ?, ?) + ON CONFLICT (home, token, kind) DO UPDATE SET device_id = excluded.device_id, epoch = excluded.epoch, + environment = excluded.environment, registered_ms = excluded.registered_ms`, + reg.Home, reg.DeviceID, int64(reg.Epoch), reg.Token, reg.Environment, reg.Kind, reg.At.UnixMilli()) + return err +} + +// PushUnregister forgets one of a device's tokens. +func (r *Registry) PushUnregister(home, deviceID string, token []byte) error { + _, err := r.db.Exec(`DELETE FROM push WHERE home = ? AND device_id = ? AND token = ?`, home, deviceID, token) + return err +} + +// PushRegs returns a home's registrations of one kind. +func (r *Registry) PushRegs(home, kind string) ([]PushReg, error) { + rows, err := r.db.Query(`SELECT device_id, epoch, token, environment, registered_ms FROM push + WHERE home = ? AND kind = ? ORDER BY registered_ms`, home, kind) + if err != nil { + return nil, err + } + defer rows.Close() + var out []PushReg + for rows.Next() { + reg := PushReg{Home: home, Kind: kind} + var epoch, ms int64 + if err := rows.Scan(®.DeviceID, &epoch, ®.Token, ®.Environment, &ms); err != nil { + return nil, err + } + reg.Epoch, reg.At = uint64(epoch), time.UnixMilli(ms) + out = append(out, reg) + } + return out, rows.Err() +} + +// PushGone forgets a token Apple says is gone, unless the phone registered +// it again after Apple last knew it valid. +func (r *Registry) PushGone(home string, token []byte, since time.Time) error { + if since.IsZero() { + _, err := r.db.Exec(`DELETE FROM push WHERE home = ? AND token = ?`, home, token) + return err + } + _, err := r.db.Exec(`DELETE FROM push WHERE home = ? AND token = ? AND registered_ms <= ?`, + home, token, since.UnixMilli()) + return err +} + +// PushPrune keeps only the registrations whose device keep allows at its +// epoch, as a new roster says. +func (r *Registry) PushPrune(home string, keep func(deviceID string, epoch uint64) bool) error { + rows, err := r.db.Query(`SELECT DISTINCT device_id, epoch FROM push WHERE home = ?`, home) + if err != nil { + return err + } + type entry struct { + id string + epoch int64 + } + var gone []entry + for rows.Next() { + var e entry + if err := rows.Scan(&e.id, &e.epoch); err != nil { + _ = rows.Close() + return err + } + if !keep(e.id, uint64(e.epoch)) { + gone = append(gone, e) + } + } + _ = rows.Close() + for _, e := range gone { + if _, err := r.db.Exec(`DELETE FROM push WHERE home = ? AND device_id = ? AND epoch = ?`, home, e.id, e.epoch); err != nil { + return err + } + } + return nil +} + +// PushExpire forgets the registrations of homes whose box has sent no +// roster for PushKeptWithoutBox, and returns how many went. +func (r *Registry) PushExpire(now time.Time) (int64, error) { + cutoff := now.Add(-PushKeptWithoutBox).UnixMilli() + res, err := r.db.Exec(`DELETE FROM push WHERE home IN (SELECT home FROM homes + WHERE (roster_ms > 0 AND roster_ms < ?) OR (roster_ms = 0 AND created_ms < ?))`, cutoff, cutoff) + if err != nil { + return 0, err + } + return res.RowsAffected() +} diff --git a/internal/registry/push_test.go b/internal/registry/push_test.go new file mode 100644 index 0000000..d2931d7 --- /dev/null +++ b/internal/registry/push_test.go @@ -0,0 +1,111 @@ +package registry + +import ( + "testing" + "time" +) + +func reg(home, device string, epoch uint64, token string, at time.Time) PushReg { + return PushReg{Home: home, DeviceID: device, Epoch: epoch, Token: []byte(token), + Environment: "sandbox", Kind: "alert", At: at} +} + +func TestARegistrationBelongsToARosterEntryAndGoesWithIt(t *testing.T) { + r := open(t) + now := time.Unix(1_791_820_800, 0) + if err := r.Claim("h1", []byte("pk"), now); err != nil { + t.Fatal(err) + } + for _, x := range []PushReg{reg("h1", "phone-a", 1, "tok-a", now), reg("h1", "phone-b", 2, "tok-b", now)} { + if err := r.PushRegister(x); err != nil { + t.Fatal(err) + } + } + got, err := r.PushRegs("h1", "alert") + if err != nil || len(got) != 2 { + t.Fatalf("%v %v", got, err) + } + + // The roster drops phone-b and moves phone-a's epoch: both go. + if err := r.PushPrune("h1", func(id string, epoch uint64) bool { return id == "phone-a" && epoch == 2 }); err != nil { + t.Fatal(err) + } + if got, _ := r.PushRegs("h1", "alert"); len(got) != 0 { + t.Fatalf("kept %v", got) + } +} + +func TestATokenMovesToTheDeviceThatRegistersIt(t *testing.T) { + r := open(t) + now := time.Unix(1_791_820_800, 0) + _ = r.Claim("h1", []byte("pk"), now) + _ = r.PushRegister(reg("h1", "phone-a", 1, "tok", now)) + _ = r.PushRegister(reg("h1", "phone-a2", 3, "tok", now.Add(time.Minute))) + got, _ := r.PushRegs("h1", "alert") + if len(got) != 1 || got[0].DeviceID != "phone-a2" || got[0].Epoch != 3 { + t.Fatalf("%+v", got) + } + if err := r.PushUnregister("h1", "phone-a", []byte("tok")); err != nil { + t.Fatal(err) + } + if got, _ := r.PushRegs("h1", "alert"); len(got) != 1 { + t.Fatal("another device unregistered a token that is not its own") + } + _ = r.PushUnregister("h1", "phone-a2", []byte("tok")) + if got, _ := r.PushRegs("h1", "alert"); len(got) != 0 { + t.Fatal("unregister kept the token") + } +} + +func TestAGoneTokenGoesUnlessRegisteredSince(t *testing.T) { + r := open(t) + now := time.Unix(1_791_820_800, 0) + _ = r.Claim("h1", []byte("pk"), now) + _ = r.PushRegister(reg("h1", "phone-a", 1, "tok", now)) + if err := r.PushGone("h1", []byte("tok"), now.Add(-time.Hour)); err != nil { + t.Fatal(err) + } + if got, _ := r.PushRegs("h1", "alert"); len(got) != 1 { + t.Fatal("a token registered after Apple last knew it was dropped") + } + _ = r.PushGone("h1", []byte("tok"), now.Add(time.Hour)) + if got, _ := r.PushRegs("h1", "alert"); len(got) != 0 { + t.Fatal("a gone token was kept") + } +} + +func TestAPurgedOrSilentHomeTakesItsRegistrationsWithIt(t *testing.T) { + r := open(t) + now := time.Unix(1_791_820_800, 0) + for _, h := range []string{"h1", "h2", "h3"} { + _ = r.Claim(h, []byte("pk-"+h), now) + _ = r.SetRoster(h, 1, []byte("g"), []byte("roster"), now) + _ = r.PushRegister(reg(h, "phone", 1, "tok-"+h, now)) + } + if err := r.Purge("h1", []byte("g"), now); err != nil { + t.Fatal(err) + } + if got, _ := r.PushRegs("h1", "alert"); len(got) != 0 { + t.Fatal("a purged home kept its registrations") + } + // h3's box keeps sending its roster; h2's went quiet a month ago. + later := now.Add(PushKeptWithoutBox + time.Hour) + _ = r.SetRoster("h3", 2, []byte("g"), []byte("roster"), later) + n, err := r.PushExpire(later) + if err != nil || n != 1 { + t.Fatalf("expired %d, %v", n, err) + } + if got, _ := r.PushRegs("h2", "alert"); len(got) != 0 { + t.Fatal("a home silent for a month kept its registrations") + } + if got, _ := r.PushRegs("h3", "alert"); len(got) != 1 { + t.Fatal("a home whose box is there lost its registrations") + } +} + +func TestARegistrationNeedsAHome(t *testing.T) { + r := open(t) + if err := r.PushRegister(reg("nowhere", "phone", 1, "tok", time.Now())); err == nil { + t.Fatal("registered a token for a home that does not exist") + } +} diff --git a/internal/registry/registry.go b/internal/registry/registry.go index 6808993..a2f1291 100644 --- a/internal/registry/registry.go +++ b/internal/registry/registry.go @@ -44,7 +44,7 @@ type Home struct { // Open opens or creates the registry at path. func Open(path string) (*Registry, error) { - db, err := sql.Open("sqlite", "file:"+path+"?_pragma=journal_mode(WAL)&_pragma=secure_delete(ON)&_pragma=busy_timeout(5000)") + db, err := sql.Open("sqlite", "file:"+path+"?_pragma=journal_mode(WAL)&_pragma=secure_delete(ON)&_pragma=busy_timeout(5000)&_pragma=foreign_keys(1)") if err != nil { return nil, err } @@ -63,11 +63,22 @@ CREATE TABLE IF NOT EXISTS tombstones ( home TEXT PRIMARY KEY, grant_id BLOB, deleted_ms INTEGER NOT NULL +); +CREATE TABLE IF NOT EXISTS suspensions ( + home TEXT NOT NULL, + device_id TEXT NOT NULL, + epoch INTEGER NOT NULL, + since_ms INTEGER NOT NULL, + PRIMARY KEY (home, device_id) );`) if err != nil { _ = db.Close() return nil, fmt.Errorf("registry: schema: %w", err) } + if err := createPushTable(db); err != nil { + _ = db.Close() + return nil, fmt.Errorf("registry: schema: %w", err) + } return &Registry{db: db}, nil } @@ -130,6 +141,9 @@ func (r *Registry) Purge(home string, grantID []byte, now time.Time) error { if _, err := tx.Exec(`DELETE FROM homes WHERE home = ?`, home); err != nil { return err } + if _, err := tx.Exec(`DELETE FROM suspensions WHERE home = ?`, home); err != nil { + return err + } if _, err := tx.Exec(`INSERT INTO tombstones (home, grant_id, deleted_ms) VALUES (?, ?, ?) ON CONFLICT (home) DO NOTHING`, home, grantID, now.UnixMilli()); err != nil { return err @@ -182,7 +196,66 @@ func (r *Registry) Expire(now time.Time) ([]string, error) { if _, err := r.db.Exec(`DELETE FROM homes WHERE home = ?`, h); err != nil { return nil, err } + if _, err := r.db.Exec(`DELETE FROM suspensions WHERE home = ?`, h); err != nil { + return nil, err + } } _, err = r.db.Exec(`DELETE FROM tombstones WHERE deleted_ms < ?`, now.Add(-TombstoneKept).UnixMilli()) return gone, err } + +// Suspend records an owner's suspension of one roster device at its current +// epoch. It only narrows access: the device must already be in the roster, +// which the caller checks, and the suspension ends when the roster drops the +// device or its epoch changes. +func (r *Registry) Suspend(home, deviceID string, epoch uint64, now time.Time) error { + _, err := r.db.Exec(`INSERT INTO suspensions (home, device_id, epoch, since_ms) VALUES (?, ?, ?, ?) + ON CONFLICT (home, device_id) DO UPDATE SET epoch = excluded.epoch, since_ms = excluded.since_ms`, + home, deviceID, int64(epoch), now.UnixMilli()) + return err +} + +// Unsuspend lifts a suspension. +func (r *Registry) Unsuspend(home, deviceID string) error { + _, err := r.db.Exec(`DELETE FROM suspensions WHERE home = ? AND device_id = ?`, home, deviceID) + return err +} + +// Suspended reports whether a device is suspended at this epoch. A read that +// fails counts as suspended: the safe answer narrows. +func (r *Registry) Suspended(home, deviceID string, epoch uint64) bool { + var held int64 + err := r.db.QueryRow(`SELECT epoch FROM suspensions WHERE home = ? AND device_id = ?`, home, deviceID).Scan(&held) + if errors.Is(err, sql.ErrNoRows) { + return false + } + return err != nil || uint64(held) == epoch +} + +// PruneSuspensions drops the suspensions a new roster ended: devices it no +// longer lists, and devices whose epoch moved. +func (r *Registry) PruneSuspensions(home string, epochs map[string]uint64) error { + rows, err := r.db.Query(`SELECT device_id, epoch FROM suspensions WHERE home = ?`, home) + if err != nil { + return err + } + var gone []string + for rows.Next() { + var id string + var epoch int64 + if err := rows.Scan(&id, &epoch); err != nil { + _ = rows.Close() + return err + } + if e, ok := epochs[id]; !ok || e != uint64(epoch) { + gone = append(gone, id) + } + } + _ = rows.Close() + for _, id := range gone { + if err := r.Unsuspend(home, id); err != nil { + return err + } + } + return nil +} diff --git a/internal/registry/registry_test.go b/internal/registry/registry_test.go index b878673..6a9a685 100644 --- a/internal/registry/registry_test.go +++ b/internal/registry/registry_test.go @@ -70,3 +70,31 @@ func TestHomesWithoutARosterExpire(t *testing.T) { t.Fatalf("after 90 days: %v", gone) } } + +// A suspension holds at the epoch it was made at, ends when the roster drops +// the device or pairs it again, and goes with the home. +func TestASuspensionHoldsUntilTheRosterMovesPastIt(t *testing.T) { + r := open(t) + now := time.Unix(1_791_820_800, 0) + _ = r.Claim("h1", []byte("pk"), now) + if err := r.Suspend("h1", "phone", 2, now); err != nil { + t.Fatal(err) + } + if !r.Suspended("h1", "phone", 2) || r.Suspended("h1", "phone", 3) || r.Suspended("h2", "phone", 2) { + t.Fatal("a suspension reached past its device and epoch") + } + if err := r.PruneSuspensions("h1", map[string]uint64{"phone": 2}); err != nil || !r.Suspended("h1", "phone", 2) { + t.Fatal("an unchanged roster lifted a suspension") + } + if err := r.PruneSuspensions("h1", map[string]uint64{"phone": 3}); err != nil || r.Suspended("h1", "phone", 3) { + t.Fatal("a new epoch kept the suspension") + } + _ = r.Suspend("h1", "phone", 3, now) + if err := r.PruneSuspensions("h1", map[string]uint64{}); err != nil || r.Suspended("h1", "phone", 3) { + t.Fatal("a device the roster dropped stayed suspended") + } + _ = r.Suspend("h1", "phone", 3, now) + if err := r.Purge("h1", nil, now); err != nil || r.Suspended("h1", "phone", 3) { + t.Fatal("a purge left the suspension") + } +} diff --git a/internal/server/server.go b/internal/server/server.go index 3e4b009..24a8230 100644 --- a/internal/server/server.go +++ b/internal/server/server.go @@ -2,7 +2,7 @@ // route is listed in Routes, and TestNoRouteReadsAHome holds the list. // // /v1/box a box's uplink (package uplink) -// /v1/app the app's Noise session to the service: C3, not built yet +// /v1/app a phone's Noise session to the service (package appsession) // /v1/now an extension's token read: C5, not built yet // /source the source of the running service, as AGPL-3.0 asks // /healthz liveness, and nothing about any home @@ -19,6 +19,7 @@ import ( "runtime/debug" "time" + "github.com/srcfl/ftw-cloud/internal/appsession" "github.com/srcfl/ftw-cloud/internal/registry" "github.com/srcfl/ftw-cloud/internal/store" "github.com/srcfl/ftw-cloud/internal/uplink" @@ -39,18 +40,20 @@ type Server struct { Store *store.Store Registry *registry.Registry Uplink *uplink.Uplink - Log *slog.Logger + // App is nil on a host without a Noise key; /v1/app then says so. + App *appsession.Server + Log *slog.Logger } // Handler is the whole HTTP surface. func (s *Server) Handler() http.Handler { mux := http.NewServeMux() mux.Handle("/v1/box", s.Uplink) - // C3: the app opens a WebSocket here and runs Noise IK to the service's - // static key, with prologue "ftw.cloud.v1:" and the home id. The - // service admits a phone only through store.For with a Device - // principal, and forwards the box's envelopes unchanged. - mux.HandleFunc("/v1/app", notYet) + if s.App != nil { + mux.Handle("/v1/app", s.App) + } else { + mux.HandleFunc("/v1/app", notYet) + } // C5: widgets read the newest live, prices and hb here with a 7-day // token bound to one roster entry. mux.HandleFunc("/v1/now", notYet) diff --git a/internal/server/server_test.go b/internal/server/server_test.go index 203c80a..a1c13f6 100644 --- a/internal/server/server_test.go +++ b/internal/server/server_test.go @@ -13,9 +13,11 @@ import ( "slices" "testing" + "github.com/srcfl/ftw-cloud/internal/appsession" "github.com/srcfl/ftw-cloud/internal/registry" "github.com/srcfl/ftw-cloud/internal/store" "github.com/srcfl/ftw-cloud/internal/uplink" + "github.com/srcfl/ftw/go/protocol/appwire" "github.com/srcfl/ftw/go/protocol/cloudwire" ) @@ -29,13 +31,16 @@ func newServer(t *testing.T) (*Server, string) { t.Cleanup(func() { _ = reg.Close() }) st := store.New(nil) log := slog.New(slog.NewTextHandler(io.Discard, nil)) - s := &Server{Store: st, Registry: reg, Uplink: uplink.New(st, reg, key, log), Log: log} + up := uplink.New(st, reg, key, log) + noise, _ := appwire.GenerateKeyPair() + s := &Server{Store: st, Registry: reg, Uplink: up, App: appsession.New(st, reg, up, noise, log), Log: log} // A home with readings in it, held the way a box's uplink holds them. - box, _ := st.For(store.Principal{Kind: store.Box, Home: "canaryhome"}) + const home = "canaryhomecanaryhomecanary" + box, _ := st.For(store.Principal{Kind: store.Box, Home: home}) box.Connected() _ = box.Put(cloudwire.MsgLive, []byte("CANARY-READING"), 1, true) - return s, "canaryhome" + return s, home } // No route reads a home. Every path answers without the canary, whatever the @@ -93,11 +98,25 @@ func TestSourceNamesRevisionAndImage(t *testing.T) { } } -func TestTheUplinkRefusesPlainHTTP(t *testing.T) { - s, _ := newServer(t) +func TestTheSocketsRefusePlainHTTP(t *testing.T) { + s, home := newServer(t) + for _, p := range []string{"/v1/box", "/v1/app?home=" + home} { + w := httptest.NewRecorder() + s.Handler().ServeHTTP(w, httptest.NewRequest(http.MethodGet, p, nil)) + if w.Code != http.StatusUpgradeRequired { + t.Fatalf("plain HTTP to %s: %d", p, w.Code) + } + } +} + +// A host without a Noise key serves boxes and says /v1/app is not there yet; +// phones read through the relay meanwhile. +func TestWithoutANoiseKeyTheAppEndpointIsOff(t *testing.T) { + s, home := newServer(t) + s.App = nil w := httptest.NewRecorder() - s.Handler().ServeHTTP(w, httptest.NewRequest(http.MethodGet, "/v1/box", nil)) - if w.Code != http.StatusUpgradeRequired { - t.Fatalf("plain HTTP to /v1/box: %d", w.Code) + s.Handler().ServeHTTP(w, httptest.NewRequest(http.MethodGet, "/v1/app?home="+home, nil)) + if w.Code != http.StatusNotImplemented { + t.Fatalf("/v1/app without a Noise key: %d", w.Code) } } diff --git a/internal/store/session.go b/internal/store/session.go new file mode 100644 index 0000000..7398408 --- /dev/null +++ b/internal/store/session.go @@ -0,0 +1,121 @@ +package store + +import ( + "bytes" + + "github.com/srcfl/ftw/go/protocol/cloudwire" +) + +// Limits on the phones reading one home. +const ( + // MaxSessionsPerHome bounds the phones one home can hold open at once. + // A household has a few phones; past this the home is busy, as it is at + // the relay. + MaxSessionsPerHome = 16 + // SessionQueue is how many envelopes may wait for one slow phone. A box + // sends a few a second while watched, so a phone this far behind has a + // dead socket. It is closed and gets a fresh snapshot when it comes + // back, rather than a stream with a hole in it. + SessionQueue = 256 + + // WebSocket close codes for the two endings that are not the protocol's + // own: the phone left, or it fell too far behind. + closeNormal = 1000 + closeSlow = 1013 +) + +// Session is one phone's live view of its home: every envelope the box sends +// after the snapshot that the device may read, in the order the box sent +// them. The store ends it when the device's access changes. +type Session struct { + x *Handle + c chan []byte + // done is closed when the store ends the session; code says why, as a + // close code from cloudwire. + done chan struct{} + code int + ended bool +} + +// Open returns the device's snapshot and a session that receives every later +// envelope it may read. Both are taken under one lock, so nothing the box +// sends falls between them. A second session from the same device replaces +// the first. +func (x *Handle) Open() (Snapshot, *Session, error) { + if x.kind != Device { + return Snapshot{}, nil, ErrDenied + } + x.h.mu.Lock() + defer x.h.mu.Unlock() + // The access checked in For may have changed since: check it again + // under the lock that roster changes take. + if _, _, ok := x.s.access(x.id, x.h, x.device.Key); !ok { + return Snapshot{}, nil, ErrDenied + } + for sess := range x.h.sessions { + if bytes.Equal(sess.x.device.Key, x.device.Key) { + sess.endLocked(cloudwire.CloseReplaced) + } + } + if len(x.h.sessions) >= MaxSessionsPerHome { + return Snapshot{}, nil, ErrBusy + } + sess := &Session{x: x, c: make(chan []byte, SessionQueue), done: make(chan struct{})} + x.h.sessions[sess] = struct{}{} + return x.snapshotLocked(), sess, nil +} + +// C carries the envelopes to forward. +func (s *Session) C() <-chan []byte { return s.c } + +// Done is closed when the store ends the session. +func (s *Session) Done() <-chan struct{} { return s.done } + +// Code is why the store ended the session, once Done is closed. +func (s *Session) Code() int { + s.x.h.mu.Lock() + defer s.x.h.mu.Unlock() + return s.code +} + +// Close is the phone leaving. It is safe to call more than once. +func (s *Session) Close() { + s.x.h.mu.Lock() + defer s.x.h.mu.Unlock() + s.endLocked(closeNormal) +} + +// deliver queues one envelope without ever blocking the box. The caller holds +// the home's lock. +func (s *Session) deliver(env []byte) { + if s.ended { + return + } + select { + case s.c <- env: + default: + s.endLocked(closeSlow) + } +} + +// endLocked ends the session once. The caller holds the home's lock. +func (s *Session) endLocked(code int) { + if s.ended { + return + } + s.ended, s.code = true, code + delete(s.x.h.sessions, s) + close(s.done) +} + +// Device is the roster entry this device matched. +func (x *Handle) Device() cloudwire.Device { return x.device } + +// Scopes is the device's effective scope. +func (x *Handle) Scopes() []string { return x.scopes } + +// Home is the home this handle reaches. +func (x *Handle) Home() string { return x.id } + +// MayRead reports whether the device may read a class. +func (x *Handle) MayRead(class string) bool { return x.mayRead(class) } diff --git a/internal/store/store.go b/internal/store/store.go index 0713484..1d46fff 100644 --- a/internal/store/store.go +++ b/internal/store/store.go @@ -44,11 +44,17 @@ type Principal struct { // ErrDenied is every refusal. It does not say whether the home exists. var ErrDenied = errors.New("store: denied") +// ErrBusy is a home with as many phone sessions open as it may have. +var ErrBusy = errors.New("store: busy") + // Store is every home the service holds. type Store struct { mu sync.Mutex homes map[string]*home now func() time.Time + // suspended reports an owner's suspension of a roster device. Nil + // suspends nobody. + suspended func(home, deviceID string, epoch uint64) bool } // New starts an empty store. now is for tests; nil uses the clock. @@ -59,6 +65,12 @@ func New(now func() time.Time) *Store { return &Store{homes: map[string]*home{}, now: now} } +// UseSuspensions sets where owners' suspensions are kept. Call it before +// serving. +func (s *Store) UseSuspensions(suspended func(home, deviceID string, epoch uint64) bool) { + s.suspended = suspended +} + type home struct { mu sync.Mutex // latest is the newest envelope of each class, exactly as the box sent @@ -70,6 +82,8 @@ type home struct { online bool conn uint64 lastSeen time.Time + // sessions are the phones reading this home now. + sessions map[*Session]struct{} } // Held is one envelope as received. @@ -85,19 +99,20 @@ type Handle struct { id string h *home kind Kind - // scopes is a device's effective scope: its roster scopes intersected - // with the grant's. + // device is the roster entry a device principal matched, and scopes its + // effective scope: its roster scopes intersected with the grant's. + device cloudwire.Device scopes []string } // For returns the principal's access to its home, or ErrDenied. A box may // create its home; a device reaches only a home whose current roster lists -// its key. +// its key, and only while no owner has suspended it. func (s *Store) For(p Principal) (*Handle, error) { s.mu.Lock() h := s.homes[p.Home] if h == nil && p.Kind == Box && p.Home != "" { - h = &home{latest: map[string]Held{}} + h = &home{latest: map[string]Held{}, sessions: map[*Session]struct{}{}} s.homes[p.Home] = h } s.mu.Unlock() @@ -110,23 +125,38 @@ func (s *Store) For(p Principal) (*Handle, error) { case Device: h.mu.Lock() defer h.mu.Unlock() - r := h.roster - if r == nil || s.now().After(h.rosterAt.Add(time.Duration(r.TTLS)*time.Second)) { + dev, scopes, ok := s.access(p.Home, h, p.DeviceKey) + if !ok { return nil, ErrDenied } - for _, d := range r.Devices { - if bytes.Equal(d.Key, p.DeviceKey) && len(p.DeviceKey) == 32 { - var scopes []string - for _, sc := range d.Scopes { - if slices.Contains(r.Grant.Scopes, sc) { - scopes = append(scopes, sc) - } - } - return &Handle{s: s, id: p.Home, h: h, kind: Device, scopes: scopes}, nil + return &Handle{s: s, id: p.Home, h: h, kind: Device, device: dev, scopes: scopes}, nil + } + return nil, ErrDenied +} + +// access is the one admission rule for a device: a roster in force that +// lists its key, and no suspension. The caller holds h.mu. +func (s *Store) access(id string, h *home, key []byte) (cloudwire.Device, []string, bool) { + r := h.roster + if r == nil || len(key) != 32 || s.now().After(h.rosterAt.Add(time.Duration(r.TTLS)*time.Second)) { + return cloudwire.Device{}, nil, false + } + for _, d := range r.Devices { + if !bytes.Equal(d.Key, key) { + continue + } + if s.suspended != nil && s.suspended(id, d.ID, d.Epoch) { + return cloudwire.Device{}, nil, false + } + var scopes []string + for _, sc := range d.Scopes { + if slices.Contains(r.Grant.Scopes, sc) { + scopes = append(scopes, sc) } } + return d, scopes, true } - return nil, ErrDenied + return cloudwire.Device{}, nil, false } // -------------------------------------------------------------------------- @@ -167,6 +197,11 @@ func (x *Handle) Put(t string, env []byte, seq uint64, carriesTable bool) error x.h.full = &held } x.h.lastSeen = x.s.now() + for sess := range x.h.sessions { + if sess.x.mayRead(t) { + sess.deliver(env) + } + } return nil } @@ -192,9 +227,32 @@ func (x *Handle) SetRoster(r *cloudwire.Roster) bool { return false } x.h.roster, x.h.rosterAt = r, x.s.now() + x.recheckLocked() return true } +// Recheck ends every session whose access is no longer what it was opened +// with, as after an owner suspends a device. The phone reconnects and learns +// its new access, or is refused. +func (x *Handle) Recheck() { + x.h.mu.Lock() + defer x.h.mu.Unlock() + x.recheckLocked() +} + +func (x *Handle) recheckLocked() { + for sess := range x.h.sessions { + dev, scopes, ok := x.s.access(x.id, x.h, sess.x.device.Key) + switch { + case !ok: + sess.endLocked(cloudwire.CloseDenied) + case dev.ID != sess.x.device.ID || dev.Epoch != sess.x.device.Epoch || dev.Role != sess.x.device.Role || + !slices.Equal(scopes, sess.x.scopes): + sess.endLocked(cloudwire.CloseChanged) + } + } +} + // Delete forgets the home. Only its box may. func (x *Handle) Delete() { if x.kind != Box { @@ -205,6 +263,9 @@ func (x *Handle) Delete() { x.s.mu.Unlock() x.h.mu.Lock() x.h.latest, x.h.full, x.h.roster = map[string]Held{}, nil, nil + for sess := range x.h.sessions { + sess.endLocked(cloudwire.CloseDenied) + } x.h.mu.Unlock() } @@ -228,9 +289,13 @@ func (x *Handle) Snapshot() (Snapshot, error) { } x.h.mu.Lock() defer x.h.mu.Unlock() + return x.snapshotLocked(), nil +} + +func (x *Handle) snapshotLocked() Snapshot { out := Snapshot{LastSeenMs: x.h.lastSeen.UnixMilli(), Online: x.h.online} if !x.h.online && x.s.now().Sub(x.h.lastSeen) > ReadingsKeptFor { - return out, nil + return out } var held []Held for t, e := range x.h.latest { @@ -245,7 +310,7 @@ func (x *Handle) Snapshot() (Snapshot, error) { for _, e := range held { out.Envelopes = append(out.Envelopes, e.Envelope) } - return out, nil + return out } func (x *Handle) mayRead(class string) bool { @@ -272,6 +337,13 @@ func (s *Store) Sweep() { if !h.online && now.Sub(h.lastSeen) > ReadingsKeptFor { h.latest, h.full = map[string]Held{}, nil } + // A roster that ran out closes the home's sessions: the box has not + // confirmed who may read for a day. + if r := h.roster; r != nil && now.After(h.rosterAt.Add(time.Duration(r.TTLS)*time.Second)) { + for sess := range h.sessions { + sess.endLocked(cloudwire.CloseDenied) + } + } h.mu.Unlock() } } @@ -280,6 +352,14 @@ func (s *Store) Sweep() { // roster. It is called by upkeep, never by a request. func (s *Store) Forget(id string) { s.mu.Lock() - defer s.mu.Unlock() + h := s.homes[id] delete(s.homes, id) + s.mu.Unlock() + if h != nil { + h.mu.Lock() + for sess := range h.sessions { + sess.endLocked(cloudwire.CloseDenied) + } + h.mu.Unlock() + } } diff --git a/internal/store/store_test.go b/internal/store/store_test.go index d0c1078..162077a 100644 --- a/internal/store/store_test.go +++ b/internal/store/store_test.go @@ -165,3 +165,98 @@ func TestDeleteForgetsTheHome(t *testing.T) { t.Fatal("a deleted home still answers") } } + +// A session's snapshot and stream join without a gap: what the box sends +// after Open reaches the session, and nothing before it does twice. +func TestASessionMissesNothingAfterItsSnapshot(t *testing.T) { + s, box, _ := filled(t) + dev, _ := s.For(Principal{Kind: Device, Home: "h1", DeviceKey: viewerKey}) + snap, sess, err := dev.Open() + if err != nil { + t.Fatal(err) + } + defer sess.Close() + if len(snap.Envelopes) != 3 { + t.Fatalf("snapshot held %d", len(snap.Envelopes)) + } + _ = box.Put(cloudwire.MsgPlan, []byte("plan-2"), 8, false) // outside the viewer's scope + _ = box.Put(cloudwire.MsgLive, []byte("live-3"), 9, false) + select { + case got := <-sess.C(): + if string(got) != "live-3" { + t.Fatalf("the session got %q", got) + } + default: + t.Fatal("nothing reached the session") + } + select { + case got := <-sess.C(): + t.Fatalf("the session also got %q", got) + default: + } +} + +// A phone too far behind is ended, never allowed to stall the box. +func TestASlowSessionIsEndedNotWaitedFor(t *testing.T) { + s, box, _ := filled(t) + dev, _ := s.For(Principal{Kind: Device, Home: "h1", DeviceKey: ownerKey}) + _, sess, _ := dev.Open() + for i := range SessionQueue + 1 { + _ = box.Put(cloudwire.MsgHB, []byte("hb"), uint64(100+i), false) + } + select { + case <-sess.Done(): + default: + t.Fatal("a full queue left the session open") + } + if sess.Code() != closeSlow { + t.Fatalf("ended with %d", sess.Code()) + } +} + +func TestARosterThatRunsOutEndsSessions(t *testing.T) { + s, _, c := filled(t) + dev, _ := s.For(Principal{Kind: Device, Home: "h1", DeviceKey: ownerKey}) + _, sess, _ := dev.Open() + c.t = c.t.Add(24*time.Hour + time.Second) + s.Sweep() + select { + case <-sess.Done(): + default: + t.Fatal("an expired roster left a session open") + } + if sess.Code() != cloudwire.CloseDenied { + t.Fatalf("ended with %d", sess.Code()) + } +} + +func TestForgettingAHomeEndsItsSessions(t *testing.T) { + s, _, _ := filled(t) + dev, _ := s.For(Principal{Kind: Device, Home: "h1", DeviceKey: ownerKey}) + _, sess, _ := dev.Open() + s.Forget("h1") + select { + case <-sess.Done(): + default: + t.Fatal("a forgotten home kept a session") + } +} + +// A suspended device is refused, and only at the epoch it was suspended at. +func TestSuspensionsAreConsultedOnEveryAdmission(t *testing.T) { + s, _, _ := filled(t) + s.UseSuspensions(func(home, id string, epoch uint64) bool { return home == "h1" && id == "viewer" && epoch == 0 }) + if _, err := s.For(Principal{Kind: Device, Home: "h1", DeviceKey: viewerKey}); !errors.Is(err, ErrDenied) { + t.Fatal("a suspended device was admitted") + } + if _, err := s.For(Principal{Kind: Device, Home: "h1", DeviceKey: ownerKey}); err != nil { + t.Fatal("the owner was refused with the viewer") + } +} + +func TestABoxHandleCannotOpenASession(t *testing.T) { + _, box, _ := filled(t) + if _, _, err := box.Open(); !errors.Is(err, ErrDenied) { + t.Fatal("the box's handle opened a session") + } +} diff --git a/internal/uplink/hooks_test.go b/internal/uplink/hooks_test.go new file mode 100644 index 0000000..8ffe84d --- /dev/null +++ b/internal/uplink/hooks_test.go @@ -0,0 +1,108 @@ +package uplink + +import ( + "bytes" + "sync" + "testing" + "time" + + "github.com/srcfl/ftw/go/protocol" + "github.com/srcfl/ftw/go/protocol/cloudwire" +) + +// heard records what the uplink tells push. +type heard struct { + mu sync.Mutex + events [][]byte + rosters []uint64 + gone []string + conns int +} + +func listen(s *service) *heard { + h := &heard{} + s.up.OnConnect = func(string) { h.mu.Lock(); h.conns++; h.mu.Unlock() } + s.up.OnEvent = func(home string, env []byte) { + h.mu.Lock() + h.events = append(h.events, env) + h.mu.Unlock() + } + s.up.OnRoster = func(home string, r *cloudwire.Roster) { + h.mu.Lock() + h.rosters = append(h.rosters, r.RosterSeq) + h.mu.Unlock() + } + s.up.OnGone = func(home, why string) { + h.mu.Lock() + h.gone = append(h.gone, why) + h.mu.Unlock() + } + return h +} + +func (h *heard) count(of func(*heard) int) func() int { + return func() int { + h.mu.Lock() + defer h.mu.Unlock() + return of(h) + } +} + +// A signed event reaches push as the envelope the box sent, and only a +// signed one. +func TestAnEventReachesPushUnchanged(t *testing.T) { + s := newService(t) + h := listen(s) + b := newBox(t) + _, _ = b.connect(s, nil) + b.send(cloudwire.MsgRoster, b.roster(protocol.ScopeLiveRead, protocol.ScopeEventsRead)) + sent := b.send(cloudwire.MsgEvent, cloudwire.Event{Header: b.header(), EventID: "e1", + Kind: protocol.PushChargingInterrupted, Severity: cloudwire.SeverityWarning, Args: map[string]string{}}) + events := h.count(func(h *heard) int { return len(h.events) }) + wait(t, "the event", func() bool { return events() == 1 }) + if !bytes.Equal(h.events[0], sent) { + t.Fatal("push got other bytes than the box sent") + } + if rosters := h.count(func(h *heard) int { return len(h.rosters) }); rosters() != 1 { + t.Fatal("push did not hear the roster") + } +} + +// The end of an uplink tells push why: the box's bye, or nothing when the +// socket dropped. A connection a newer one replaced says nothing. +func TestTheEndOfAnUplinkSaysWhy(t *testing.T) { + s := newService(t) + h := listen(s) + gone := h.count(func(h *heard) int { return len(h.gone) }) + conns := h.count(func(h *heard) int { return h.conns }) + + b := newBox(t) + _, _ = b.connect(s, nil) + b.send(cloudwire.MsgRoster, b.roster(protocol.ScopeLiveRead)) + wait(t, "the connect", func() bool { return conns() == 1 }) + b.send(cloudwire.MsgBye, cloudwire.Bye{Header: b.header(), Why: cloudwire.ByeUpdate}) + wait(t, "the bye", func() bool { return gone() == 1 }) + if h.gone[0] != cloudwire.ByeUpdate { + t.Fatalf("gone with %q", h.gone[0]) + } + + _, _ = b.connect(s, nil) + second := *b + second.conn = nil + _, _ = second.connect(s, nil) + if !b.closed() { + t.Fatal("two uplinks for one home") + } + time.Sleep(100 * time.Millisecond) + if gone() != 1 { + t.Fatal("a replaced uplink told push the box was gone") + } + _ = second.conn.Close() + wait(t, "the drop", func() bool { return gone() == 2 }) + if h.gone[1] != "" { + t.Fatalf("a dropped socket gave %q", h.gone[1]) + } + if conns() != 3 { + t.Fatalf("connects %d", conns()) + } +} diff --git a/internal/uplink/uplink.go b/internal/uplink/uplink.go index da1e4f6..870d7f7 100644 --- a/internal/uplink/uplink.go +++ b/internal/uplink/uplink.go @@ -62,6 +62,19 @@ type Uplink struct { // Dropped counts box messages of a type this service does not know. A // newer box works with an older service by this rule. Dropped atomic.Int64 + + // OnConnect runs after a box's hello_ok, so the app sessions can ask a + // box that just came back for one-second updates and push can stop + // waiting to say it is out of reach. Set before serving. + OnConnect func(home string) + // OnEvent gets each event the box signed, as the envelope it arrived + // in; OnRoster each roster the service keeps; OnGone the end of the + // home's current uplink, with the reason the box gave in its bye, or + // "" when the socket just dropped. Push uses them. Each runs on the + // box's goroutine and must not block. Set before serving. + OnEvent func(home string, envelope []byte) + OnRoster func(home string, r *cloudwire.Roster) + OnGone func(home, why string) } // New builds the handler. key is the service key that signs hello_ok and @@ -107,7 +120,7 @@ func (u *Uplink) ServeHTTP(w http.ResponseWriter, r *http.Request) { } // Watch asks a home's box for one-second updates for a minute. The app -// session (C3) calls it while a phone is looking. +// sessions call it while a phone is looking. func (u *Uplink) Watch(home string) { u.mu.Lock() c := u.conns[home] @@ -130,6 +143,8 @@ type boxConn struct { pub *ecdsa.PublicKey handle *store.Handle purged bool + // bye is the reason the box gave for leaving, if it gave one. + bye string tokens float64 refilled time.Time @@ -192,16 +207,23 @@ func (c *boxConn) run() error { c.u.mu.Unlock() defer func() { c.u.mu.Lock() - if c.u.conns[c.home] == c { + current := c.u.conns[c.home] == c + if current { delete(c.u.conns, c.home) } c.u.mu.Unlock() + if current && c.u.OnGone != nil { + c.u.OnGone(c.home, c.bye) + } }() } if err := c.send(helloOK{BoxNonce: hello.BoxNonce, EKM: c.ekm, Have: map[string]string{}, RosterSeq: c.rosterSeq()}); err != nil { return err } + if !c.purged && c.u.OnConnect != nil { + go c.u.OnConnect(c.home) + } done := make(chan struct{}) defer close(done) @@ -327,6 +349,7 @@ func (c *boxConn) apply(env cloudwire.Envelope) (stop bool, err error) { if err := cloudwire.Unmarshal(env.B, &b); err != nil { return true, err } + c.bye = b.Why if b.Purge { return true, c.purge() } @@ -343,7 +366,13 @@ func (c *boxConn) apply(env cloudwire.Envelope) (stop bool, err error) { if err != nil { return true, err } - return false, c.handle.Put(env.T, raw, h.Seq, env.T == cloudwire.MsgLive && carriesTable(env.B)) + if err := c.handle.Put(env.T, raw, h.Seq, env.T == cloudwire.MsgLive && carriesTable(env.B)); err != nil { + return true, err + } + if env.T == cloudwire.MsgEvent && c.u.OnEvent != nil { + c.u.OnEvent(c.home, raw) + } + return false, nil } // carriesTable reports whether a live body holds its source table. @@ -381,11 +410,23 @@ func (c *boxConn) roster(env cloudwire.Envelope) error { if !c.handle.SetRoster(&r) { return nil // older than what is held: ignored } + if c.u.OnRoster != nil { + c.u.OnRoster(c.home, &r) + } raw, err := env.Encode() if err != nil { return err } - return c.u.registry.SetRoster(c.home, r.RosterSeq, r.Grant.ID, raw, c.u.now()) + if err := c.u.registry.SetRoster(c.home, r.RosterSeq, r.Grant.ID, raw, c.u.now()); err != nil { + return err + } + // A device the roster dropped, or whose epoch moved, is no longer + // suspended: it is gone, or paired again. + epochs := make(map[string]uint64, len(r.Devices)) + for _, d := range r.Devices { + epochs[d.ID] = d.Epoch + } + return c.u.registry.PruneSuspensions(c.home, epochs) } // purge deletes the home and sends the signed receipt. For a home already