diff --git a/.changeset/cloud-tun-cap.md b/.changeset/cloud-tun-cap.md new file mode 100644 index 000000000..4584eca29 --- /dev/null +++ b/.changeset/cloud-tun-cap.md @@ -0,0 +1,5 @@ +--- +"ftw": patch +--- + +A box whose FTW Cloud link carries the phones' sessions says so in `hello_ok.caps` as `cloud.tun`, a new registry capability, so a phone uses that path only when the box takes it. The registry also names the carrier state `tunnel` for a session that FTW Cloud carries. diff --git a/.changeset/cloud-tunnel.md b/.changeset/cloud-tunnel.md new file mode 100644 index 000000000..b6f8ca50f --- /dev/null +++ b/.changeset/cloud-tunnel.md @@ -0,0 +1,5 @@ +--- +"ftw": minor +--- + +A phone can reach its box through FTW Cloud, with the relay unreachable. The box links to `cloud.ftw.energy` when it first shows a QR pairing code or has a paired phone, and carries each phone's own Noise session (`ftw.session.v1`) as `tun {dev, sid, d}` on that link. FTW Cloud cannot read or forge `d`; the box admits a session by the key its handshake proves, exactly as over the relay, and drops one the service put under another device. A new phone can pair through FTW Cloud: while a QR code is shown, the roster lists the code's hash, never the code. The readable copy stays a separate grant: with it off the box sends only its roster and the tunnel, and turning it off now deletes the copy at the service while the link and the home id stay. The link ends, and `cloud.key` with it, when no phone may use it and no code is shown. The tunnel has its own queues and a 250 MB a day budget apart from the readable copy's 20 MB: a command and its answer go first, and bulk is paced under FTW Cloud's limits. Each device now holds one session across the relay and FTW Cloud, in place of four places shared by all, and may open at most 12 through FTW Cloud a minute. `GET /api/cloud` shows `link`, `tunBytesLast24h`, `tunBudgetBytes` and `tunDropped`. For every linked home, FTW Cloud sees a stable home id, the box's address and when traffic goes. The relay keeps working as before. diff --git a/.github/scripts/check-cloudlink-imports.sh b/.github/scripts/check-cloudlink-imports.sh index 0c061cc5f..0616eb0ff 100755 --- a/.github/scripts/check-cloudlink-imports.sh +++ b/.github/scripts/check-cloudlink-imports.sh @@ -1,6 +1,7 @@ #!/usr/bin/env bash -# go/internal/cloudlink streams the box's state to FTW Cloud, and FTW Cloud -# reads and never controls. This check keeps the uplink's build graph honest: +# go/internal/cloudlink links the box to FTW Cloud, which never chooses an +# action: it carries the phones' sessions unread and, when an owner allows +# it, a readable copy. This check keeps the uplink's build graph honest: # it may import go/protocol and nothing else of the box's own, so no control, # API, loadpoint, planner or driver code can be reached from the code that # talks to the service. Its tests are held to the same rule. diff --git a/AGENTS.md b/AGENTS.md index b94d6c7b0..1b2e6938c 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -70,10 +70,13 @@ Read [docs/architecture.md](docs/architecture.md) for the system map and - Database queries stay in [`go/internal/state`](go/internal/state). - Lane 0 frames have a fixed length and cadence; the padding is a privacy control. The proposed FTW Cloud uplink is not padded, so it sends lane 1 - frames without the bulk buckets and never lane 0. -- FTW Cloud may hold a readable copy of what the owner enabled. It never sends - control and never chooses actions. See - [ADR 0008](docs/adr/0008-ftw-cloud.md), proposed. + frames without the bulk buckets and never lane 0. A phone's own session + rides inside it as `tun`, with its lane 0 padding intact. +- FTW Cloud may hold a readable copy of what the owner enabled. It carries the + phones' sessions with the box as ciphertext it cannot read or forge, and it + never chooses actions; the box judges what arrives as it does over the + relay. See [ADR 0008](docs/adr/0008-ftw-cloud.md), proposed, and the + tunnel in [docs/protocol-cloud.md](docs/protocol-cloud.md). ## Drivers diff --git a/VISION.md b/VISION.md index d7b0dfc71..7f2080c5a 100644 --- a/VISION.md +++ b/VISION.md @@ -240,9 +240,13 @@ FTW Cloud is such an endpoint. It is proposed in the way into it, and it may become a paid service. When an owner turns it on, the box sends Sourceful a signed, readable copy of what the owner enabled, so the app starts fast and its alerts, widgets and Live Activities work while it -is closed. FTW Cloud never sends control and never chooses which actions a -user is offered; commands still go from the app to the box over the encrypted -session. Statistics need their own consent, off by default. This turns away +is closed. From the first pairing the box also links to FTW Cloud, so the +phones reach it through one service: their encrypted session with the box +rides through FTW Cloud as ciphertext it cannot read or forge, and the box +judges every command as it does over the relay. For every linked home the +service sees a stable home id, the box's address and when traffic goes, with +the readable copy off too. FTW Cloud never chooses which actions a user is +offered. Statistics need their own consent, off by default. This turns away from [#1323](https://github.com/srcfl/ftw/issues/1323), which asks that no feature need a Sourceful service: the iPhone app needs FTW Cloud, while the box does not. A household that does not want FTW Cloud runs the box itself, diff --git a/contract/registry.yaml b/contract/registry.yaml index 0cfdaecc8..63b4de553 100644 --- a/contract/registry.yaml +++ b/contract/registry.yaml @@ -103,6 +103,10 @@ capabilities: # only and offers no FTW Cloud. It says nothing about whether the owner # turned FTW Cloud on; the box's grant decides that. - cloud.link + # The box takes its app session through FTW Cloud, end to end, as `tun` + # (docs/protocol-cloud.md, Tunnel, in srcfl/ftw). Absent means a phone + # reaches the box through the relay, even when it knows the home id. + - cloud.tun # The box can take a watch: it opens the relay's exchange socket and mints # watch codes. Absent means the phone offers no Add watch. The wire is # docs/watch-exchange.md in srcfl/ftw-webapp. @@ -274,7 +278,9 @@ client_errors: # the two into one enum cannot express "connected, but the inverter is quiet". # --------------------------------------------------------------------------- source_states: [live, lagging, stale, down, never] -carrier_states: [webrtc, relay, cloud, cache, none] +# tunnel is the box's own session carried by FTW Cloud, which cannot read it. +# cloud is the box-signed copy FTW Cloud holds and can read. +carrier_states: [webrtc, relay, tunnel, cloud, cache, none] # --------------------------------------------------------------------------- # History resolutions backed by what the box actually stores. diff --git a/docs/architecture.md b/docs/architecture.md index ca68dc414..f4395e4b5 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -409,9 +409,13 @@ The properties that matter: intact. Proposed, not shipped: [ADR 0008](adr/0008-ftw-cloud.md) adds FTW Cloud, a -second outbound connection for homes whose owner turns it on. It carries to -`cloud.ftw.energy` a signed, readable copy of what the owner enabled, and it -never carries control. Commands and long history stay on the relay path above. +second outbound connection to `cloud.ftw.energy`. A box links when it is first +paired. The link carries each phone's own session with the box as `tun`, +ciphertext the service cannot read or forge, so a phone reaches the box with +the relay unreachable; and, when an owner turns it on, a signed, readable copy +of what the owner enabled. The box admits a session through the tunnel by the +key its handshake proves, exactly as over the relay, and keeps one session per +device across both. The relay path above stays while every client moves. [protocol-cloud.md](protocol-cloud.md) specifies its wire format. [ADR 0009](adr/0009-watch-exchange.md) lets an Apple Watch work without the @@ -419,8 +423,8 @@ phone. watchOS allows no WebSocket, so each step is one HTTPS request. Commands go to the box through the relay's exchange: the box keeps a second socket, `/r///exchange`, under the uplink's handle, and the relay hands it one Noise message per request behind an id of its own. The box holds those -sessions in a table of eight, apart from the phones' four, so a watch never -pushes a phone out; each ends after four exchanges, a minute idle or three +sessions in a table of eight, apart from the phones' sessions, which hold one +slot per device, so a watch never pushes a phone out; each ends after four exchanges, a minute idle or three minutes. Every frame on it is lane 0 apart from `hello_ok`, and every answer is two frames, so a read, a command and a refusal have one length. A command answers no sooner than 400 ms after it arrives, so an applied one and a refused diff --git a/docs/protocol-cloud.md b/docs/protocol-cloud.md index 30050d8ec..5a3341d39 100644 --- a/docs/protocol-cloud.md +++ b/docs/protocol-cloud.md @@ -4,7 +4,8 @@ Proposed with [ADR 0008](adr/0008-ftw-cloud.md). Nothing here has shipped. This document is normative: "must" is a requirement on the box, the service or the phone. The bytes live in `go/protocol/cloudwire`, with golden vectors in its `testdata`: `cloud-vectors.json` for the uplink, -`cloud-app-vectors.json` for one whole app session and +`cloud-app-vectors.json` for one whole app session, its +[tunnel](#tunnel) messages and a pairing proof, and `cloud-once-vectors.json` for one [one-shot read](#one-shot-read). `srcfl/ftw-cloud`, the phone and the watch replay them. `cloudwire.View` is the phone's [freshness](#freshness) rules in Go, and the cases listed there are its tests. @@ -21,7 +22,9 @@ for both live in `go/protocol`, under Apache-2.0 ## Parties and connections -- **Box:** FTW Core with an FTW Cloud grant. +- **Box:** FTW Core linked to FTW Cloud. A box links when it first shows a + pairing code or an owner turns the readable copy on, and stays linked while + any phone may reach it (see [Link](#link)). - **Service:** `ftwcloudd`, at `cloud.ftw.energy`. - **Phone:** the iPhone app, paired with the box over Noise as today. - **Extension:** a widget, notification service extension or Live Activity @@ -33,7 +36,7 @@ for both live in `go/protocol`, under Apache-2.0 | Connection | Opened by | Transport | What protects it | |---|---|---|---| | Uplink | Box | `wss://cloud.ftw.energy/v1/box` | TLS 1.3, a handshake bound to the TLS session on both sides, every box message signed | -| App session | Phone | `wss://cloud.ftw.energy/v1/app?home=` | TLS 1.3, then Noise IK to the service's static key | +| App session | Phone | `wss://cloud.ftw.energy/v1/app?home=` | TLS 1.3, then Noise IK to the service's static key. Inside it, the phone's own Noise session with its box rides as [`tun`](#tunnel) | | One-shot read | Watch | `POST https://cloud.ftw.energy/v1/app/once?home=` | TLS 1.3, then Noise IK to the service's static key, inside one request | | Extension read | Extension | `GET https://cloud.ftw.energy/v1/now` | TLS and an extension token; the data inside is box-signed | | Push | Service | APNs | Box-signed payloads where they carry data | @@ -100,10 +103,11 @@ Envelope = { t: text, b: bstr, sig: bstr } - **Signed input:** the bytes `ftw.cloud.v1\0` (12 ASCII bytes and a zero byte), then the deterministic CBOR encoding of the array `[home, t, b]`, where `home` is the text home id. -- The box signs with `cloud.key`. The service signs `hello_ok` and `purged` - with its service key, over the box's home id. Every other envelope the - service sends, and every envelope between phone and service, has an empty - `sig`. +- The box signs with `cloud.key`, every envelope but `tun`. The service signs + `hello_ok` and `purged` with its service key, over the box's home id. Every + other envelope the service sends, `tun` in both directions, and every + envelope between phone and service has an empty `sig`. A `tun` needs none: + its `d` is Noise ciphertext that the phone and the box authenticate. - A receiver verifies over the bytes it received, never over a re-encoding. - The box's public key reaches phones over Noise, beside the tables of opaque ids, the home id and the box's current `boot`, in the @@ -118,7 +122,7 @@ Envelope = { t: text, b: bstr, sig: bstr } | Name | What it is | |---|---| -| `cloud.key` | ES256 (P-256) key the box makes when an owner grants FTW Cloud. The box never backs it up | +| `cloud.key` | ES256 (P-256) key the box makes when it links (see [Link](#link)). The box never backs it up | | `pk` | Its public key as a 65-byte uncompressed SEC1 point | | `home` | Lowercase base32, no padding, of the first 16 bytes of SHA-256(`ftw.cloud.home.v1` ‖ `pk`). 26 characters | | Service key | P-256. Core carries up to two public halves, so the key can rotate across a release | @@ -191,9 +195,9 @@ with two different values, so neither side's check passes through it. | `boxNonce` | bstr, 32 B | Random, for the service to sign | | `build` | text | Core version | | `clock` | text | `ntp` or `none` | - | `caps` | [text] | The classes this box sends under the grant, `roster` included | + | `caps` | [text] | The classes this box sends under the grant at connect, `roster` included, and `tun` when the box takes the [tunnel](#tunnel) | | `rosterSeq` | uint | The box's current roster number | - | `grantId` | bstr, 16 B | The grant this connection serves | + | `grantId` | bstr, 16 B or empty | The grant at connect. Empty while the readable copy is off or a revoke waits for its receipt | 3. **Service → box: `hello_ok`**, signed by the service key over the box's home id. @@ -205,6 +209,12 @@ with two different values, so neither side's check passes through it. | `have` | map text→text | Tile id to etag, for the tiles the service holds | | `rosterSeq` | uint | The roster number the service holds | + `caps` and `grantId` describe the connection as it starts. The readable + copy can turn on or off while the link stays up, so the service takes + what it may hold from the newest roster's grant, never from `hello`, and + enforces neither per connection; only `tun` in `caps` decides whether it + sends the box `tun`. + The box checks the signature against a service key it carries, that `boxNonce` is its own and that `ekm` equals its own exporter value. If any check fails, the box closes and backs off. **It sends nothing but `hello` @@ -212,8 +222,8 @@ with two different values, so neither side's check passes through it. public key and grant id, and cannot use them. 4. **Box → service:** a `roster`, at every connect (see [Roster](#roster)); then one full message of each class the grant allows; then the tiles - missing from `have`. While a purge is pending, the box sends only `bye` - instead (see [Revocation](#revocation-and-deletion)). + missing from `have`. While the link waits for a purge, the box sends only + `bye` instead (see [Revocation](#revocation-and-deletion)). 5. After that the box sends on change, by the cadence rules. ## Classes @@ -416,10 +426,11 @@ The service never forwards these to a phone. | Field | Type | Meaning | |---|---|---| -| `rosterSeq` | uint | Persisted on the box. Rises by 1 at each pair, role change, revoke or grant change | +| `rosterSeq` | uint | Persisted on the box. Rises by 1 at each pair, role change, revoke, grant change or pairing code shown | | `ttlS` | uint | 86,400. Counted from when the service received the roster | | `devices` | [Device] | Every device paired with the box | | `grant` | Grant | What the owner granted | +| `pending` | [Pending] | The pairing codes the box shows now, at most one | **Device** @@ -430,16 +441,32 @@ The service never forwards these to a phone. | `role` | text | `owner`, `viewer` or `watch` | | `scopes` | [text] | Scopes the box derives from the role. For a watch, only those its phone's role also carries now. The service holds no role table | | `epoch` | uint | The device's epoch on the box | +| `tun` | bool | The device may open a session to the box through the service. True for a phone, false for a watch. It needs no grant | **Grant** | Field | Type | Meaning | |---|---|---| -| `id` | bstr, 16 B | Grant id | -| `scopes` | [text] | A subset of `cloud_scopes` | -| `purposes` | [text] | A subset of `cloud_purposes`: `serve` always, and `stats` while the owner contributes to statistics | +| `id` | bstr, 16 B or empty | Grant id. Empty while the readable copy is off | +| `scopes` | [text] | A subset of `cloud_scopes`. Empty while the copy is off or being withdrawn | +| `purposes` | [text] | A subset of `cloud_purposes`: `serve` while the copy is on, and `stats` while the owner contributes to statistics. Empty while the copy is off or being withdrawn | | `controller` | text | The data controller: `sourceful` unless a partner pairing flow sets another. No such flow exists yet | +A grant has three shapes. With an id and `serve`, the readable copy is on. +With an id and no purposes, the owner has withdrawn that copy: the service +deletes what it holds under the grant and answers `purged {what: "copy"}`, +each time it receives such a roster. With no id, there is no copy. + +**Pending** + +| Field | Type | Meaning | +|---|---|---| +| `hash` | bstr, 32 B | SHA-256(`ftw.cloud.pair.v1\0` ‖ home id ‖ pairing code) | +| `ttlS` | uint | Seconds the code has left, counted from when the service received the roster | + +Only a QR code is listed. A spoken code has 40 bits, and its hash would let +the service find it offline; it pairs over the relay or the home network. + **`stats`**, only while `grant.purposes` contains `stats`. | Field | Type | Meaning | @@ -490,8 +517,9 @@ back `box.unreachable` until the box returns or an hour passes. ## Service to box -The service may send the box only these. Anything else, a frame over 4 KiB, -a text message or a lane 0 frame closes the uplink. A fuzz test covers it. +The service may send the box only these. Anything else, a `tun` frame over +`MaxTunFrame` (16,664 bytes), any other frame over 4 KiB, a text message or +a lane 0 frame closes the uplink. A fuzz test covers it. | `t` | Body | Rule | |---|---|---| @@ -499,10 +527,13 @@ a text message or a lane 0 frame closes the uplink. A fuzz test covers it. | `hello_ok` | See Handshake | Signed by the service key | | `watch` | `{hz: 1}` | Raises the period to 1 s for 60 s. While any phone of the home looks, the service sends it at once, every 50 s after that, and again when the box connects. Without it the box returns to 5 s. The box ignores any other `hz` | | `resync` | `{what, tiles}` | `what` holds `full`, `tiles` or `roster`; `tiles` lists tile ids. The box honours at most one per 5 minutes and ignores the rest | -| `purged` | `{atMs, what, grantId, statsId}` | Signed by the service key. `what` is `home` or `stats`. Sent once every deletion it names is done | +| `purged` | `{atMs, what, grantId, statsId}` | Signed by the service key. `what` is `home`, `copy` or `stats`. Sent once every deletion it names is done | +| `tun` | `{dev, sid, d}` | A phone's own session with the box. See [Tunnel](#tunnel). Only to a box that lists `tun` in `hello.caps` | The box acts on none of these beyond what the table says. It takes no time, -config, update or instruction from the service. +config, update or instruction from the service. What a `tun` carries is the +phone's, encrypted to the box's static key; the box decrypts it and decides +as it decides for the same session over the relay. ## Backpressure and limits @@ -517,6 +548,15 @@ config, update or instruction from the service. `roster`, `stats.withdraw` and `bye` until the count falls. It sends no `hb` then, so data on phones ages to stale rather than looking current. `GET /api/cloud` shows the count and whether the budget is reached. +- **`tun` has its own queues and budget.** `tun` waits for the socket apart + from the readable copy, in two queues. A `tun` carrying a lane 0 frame (a + command, its answer, a tick) goes first and is never held; bulk is paced + to nine tenths of the service's [tunnel limits](#tunnel), so the service + never drops for this box, and waits its turn. The queues hold 64 and 256; + the newest is dropped when one is full. They do not count against the + 20 MB, so a phone at work never stops `hb`. The box counts them against + 250 MB per 24 hours of uptime, and past that sends no `tun` until the count + falls. ## Roster @@ -532,7 +572,8 @@ config, update or instruction from the service. sessions with 4403 and refuses new sessions and token reads until a roster arrives. - **Effective scope** of a device is its `scopes` intersected with - `grant.scopes`. + `grant.scopes`. It is empty while the readable copy is off, and a device + with `tun` still opens a session for its [tunnel](#tunnel). - A device that leaves the roster loses its sessions (4403), tokens and APNs registrations within a second of the service receiving the roster, which needs the uplink to be up. A device whose `epoch`, `role` or effective scope @@ -543,17 +584,25 @@ config, update or instruction from the service. ## App session 1. The phone opens `wss://cloud.ftw.energy/v1/app?home=` over TLS 1.3, - with no `Origin` header, and runs `Noise_IK_25519_ChaChaPoly_SHA256` as + with no `Origin` header (the web app's is `https://app.ftw.energy`, the one + origin the service accepts), and runs `Noise_IK_25519_ChaChaPoly_SHA256` as initiator. The responder's static key is a service Noise key built into the app. The prologue is `ftw.cloud.v1:` followed by the home id, which the phone learned from the box (see [Session information](#session-information)). The query only says which home to look in: the prologue binds it, so a query that names another home fails the handshake. -2. Handshake message 1 carries an unsigned `now` envelope, `{hz}`: 1 while the - phone is looking at the home, 0 while it is not. +2. Handshake message 1 carries an unsigned `now` envelope, `{hz, pair, tun}`: + `hz` is 1 while the phone is looking at the home, 0 while it is not. `pair` + is only for a new phone; see + [Pairing through the service](#pairing-through-the-service). `tun`, `{sid, + d}`, is optional and carries message 1 of the phone's session with its + box. The service forwards it as soon as it admits the phone, so that + session is up one round trip after TLS. 3. The service admits the phone only if the static key that message 1 proved - is in the home's roster, the roster has not run out, and no owner has - suspended the device. Every refusal closes with 4403 before message 2. + is in the home's roster, with `tun` or while the readable copy is on, the + roster has not run out, and no owner has suspended the device; or if + `pair` matches a pending entry. Every refusal closes with 4403 before + message 2. 4. Message 2 carries an unsigned `snapshot` envelope, below. 5. After that the service forwards each new box envelope the device may read, unchanged and in the order the box sent them, one unpadded lane 1 frame per @@ -564,7 +613,7 @@ config, update or instruction from the service. | Field | Type | Meaning | |---|---|---| -| `envelopes` | [Envelope] | Box envelopes in rising `seq`, each the box's own CBOR map, unchanged: for each class the device may read, the newest one the service holds; the newest `live` that carried `sources`; and the newest `hb`. Empty after an hour without the box | +| `envelopes` | [Envelope] | Box envelopes in rising `seq`, each the box's own CBOR map, unchanged: for each class the device may read, the newest one the service holds; the newest `live` that carried `sources`; and the newest `hb`. Empty after an hour without the box, and while the readable copy is off | | `lastSeenMs` | int | Service wall time when it last heard from the box | | `online` | bool | The box's uplink is up now | | `more` | uint | How many envelopes did not fit in message 2. They follow it as ordinary frames, in rising `seq` | @@ -585,6 +634,7 @@ the table. | `t` | Who | Effect | |---|---|---| | `watch` | Any device | `{hz}`: 1 when the phone looks at the home, 0 when it looks away. It holds until the next `watch` or the end of the session | +| `tun` | A device with `tun` | `{sid, d}`. The phone's session with its box. See [Tunnel](#tunnel). The service answers `tun` the same way | | `tiles.get` | Devices with `ftw.history.read` | `{have}`, tile id to etag. The service answers with the `tiles` envelopes the phone lacks | | `token.get` | Any device | Answered with `token {token, expiresInS}`. A new token replaces the device's old one | | `push.register` | Any device | `{apnsToken, environment, kind, activityId, app}`. One registration per APNs token; a Live Activity and its push-to-start each register their own. `app` is `iphone`, the default when absent, or `watch`, and picks the APNs topic | @@ -606,7 +656,7 @@ nothing about which check failed. | Code | Meaning | The phone | |---|---|---| | 4400 | The phone broke the protocol | Retries with backoff | -| 4403 | Refused: no such home, a device not in the roster, a suspended device, a roster that ran out, a grant that is off, a purged home, or a handshake to another service key or home | Before message 2, tries the app's other service key once, at once. Otherwise asks again slowly | +| 4403 | Refused: no such home, a device not in the roster, a suspended device, a roster that ran out, a device that may neither read nor open a tunnel, a purged home, or a handshake to another service key or home | Before message 2, tries the app's other service key once, at once. Otherwise asks again slowly | | 4408 | A newer session from the same device took over | Does not retry | | 4409 | The roster changed what this device may read | Reconnects at once, once; then retries with backoff | | 4429 | The home has as many sessions as the service allows, as at the relay | Retries with backoff | @@ -623,11 +673,14 @@ nothing about which check failed. **Limits.** Per home, 16 sessions. Per device, one session; a newer one replaces the older with 4408. Per phone, 256 envelopes queued, past which the -session closes with 1013. A phone message larger than 4 KiB, a text message, -or more than 20 messages in a burst and then 2 a second closes the session -with 4400. Message 1 must arrive within 10 s. Pings and deadlines are the -uplink's. A restarted service refuses a home's phones with 4403 until that -home's box has connected again (see [Service state](#service-state)). +session closes with 1013. A `tun` message larger than `MaxTunFrame` and its +Noise overhead, any other phone message larger than 4 KiB, a text message, +or more than 20 messages other than `tun` in a burst and then 2 a second +closes the session with 4400. `tun` has limits of its own (see +[Tunnel](#tunnel)). Message 1 must arrive within 10 s. Pings and deadlines +are the uplink's. A restarted service reads the rosters it holds back from +its registry, so it admits a home's phones before the box has connected +again (see [Service state](#service-state)). ### Session information @@ -642,14 +695,126 @@ Noise session, in the app protocol's `hello_ok.cloud`. | `sources` | map text→text | Opaque source id to driver name | | `loadpoints` | map text→text | Opaque loadpoint id to loadpoint id | -- The box sends `cloud` only while the grant is on. It leaves it out while - the grant is off or a revoke waits for its receipt, when it holds no - `cloud.key`, and in floor mode. Without it the phone reads through the - relay only. +- The box sends `cloud` while it is linked. It leaves it out when it holds + no `cloud.key`, while the link waits for a purge, and in floor mode. + Without it the phone reaches the box through the relay only. +- `sources` and `loadpoints` are cut with the grant's `grantKey`, so they are + empty while the readable copy is off. - The tables name the drivers and loadpoints the box knows at `hello`. One added later has no name on the phone until its next session with the box. - The box lists `cloud.link` in `hello_ok.caps` when its build can run the uplink. A build that names no service does not. +- It also lists `cloud.tun` when its link carries the phones' own sessions + as [`tun`](#tunnel). A phone sends its box session through the service + only to a box that said so at its last hello, or to one it has never + reached when a v3 QR named the home. Otherwise it uses the relay. + +## Tunnel + +The phone's own session with its box, `ftw.session.v1` (Noise IK to the +box's static key, as the app protocol specifies), rides inside the app +session. Nothing in it changes: lane 0 keeps its buckets and its cadence, so +the service sees when a message goes, not what it says or how large a command +is. + +### `tun` + +| Field | Type | Meaning | +|---|---|---| +| `dev` | text | The roster device id whose session this is | +| `sid` | uint | The phone's number for one session with the box, below 2³², drawn at random for each new session so that the late end of an older one never ends it | +| `d` | bstr | One message of that session, byte for byte, or empty to end it | + +On the app session the phone and the service send `tun {sid, d}`, with no +`dev`. On the uplink the service and the box send `tun {dev, sid, d}`. + +- **The service names the device.** It sets `dev` from the roster entry its + own handshake with the phone matched, and ignores a `dev` a phone adds. It + never forwards a phone's `tun` to another home or under another device. +- **The box checks it again.** A session starts with handshake message 1 of + the box session under a new `sid`. The box runs the handshake and checks + the phone exactly as it checks one over the relay: a known key, or a live + pairing code in the payload. If the key the handshake proved is not the + device `dev` names, the box drops the message and answers nothing. A `d` + for a `(dev, sid)` the box does not hold that is not message 1 gets an empty + `d` back, so the phone starts again. +- **A session proves itself first.** The service sees every message 1 and + could replay one under a new `sid`. So the box answers a message 1 but + holds the session as pending: it takes no device's place, stamps nothing + and spends no pairing code until its first transport message decrypts. A + replay never gets that far. The same holds over the relay. +- **One session per device.** A session that proves itself, through the + service or the relay, ends the device's older one. +- **Ends.** An empty `d` ends `(dev, sid)`. An end for a session a newer + one replaced looks the same as any other, so a phone ignores an end for + any `sid` but its current one and never restarts on it; if it did, it + would fight its own newer session. An end for its current `sid` means the + box dropped it, and the phone starts again with backoff. The box sends an + end when it drops a session; the service sends one to the box when the phone's app session + ends, and one to the phone when the box's uplink ends or is replaced by a + newer one, when the box does not take `tun`, or when the phone names a + session the box does not hold. + Every tunnel session ends with the uplink that carried it. +- **What the service can do.** Drop or delay a `tun`. It cannot read or forge + `d`, and Noise transport refuses a replay. Every `cmd` carries its own + expiry. +- **Order.** The service forwards a device's `tun` in the order it received + them. +- **Limits.** `d` is at most 16,408 bytes, one transport message around the + largest bulk bucket; a phone that sends more is closed with 4400. Per home + and per direction the service passes 240 messages in a burst and then 60 a + second, and 4 MiB in a burst and then 512 KiB a second, the limits the + relay puts on a box's uplink. Past them it drops `tun` and counts the drop; + at twice them it closes the box's uplink, or the phone's session. `tun` + does not count against the uplink's own message rate. A box handshake + through the tunnel costs the box up to four X25519 operations, so the box + answers at most 12 a minute per device and 240 in all. + +### Pairing through the service + +A new phone is in no roster, so the service would refuse it. While the box +shows a QR code, its roster lists the code's hash in `pending`. The phone +scans the code, computes the same hash and puts it in `now.pair`. The service +admits the handshake once for that hash, removes the entry and names the +device by its device id: the first 8 characters of its static key in +base64url, which is how the box names it too. The device may then send only +`tun`. The box decides as over the relay: the code rides inside handshake +message 1, encrypted to the box, so the service never sees it. When the +roster that lists the new device arrives, the session takes that entry +without closing. If the pairing fails, the session ends when the entry's +`ttlS` runs out. + +**The QR, v3.** A phone that has never reached the box needs the home id +before any session with it. Pairing payload `v3` is `v2` with a sixth segment: + +``` +https://app.ftw.energy/p#v3..... +``` + +`` is base64url, without padding, of the home id's 26 ASCII +characters, like the hint. The other five segments are `v2`'s, byte for byte, +and everything stays in the fragment. The box shows `v2` until one constant, +`appenroll.QRNamesHome`, turns `v3` on. It stays off until the apps parse +`v3` in builds people have: ftw-app's `Enrollment.swift` and `Enrollment.kt` +and ftw-webapp's `enrollment.ts` each take exactly five `v2` segments today, +so an earlier `v3` would stop pairing on every shipped app. Until then a new +phone learns the home id from `hello_ok.cloud` on its first session with the +box, which the box sends from its first pairing on. + +### Link + +- A box links when it first shows a QR pairing code, when a device that may + open a tunnel is paired, or when an owner turns the readable copy on. It + makes `cloud.key` then, and the home id with it. +- A linked box sends its roster and takes `tun`. It sends no readings until + the owner turns the readable copy on. +- The link ends when no paired device may open a tunnel, no code is shown + and the readable copy is off. The box then sends + `bye {why: "revoked", purge: true}` and deletes `cloud.key` on the + `purged {what: "home"}` receipt. The next link has a new home id. +- A build that names no service never links. +- For every linked home the service sees a stable home id, the box's address + and when traffic goes, with the readable copy off too. ## One-shot read @@ -668,8 +833,8 @@ exactly as `/v1/app` admits a phone and keeps nothing after the answer. `push.register` or `push.unregister`. The service refuses any other type, an owner's `suspend` among them, with 400 before message 2. 3. The service admits the watch by the rules of the app session: its key in - the home's roster, a roster that has not run out, no suspension, a grant - that is on. A refusal is 403 with an empty body, before message 2, and does + the home's roster, a roster that has not run out, no suspension, and a + non-empty effective scope, so the readable copy must be on. A refusal is 403 with an empty body, before message 2, and does not say which check failed. 4. The answer is 200 with `content-type: application/octet-stream`. Its body is message 2 and then, for a `read`, the envelopes that did not fit, as @@ -872,6 +1037,8 @@ consent text. a new home id and a new `grantKey`. The old home at the service ages out as under [Silence](#revocation-and-deletion). - A grant works with the relay off. +- A grant uses the link's `cloud.key` and home id, and makes the link when + there is none. Withdrawing it keeps both. | Route | Who may call it | |---|---| @@ -899,14 +1066,21 @@ app's, so the grant screen claims no more than an owner's session. ## Revocation and deletion - **Device revoke** on the box: a roster without the device. See Roster. -- **Grant revoke:** the box sends `bye {why: "revoked", purge: true}`. The +- **Grant revoke:** the box stops sending readings and sends a roster whose + grant keeps its id and has no scopes or purposes. The service deletes what + it holds under that grant (the live copy, events, tiles, extension tokens + and statistics rows) and answers `purged {what: "copy", grantId}`, which it + sends again for every such roster. The box shows the revoke as pending + until a valid receipt arrives, then deletes the grant and shows the receipt + time. The link, the home id and the tunnel stay. +- **Link end:** the box sends `bye {why: "revoked", purge: true}`. The service deletes the home: registry row, tokens, APNs registrations, live copy, events, tiles and statistics rows. It keeps a tombstone of the home id until it has delivered `purged {what: "home"}`, which it sends once all of that is gone, statistics rows included. Until then the box keeps - `cloud.key`, sends only `hello` and `bye` on each connect, and shows the - revoke as pending. With a valid `purged` it deletes the key and the grant - and shows the receipt time. + `cloud.key` and sends only `hello` and `bye` on each connect. With a valid + `purged` it deletes the key. A box of an earlier release still ends its + grant this way. - **Statistics withdrawal:** `stats.withdraw`. The service deletes the rows within 24 hours and sends `purged {what: "stats"}` when done, on the same or a later connection. @@ -925,9 +1099,12 @@ only readings it stores. Its registry holds the home id, the box key, the roster, suspensions, the current and deleted `statsId`s, APNs registrations and token hashes. -A restarted service holds the registry but no readings and no rosters in -memory. It admits a home's phones again once that home's box has connected -and sent its roster. +A restarted service holds the registry but no readings. At start it reads +each home's signed roster back from the registry, checks its signature with +the home's box key, and admits phones by it, counting `ttlS` from when the +roster first arrived. It also starts the wait for `box.unreachable` again for +each home, from when the box was last seen, so a restart neither forgets nor +repeats one. Without a box, a phone's `tun` gets an empty `d` back. A rebuilt host starts with an empty registry. Boxes send their rosters again and phones register again at their next start. Suspensions are lost, so an @@ -948,6 +1125,8 @@ to the app. | `cloud_purposes` | List: `serve`, `stats` | `registry.yaml`, new block | | `cloud.link` | Capability: the box can run the FTW Cloud uplink. Absent means the app uses the relay only and offers no FTW Cloud | `registry.yaml`, `capabilities` | | `cloud` | Carrier state, between `relay` and `cache` | `registry.yaml`, `carrier_states` | +| `cloud.tun` | Capability: the box takes its app session through the service as `tun`. Absent means the phone uses the relay, even when it knows the home id | `registry.yaml`, `capabilities` | +| `tunnel` | Carrier state, between `relay` and `cloud`: the box's own session, carried by the service | `registry.yaml`, `carrier_states` | | `cloud.granted`, `cloud.revoked` | Push kinds, sent to every owner when FTW Cloud is turned on or off | `push-catalogue.yaml` | Class names, field names, close codes and the signing tags are not registry @@ -978,8 +1157,13 @@ the registry's roles would remove that copy. | `resync` honoured | At most one per 5 minutes | | Event ring on the box | 32 | | Byte budget | 20 MB per 24 hours of uptime | -| Largest frame from the service to the box | 4 KiB | -| Largest message from a phone | 4 KiB | +| Largest frame from the service to the box | 4 KiB; a `tun` frame 16,664 bytes | +| Largest message from a phone | 4 KiB; a `tun` message 16,664 bytes and Noise's 24 | +| `tun` per home and direction | 240 messages, then 60 a second; 4 MiB, then 512 KiB a second | +| `tun` queues and budget on the box | 64 lane 0, 256 bulk paced to 90 % of the limits; 250 MB per 24 hours of uptime | +| Box handshakes through the tunnel | 12 a minute per device, 240 in all | +| Pending pairing entry | One QR code, for its own ten minutes; admits one handshake | +| Box session up after TLS | One round trip: its message 1 rides in `now.tun` | | Sessions | 16 per home; one per device | | Envelopes queued per phone | 256 | | Phone messages | 20 in a burst, then 2 a second | diff --git a/go/cmd/ftw/cloud_link.go b/go/cmd/ftw/cloud_link.go index 4731c787a..5d475265d 100644 --- a/go/cmd/ftw/cloud_link.go +++ b/go/cmd/ftw/cloud_link.go @@ -16,6 +16,7 @@ import ( "github.com/srcfl/ftw/go/internal/api" "github.com/srcfl/ftw/go/internal/appenroll" "github.com/srcfl/ftw/go/internal/appproto" + "github.com/srcfl/ftw/go/internal/appuplink" "github.com/srcfl/ftw/go/internal/cloudlink" "github.com/srcfl/ftw/go/internal/control" "github.com/srcfl/ftw/go/internal/loadpoint" @@ -29,11 +30,13 @@ import ( // FTW Cloud's uplink, wired to the box. // -// Everything here reads. cloudlink may import nothing from the box, so these -// adapters answer its questions from the same sources the app session and the -// HTTP API use: the live snapshot, the loadpoint view GET /api/loadpoints -// composes, and the price store. None of them is handed anything that can -// change state. +// Everything here reads, but for the tunnel. cloudlink may import nothing +// from the box, so these adapters answer its questions from the same sources +// the app session and the HTTP API use: the live snapshot, the loadpoint view +// GET /api/loadpoints composes, and the price store. None of them is handed +// anything that can change state. The tunnel is the app uplink's session +// table, which cloudlink reaches through its Tunnel port and hands the +// phones' Noise sessions unread. // cloudSite is the box's state as the uplink wants it. type cloudSite struct { @@ -273,21 +276,6 @@ func roundW(v float64) int64 { func ptr[T any](v T) *T { return &v } -// cloudRoster lists paired devices with the scopes the box derives for each: -// a phone's role, or for a watch what its role and its phone's role share. -type cloudRoster struct { - enroll *appenroll.Identity -} - -func (r cloudRoster) Devices() []cloudlink.RosterDevice { - var out []cloudlink.RosterDevice - for _, e := range r.enroll.Roster() { - out = append(out, cloudlink.RosterDevice{ID: e.ID, Key: e.Key, Role: e.Role, Epoch: e.Epoch, - Scopes: appproto.GrantScopes(e.Role, e.ParentRole)}) - } - return out -} - // cloudNotices reaches the notifications service once it exists; it is built // after the link starts. type cloudNotices struct { @@ -300,14 +288,18 @@ func (n *cloudNotices) notify(kind string) { } } -// startCloudLink loads the FTW Cloud grant and keeps the uplink up while an -// owner has it on. It needs the state store, which holds the grant, and +// startCloudLink loads the FTW Cloud link and grant and keeps the uplink up +// while the box is linked: from its first pairing code, or an owner's grant. +// It needs the state store, which holds the link and the grant, and // enrolment, which holds the phones the roster lists; without either it -// returns nil and the routes say FTW Cloud is unavailable. +// returns nil and the routes say FTW Cloud is unavailable. tunnel is the app +// uplink's session table, nil when the app link is off, and then the box +// takes no tun and links only for a grant. func startCloudLink( ctx context.Context, statePath, build string, st *state.Store, enroll *appenroll.Identity, tel *telemetry.Store, ctrl *control.State, ctrlMu *sync.Mutex, revision *control.Revision, siteMeterStale time.Duration, priceSvc *prices.Service, gateway *lateAPI, notices *cloudNotices, + tunnel *appuplink.Tunnel, ) *cloudlink.Link { if st == nil || enroll == nil { return nil @@ -319,21 +311,32 @@ func startCloudLink( if priceSvc != nil && priceSvc.Zone != "" && priceSvc.Store != nil { site.prices = &appPrices{svc: priceSvc} } - link, err := cloudlink.New(cloudlink.Options{ + opts := cloudlink.Options{ Site: site, - Roster: cloudRoster{enroll: enroll}, + Roster: appuplink.CloudRoster{Enroll: enroll, Tunnel: tunnel != nil}, KV: st, KeyPath: filepath.Join(filepath.Dir(statePath), "cloud.key"), Build: build, Started: processStarted, ClockSynced: cloudlink.ClockSynced, Notify: notices.notify, - }) + } + // Set only with a table: a typed nil would read as a tunnel. + if tunnel != nil { + opts.Tunnel = tunnel + } + link, err := cloudlink.New(opts) if err != nil { slog.Warn("FTW Cloud unavailable", "err", err) return nil } + if tunnel != nil { + tunnel.Attach(link) + } enroll.OnChange(link.RosterChanged) + // A code shown links the box at once, so a QR that names the home can. + enroll.OnPairingCode(link.RosterChanged) + enroll.SetHome(link.Home) go func() { if err := link.Run(ctx); err != nil && !errors.Is(err, context.Canceled) { slog.Warn("FTW Cloud uplink stopped", "err", err) @@ -361,10 +364,16 @@ func (c *cloudSession) load() *cloudlink.Link { } // caps adds cloud.link to an app session's capabilities when this build can -// run the uplink. A build that names no service says nothing. +// run the uplink, and cloud.tun when the link also carries the phones' own +// sessions. A build that names no service says nothing. func (c *cloudSession) caps(caps []string) []string { - if link := c.load(); link != nil && link.Available() { - return append(slices.Clip(caps), appproto.CapCloudLink) + link := c.load() + if link == nil || !link.Available() { + return caps + } + caps = append(slices.Clip(caps), appproto.CapCloudLink) + if link.TakesTun() { + caps = append(caps, appproto.CapCloudTun) } return caps } diff --git a/go/cmd/ftw/cloud_link_test.go b/go/cmd/ftw/cloud_link_test.go index f9715d264..08aa2459b 100644 --- a/go/cmd/ftw/cloud_link_test.go +++ b/go/cmd/ftw/cloud_link_test.go @@ -104,6 +104,8 @@ type noPhones struct{} func (noPhones) Devices() []cloudlink.RosterDevice { return nil } +func (noPhones) Pending() ([]byte, time.Time, bool) { return nil, time.Time{}, false } + func newTestCloudLink(t *testing.T, keys []*ecdsa.PublicKey) *cloudlink.Link { t.Helper() st, err := state.Open(filepath.Join(t.TempDir(), "state.db")) @@ -162,8 +164,53 @@ func TestAppSessionsHearOfFTWCloudOnlyWhenItCanRun(t *testing.T) { t.Fatalf("source %q named %q", id, name) } } + if slices.Contains(caps, appproto.CapCloudTun) { + t.Fatalf("cloud.tun from a link that takes no tun: %v", caps) + } +} + +// A phone hears cloud.tun only from a box whose link carries its session, so +// a phone that knows the home id never waits on a tunnel the box cannot take. +func TestAppSessionsHearOfTheTunnelOnlyWhenTheLinkTakesIt(t *testing.T) { + key, err := ecdsa.GenerateKey(elliptic.P256(), rand.Reader) + if err != nil { + t.Fatal(err) + } + keys := []*ecdsa.PublicKey{&key.PublicKey} + open := func(tunnel cloudlink.Tunnel, keys []*ecdsa.PublicKey) []string { + st, err := state.Open(filepath.Join(t.TempDir(), "state.db")) + if err != nil { + t.Fatal(err) + } + t.Cleanup(func() { st.Close() }) + link, err := cloudlink.New(cloudlink.Options{ + Site: quietCloudSite{}, Roster: noPhones{}, KV: st, Tunnel: tunnel, + KeyPath: filepath.Join(t.TempDir(), "cloud.key"), ServiceKeys: keys, + Logger: slog.New(slog.NewTextHandler(io.Discard, nil)), + }) + if err != nil { + t.Fatal(err) + } + c := &cloudSession{} + c.bind(link) + return c.caps(appproto.DefaultCaps()) + } + if caps := open(nil, keys); slices.Contains(caps, appproto.CapCloudTun) { + t.Fatalf("no tunnel: %v", caps) + } + if caps := open(nopTunnel{}, []*ecdsa.PublicKey{}); slices.Contains(caps, appproto.CapCloudTun) || slices.Contains(caps, appproto.CapCloudLink) { + t.Fatalf("a build that names no service: %v", caps) + } + if caps := open(nopTunnel{}, keys); !slices.Contains(caps, appproto.CapCloudTun) || !slices.Contains(caps, appproto.CapCloudLink) { + t.Fatalf("a link with a tunnel: %v", caps) + } } +type nopTunnel struct{} + +func (nopTunnel) Deliver(cloudwire.Tun) {} +func (nopTunnel) Drop() {} + // FTW Cloud's live body is the same field set the app session streams, and // each phase follows the site meter's freshness there too. The fuse is // config, so it names no driver. diff --git a/go/cmd/ftw/cloud_roster_test.go b/go/cmd/ftw/cloud_roster_test.go index 000d3c5a0..6f6d9ab78 100644 --- a/go/cmd/ftw/cloud_roster_test.go +++ b/go/cmd/ftw/cloud_roster_test.go @@ -8,6 +8,7 @@ import ( "github.com/srcfl/ftw/go/internal/apiauth" "github.com/srcfl/ftw/go/internal/appenroll" + "github.com/srcfl/ftw/go/internal/appuplink" ) func rosterKey(t *testing.T) []byte { @@ -32,7 +33,7 @@ func rosterPhone(t *testing.T, id *appenroll.Identity, role string) appenroll.Gr return grant } -func rosterScopes(t *testing.T, roster cloudRoster, deviceID string) []string { +func rosterScopes(t *testing.T, roster appuplink.CloudRoster, deviceID string) []string { t.Helper() for _, d := range roster.Devices() { if d.ID == deviceID { @@ -60,7 +61,7 @@ func TestTheRosterNarrowsAWatchToWhatItsPhoneHolds(t *testing.T) { if err != nil { t.Fatal(err) } - roster := cloudRoster{enroll: id} + roster := appuplink.CloudRoster{Enroll: id} if got := rosterScopes(t, roster, watch.DeviceID); !slices.Equal(got, apiauth.RoleScopes[apiauth.RoleWatch]) { t.Fatalf("under an owner the roster gives the watch %v", got) diff --git a/go/cmd/ftw/main.go b/go/cmd/ftw/main.go index da5997e12..d8994b0ec 100644 --- a/go/cmd/ftw/main.go +++ b/go/cmd/ftw/main.go @@ -2441,11 +2441,17 @@ func main() { slog.Info("app link disabled — set app_link.enabled to turn it on") } - // FTW Cloud: a signed, readable copy of the house for the owner's phones, - // off until an owner turns it on. It reads; it never controls. + // FTW Cloud: the phones' own sessions with the box, carried as + // ciphertext the service cannot read, from the first pairing on; and a + // signed, readable copy of the house, off until an owner turns it on. + // The relay keeps running beside it. + var appTunnel *appuplink.Tunnel + if appUplink != nil { + appTunnel = appUplink.Tunnel() + } cloudNotify := &cloudNotices{} cloudLink := startCloudLink(ctx, statePath, Version, st, appEnroll, - tel, ctrl, ctrlMu, controlRev, appLinkWatchdog, priceSvc, appAPI, cloudNotify) + tel, ctrl, ctrlMu, controlRev, appLinkWatchdog, priceSvc, appAPI, cloudNotify, appTunnel) appCloud.bind(cloudLink) // The anonymous fleet ping: how many boxes, which versions, which drivers. diff --git a/go/internal/api/api_cloud.go b/go/internal/api/api_cloud.go index e73e2e016..539dd0129 100644 --- a/go/internal/api/api_cloud.go +++ b/go/internal/api/api_cloud.go @@ -136,7 +136,7 @@ func (s *Server) handleCloudGrant(w http.ResponseWriter, r *http.Request) { writeJSON(w, http.StatusOK, link.Status()) case errors.Is(err, cloudlink.ErrGrantRequest): writeAppLinkError(w, http.StatusBadRequest, err.Error()) - case errors.Is(err, cloudlink.ErrPurgePending): + case errors.Is(err, cloudlink.ErrPurgePending), errors.Is(err, cloudlink.ErrLinkEnding): writeAppLinkError(w, http.StatusConflict, err.Error()) case errors.Is(err, cloudlink.ErrNoService): writeAppLinkError(w, http.StatusServiceUnavailable, err.Error()) diff --git a/go/internal/api/api_cloud_test.go b/go/internal/api/api_cloud_test.go index 58be0f419..3245c73d6 100644 --- a/go/internal/api/api_cloud_test.go +++ b/go/internal/api/api_cloud_test.go @@ -13,6 +13,7 @@ import ( "strings" "sync" "testing" + "time" "github.com/srcfl/ftw/go/internal/apiauth" "github.com/srcfl/ftw/go/internal/cloudlink" @@ -164,6 +165,7 @@ func TestCloudGrantRefusalsKeepTheirMeaning(t *testing.T) { for err, want := range map[error]int{ cloudlink.ErrGrantRequest: http.StatusBadRequest, cloudlink.ErrPurgePending: http.StatusConflict, + cloudlink.ErrLinkEnding: http.StatusConflict, cloudlink.ErrNoService: http.StatusServiceUnavailable, } { s := New(&Deps{Cloud: &stubCloud{err: err}}) @@ -363,6 +365,8 @@ type cloudRosterStub struct{} func (cloudRosterStub) Devices() []cloudlink.RosterDevice { return nil } +func (cloudRosterStub) Pending() ([]byte, time.Time, bool) { return nil, time.Time{}, false } + type cloudKV struct { mu sync.Mutex m map[string]string diff --git a/go/internal/appenroll/enroll.go b/go/internal/appenroll/enroll.go index a97ec97c9..46d55ddb8 100644 --- a/go/internal/appenroll/enroll.go +++ b/go/internal/appenroll/enroll.go @@ -212,6 +212,12 @@ type Identity struct { // onChange hears about every change to who is paired and what they // may do, after it is saved. Not about lastSeen stamps. onChange []func() + // onPairingCode hears about a phone's code minted, which puts a new + // code on the screen or takes one off it. + onPairingCode []func() + // home is the home id at FTW Cloud, for a QR that names it. See + // SetHome. + home func() string } // appMeta is what the box remembers about one paired phone. @@ -496,6 +502,7 @@ func (i *Identity) MintPairingCode(role string, ttl time.Duration) ([]byte, time i.mu.Lock() i.pairing = &pairingCode{code: code, expiresAt: expires, role: role, kind: PairingKindQR} i.mu.Unlock() + i.minted() return append([]byte(nil), code...), expires, nil } @@ -522,10 +529,44 @@ func (i *Identity) MintSpokenCode(role string) (string, time.Time, error) { i.mu.Lock() i.pairing = &pairingCode{code: code, expiresAt: expires, role: role, kind: PairingKindSpoken} i.mu.Unlock() + i.minted() return GroupSpokenCode(encodeSpoken(code)), expires, nil } +// PendingCode is the QR code a phone may pair with now, for FTW Cloud's +// roster to list by its hash, so the service admits that phone's first +// session. A spoken code is never one: forty bits behind a hash are forty +// bits the service could find. Nor is a watch code, an expired code or a +// spent one. +func (i *Identity) PendingCode() ([]byte, time.Time, bool) { + i.mu.Lock() + defer i.mu.Unlock() + p := i.pairing + if p == nil || p.kind != PairingKindQR || !i.now().Before(p.expiresAt) { + return nil, time.Time{}, false + } + return append([]byte(nil), p.code...), p.expiresAt, true +} + +// OnPairingCode registers fn to run after a phone's code is minted, which +// shows a new one and takes any older one off the screen. A code spent is +// a pairing, and OnChange hears about it. +func (i *Identity) OnPairingCode(fn func()) { + i.mu.Lock() + defer i.mu.Unlock() + i.onPairingCode = append(i.onPairingCode, fn) +} + +func (i *Identity) minted() { + i.mu.Lock() + fns := append([]func(){}, i.onPairingCode...) + i.mu.Unlock() + for _, fn := range fns { + fn() + } +} + // knownRole refuses a role the registry does not define. // // Checked at every door into the stored state rather than trusted from the @@ -566,38 +607,62 @@ func phoneRole(role string) error { // authenticated; until now it threw that away and answered only yes or no, // which is why nothing downstream could tell two phones apart. func (i *Identity) Authorise(appStatic, payload []byte) (Grant, error) { + _, commit, err := i.Check(appStatic, payload) + if err != nil { + return Grant{}, err + } + return commit() +} + +// Check is Authorise in two steps: it decides now, and changes nothing but a +// wrong guess's count until commit runs. A carrier commits once the session +// has proved itself with its first transport message, so a handshake message +// replayed by whoever carries it costs a reconnect stamp nothing, spends no +// code and takes no device's place. commit returns the grant as it stands +// then, and fails if a revoke, another pairing or the code's end came first. +func (i *Identity) Check(appStatic, payload []byte) (Grant, func() (Grant, error), error) { key := base64.RawURLEncoding.EncodeToString(appStatic) i.mu.Lock() if meta := i.authorised[key]; meta != nil { if meta.role == apiauth.RoleWatch { // A watch has no stream. It reaches the box through the - // exchange, which never takes one of a phone's four places, and + // exchange, which never takes a phone's place, and // it answers here as a stranger would. i.mu.Unlock() - return Grant{}, ErrNoPairing + return Grant{}, nil, ErrNoPairing } + grant := i.grantLocked(key, meta) + i.mu.Unlock() // A reconnect. The stamp is what lets the settings page tell a // phone in daily use from a key that paired once and vanished — // which is exactly the row someone wants to revoke. - meta.lastSeenMs = i.now().UnixMilli() - grant := i.grantLocked(key, meta) - i.mu.Unlock() - // A stamp that could not be persisted must not block a session. - _ = i.save() - return grant, nil + return grant, func() (Grant, error) { + i.mu.Lock() + meta := i.authorised[key] + if meta == nil { + i.mu.Unlock() + return Grant{}, ErrNoPairing + } + meta.lastSeenMs = i.now().UnixMilli() + grant := i.grantLocked(key, meta) + i.mu.Unlock() + // A stamp that could not be persisted must not block a session. + _ = i.save() + return grant, nil + }, nil } // The rest happens under the lock, because counting a wrong guess is a // read and a write of the same code and two handshakes racing must not // each see four attempts and let a fifth and a sixth through. if len(payload) == 0 { i.mu.Unlock() - return Grant{}, ErrNoPairing + return Grant{}, nil, ErrNoPairing } pairing := i.pairing if pairing == nil || !i.now().Before(pairing.expiresAt) { i.mu.Unlock() - return Grant{}, ErrBadPairing + return Grant{}, nil, ErrBadPairing } // Constant time, because a byte-at-a-time comparison against a live code // is exactly the oracle an attacker with a relay socket would want. @@ -622,13 +687,43 @@ func (i *Identity) Authorise(appStatic, payload []byte) (Grant, error) { i.pairing = nil } i.mu.Unlock() - return Grant{}, ErrBadPairing + return Grant{}, nil, ErrBadPairing } + grant := Grant{DeviceID: deviceID(key), Role: i.pairingRoleLocked(pairing), Epoch: firstEpoch} + i.mu.Unlock() - now := i.now().UnixMilli() - // The role travels with the code rather than with the handshake, so a - // code minted for a viewer stamps a viewer here — and nothing the app - // sends can change it. + return grant, func() (Grant, error) { + i.mu.Lock() + if meta := i.authorised[key]; meta != nil { + // The same phone's other session committed first. + grant := i.grantLocked(key, meta) + i.mu.Unlock() + return grant, nil + } + // The code must still be the one this handshake carried: not spent + // by another phone, replaced or out of time. + if i.pairing != pairing || !i.now().Before(pairing.expiresAt) { + i.mu.Unlock() + return Grant{}, ErrBadPairing + } + now := i.now().UnixMilli() + role := i.pairingRoleLocked(pairing) + i.authorised[key] = &appMeta{addedAtMs: now, lastSeenMs: now, role: role, epoch: firstEpoch} + // Spent. A second phone offering the same photographed code now meets + // ErrBadPairing rather than a pairing. + i.pairing = nil + i.mu.Unlock() + + err := i.save() + i.changed() + return Grant{DeviceID: deviceID(key), Role: role, Epoch: firstEpoch}, err + }, nil +} + +// pairingRoleLocked is the role a code stamps. The role travels with the code +// rather than with the handshake, so a code minted for a viewer stamps a +// viewer, and nothing the app sends can change it. The caller holds i.mu. +func (i *Identity) pairingRoleLocked(pairing *pairingCode) string { role := pairing.role if role == "" { role = apiauth.RoleOwner @@ -641,15 +736,7 @@ func (i *Identity) Authorise(appStatic, payload []byte) (Grant, error) { // phone would have built exactly that. role = apiauth.RoleOwner } - i.authorised[key] = &appMeta{addedAtMs: now, lastSeenMs: now, role: role, epoch: firstEpoch} - // Spent. A second phone offering the same photographed code now meets - // ErrBadPairing rather than a pairing. - i.pairing = nil - i.mu.Unlock() - - err := i.save() - i.changed() - return Grant{DeviceID: deviceID(key), Role: role, Epoch: firstEpoch}, err + return role } // hasOwnerLocked reports whether any enrolment can still administer this box. diff --git a/go/internal/appenroll/payload.go b/go/internal/appenroll/payload.go index 5429ddc3d..e44247eb3 100644 --- a/go/internal/appenroll/payload.go +++ b/go/internal/appenroll/payload.go @@ -4,11 +4,17 @@ import ( "encoding/base64" "fmt" "strings" + + "github.com/srcfl/ftw/go/protocol/cloudwire" ) // The enrollment payload, printed as a QR code. // // https://app.ftw.energy/p#v2.... +// https://app.ftw.energy/p#v3..... +// +// v3 adds the home id at FTW Cloud. The box shows v2 until QRNamesHome says +// otherwise. // // Everything after the '#' is a URL fragment, and a fragment is never sent in // an HTTP request. That is the whole design: the app's trust anchor reaches it @@ -25,6 +31,12 @@ const ( // payload rather than parse three quarters of it. PayloadVersion = "v2" + // PayloadVersionHome is v2 with a sixth segment: the home id at FTW + // Cloud, base64url of its 26 ASCII characters, so a new phone knows + // where to dial before any session with the box. No shipped app reads + // it; each parser takes exactly five segments of v2. + PayloadVersionHome = "v3" + // PayloadOrigin is where the app is served. It is also the passkey RP // ID, which is why it is one constant and not two. PayloadOrigin = "https://app.ftw.energy" @@ -35,12 +47,33 @@ const ( MaxLANHintChars = 64 ) +// QRNamesHome is the one switch that puts the home id in the QR, as v3. It +// stays false until the apps parse v3 and those builds are out: ftw-app's +// Enrollment.swift and Enrollment.kt, and ftw-webapp's enrollment.ts. A v3 +// QR before then stops pairing on every shipped app. Until it flips, a new +// phone learns the home id from hello_ok.cloud on its first session. +const QRNamesHome = false + +// SetHome tells enrolment where to find the home id at FTW Cloud, for a QR +// that names it. fn returns "" while the box is not linked. +func (i *Identity) SetHome(fn func() string) { + i.mu.Lock() + defer i.mu.Unlock() + i.home = fn +} + // EnrollmentURL builds the payload for the QR code. // // lanHint is where the box believes it can be reached on the local network, // and is allowed to be empty: a box that does not know its own address should // say nothing rather than guess, and the app falls back to the relay. func (i *Identity) EnrollmentURL(code []byte, lanHint string) (string, error) { + return i.enrollmentURL(code, lanHint, QRNamesHome) +} + +// enrollmentURL builds v2, or v3 with the home id when withHome is set and +// the box is linked. +func (i *Identity) enrollmentURL(code []byte, lanHint string, withHome bool) (string, error) { if len(code) != PairingCodeBytes { return "", fmt.Errorf("appenroll: pairing code is %d bytes, need %d", len(code), PairingCodeBytes) } @@ -56,13 +89,26 @@ func (i *Identity) EnrollmentURL(code []byte, lanHint string) (string, error) { } enc := base64.RawURLEncoding - fragment := strings.Join([]string{ + segments := []string{ PayloadVersion, enc.EncodeToString(i.StaticKey().Public()), enc.EncodeToString(code), enc.EncodeToString([]byte(lanHint)), enc.EncodeToString(i.RendezvousSecret()), - }, ".") + } + if home := i.homeID(); withHome && cloudwire.ValidHome(home) { + segments[0] = PayloadVersionHome + segments = append(segments, enc.EncodeToString([]byte(home))) + } + return PayloadOrigin + PayloadPath + "#" + strings.Join(segments, "."), nil +} - return PayloadOrigin + PayloadPath + "#" + fragment, nil +func (i *Identity) homeID() string { + i.mu.Lock() + fn := i.home + i.mu.Unlock() + if fn == nil { + return "" + } + return fn() } diff --git a/go/internal/appenroll/payload_home_test.go b/go/internal/appenroll/payload_home_test.go new file mode 100644 index 000000000..19b5d3e8f --- /dev/null +++ b/go/internal/appenroll/payload_home_test.go @@ -0,0 +1,84 @@ +package appenroll + +import ( + "bytes" + "encoding/base64" + "strings" + "testing" + + "github.com/srcfl/ftw/go/internal/apiauth" +) + +const testHome = "fjzyr67buzpyncqmjliwgezvde" + +// v3 is v2 with the home id as a sixth segment, base64url like the hint. +func TestAV3PayloadNamesTheHome(t *testing.T) { + id, _ := newIdentity(t) + id.SetHome(func() string { return testHome }) + code, _, err := id.MintPairingCode(apiauth.RoleOwner, PairingTTL) + if err != nil { + t.Fatal(err) + } + url, err := id.enrollmentURL(code, "192.168.1.42:8443", true) + if err != nil { + t.Fatal(err) + } + before, fragment, _ := strings.Cut(url, "#") + if strings.Contains(before, testHome) || strings.Contains(before, "?") { + t.Fatalf("the home id is outside the fragment: %q", url) + } + parts := strings.Split(fragment, ".") + if len(parts) != 6 || parts[0] != PayloadVersionHome { + t.Fatalf("segments %q", parts) + } + home, err := base64.RawURLEncoding.DecodeString(parts[5]) + if err != nil || string(home) != testHome { + t.Fatalf("segment 5 is %q: %v", home, err) + } + // The first five are v2's, byte for byte. + v2, _ := id.enrollmentURL(code, "192.168.1.42:8443", false) + _, v2fragment, _ := strings.Cut(v2, "#") + if strings.Join(parts[1:5], ".") != strings.Join(strings.Split(v2fragment, ".")[1:], ".") { + t.Fatal("v3 changed one of v2's segments") + } + if !bytes.Equal(mustDecode(t, parts[2]), code) { + t.Fatal("segment 2 is not the code") + } +} + +// A box that is not linked has no home to name, so it shows v2. +func TestAnUnlinkedBoxShowsV2(t *testing.T) { + id, _ := newIdentity(t) + code, _, _ := id.MintPairingCode(apiauth.RoleOwner, PairingTTL) + for _, home := range []func() string{nil, func() string { return "" }, func() string { return "not a home" }} { + id.SetHome(home) + url, err := id.enrollmentURL(code, "", true) + if err != nil { + t.Fatal(err) + } + if _, fragment, _ := strings.Cut(url, "#"); !strings.HasPrefix(fragment, PayloadVersion+".") || + len(strings.Split(fragment, ".")) != 5 { + t.Fatalf("payload %q", url) + } + } +} + +// The QR stays v2 until every shipped app parses v3: ftw-app's +// Enrollment.swift and Enrollment.kt and ftw-webapp's enrollment.ts each +// take exactly five segments today. Flipping QRNamesHome is step 2's, and +// this test is there to be changed with it. +func TestTheQRStaysV2UntilTheAppsParseV3(t *testing.T) { + if QRNamesHome { + t.Fatal("QRNamesHome is on; check that ftw-app (iOS and Android) and ftw-webapp parse v3 in builds people have, then change this test") + } + id, _ := newIdentity(t) + id.SetHome(func() string { return testHome }) + code, _, _ := id.MintPairingCode(apiauth.RoleOwner, PairingTTL) + url, err := id.EnrollmentURL(code, "") + if err != nil { + t.Fatal(err) + } + if strings.Contains(url, base64.RawURLEncoding.EncodeToString([]byte(testHome))) || !strings.Contains(url, "#"+PayloadVersion+".") { + t.Fatalf("the QR names the home before the apps can read it: %q", url) + } +} diff --git a/go/internal/appenroll/pending_test.go b/go/internal/appenroll/pending_test.go new file mode 100644 index 000000000..fb0f065e2 --- /dev/null +++ b/go/internal/appenroll/pending_test.go @@ -0,0 +1,79 @@ +package appenroll + +import ( + "bytes" + "encoding/base64" + "testing" + "time" + + "github.com/srcfl/ftw/go/internal/apiauth" + "github.com/srcfl/ftw/go/protocol/cloudwire" +) + +// The roster lists the QR code on the screen and nothing else: not a spoken +// code, not a watch code, not one spent or expired. +func TestPendingIsTheLiveQRCodeOnly(t *testing.T) { + id, _ := newIdentity(t) + now := time.Now() + id.now = func() time.Time { return now } + if _, _, ok := id.PendingCode(); ok { + t.Fatal("a code is pending before any was minted") + } + + code, expires, err := id.MintPairingCode(apiauth.RoleOwner, PairingTTL) + if err != nil { + t.Fatal(err) + } + got, until, ok := id.PendingCode() + if !ok || !bytes.Equal(got, code) || !until.Equal(expires) { + t.Fatalf("pending %x until %v, minted %x until %v", got, until, code, expires) + } + + now = expires + if _, _, ok := id.PendingCode(); ok { + t.Fatal("an expired code is still pending") + } + now = time.Now() + + if _, _, err := id.MintSpokenCode(apiauth.RoleOwner); err != nil { + t.Fatal(err) + } + if _, _, ok := id.PendingCode(); ok { + t.Fatal("a spoken code is listed") + } + + code, _, _ = id.MintPairingCode(apiauth.RoleOwner, PairingTTL) + if _, err := id.Authorise(appKey(t), code); err != nil { + t.Fatal(err) + } + if _, _, ok := id.PendingCode(); ok { + t.Fatal("a spent code is still pending") + } +} + +// Minting a phone's code is heard about, so the roster lists it while it is +// on the screen. It is not a roster change in itself. +func TestMintingIsHeardApartFromChanges(t *testing.T) { + id, _ := newIdentity(t) + minted, changes := 0, 0 + id.OnPairingCode(func() { minted++ }) + id.OnChange(func() { changes++ }) + if _, _, err := id.MintPairingCode(apiauth.RoleOwner, PairingTTL); err != nil { + t.Fatal(err) + } + if _, _, err := id.MintSpokenCode(apiauth.RoleOwner); err != nil { + t.Fatal(err) + } + if minted != 2 || changes != 0 { + t.Fatalf("%d mints heard, %d changes", minted, changes) + } +} + +// FTW Cloud names a device as the box does, so the device id the service +// stamps on a tun is the one the box admits it under. +func TestTheServiceNamesDevicesAsTheBoxDoes(t *testing.T) { + key := appKey(t) + if got, want := cloudwire.DeviceID(key), deviceID(base64.RawURLEncoding.EncodeToString(key)); got != want { + t.Fatalf("cloudwire says %q, the box %q", got, want) + } +} diff --git a/go/internal/appproto/command.go b/go/internal/appproto/command.go index d9a090a9f..eeca2f2b9 100644 --- a/go/internal/appproto/command.go +++ b/go/internal/appproto/command.go @@ -164,6 +164,14 @@ func (l *CmdLog) capRoot(fresh cmdKey) { } } +// forget drops a command id the box took but never acted on, so a retry +// acts. +func (l *CmdLog) forget(root, cmdID string) { + l.mu.Lock() + defer l.mu.Unlock() + delete(l.entries, cmdKey{root, cmdID}) +} + // settle attaches the outcome to a remembered command. func (l *CmdLog) settle(root, cmdID string, res CmdResult) { l.mu.Lock() diff --git a/go/internal/appproto/command_test.go b/go/internal/appproto/command_test.go index 3e17d8e2a..fe2a180bf 100644 --- a/go/internal/appproto/command_test.go +++ b/go/internal/appproto/command_test.go @@ -629,3 +629,32 @@ func TestARevokedDevicesCommandIsRefusedAndTheSessionEnds(t *testing.T) { t.Fatalf("termination reason %q, want %q", term.Reason, TerminateRevoked) } } + +// An ack that cannot leave leaves the command untaken: the box does not act, +// and the retry acts and gets its result, rather than an ack and silence. +func TestAnAckThatCannotLeaveForgetsTheCommand(t *testing.T) { + h, box, rec, _ := newRig(t) + subscribe(t, h, rec) + + rec.mu.Lock() + rec.fail = errors.New("no uplink") + rec.mu.Unlock() + cmd := cmdSetMode(string(control.ModeCharge), 7, 200_000, nil) + env, _ := newEnvelope(MsgCmd, nil, cmd) + frame, _ := testCodec{}.EncodeFrame(Frame{Lane: LaneControl, Bucket: 512, Envelope: env}) + if err := h.Handle(t.Context(), frame); err == nil { + t.Fatal("a command whose ack could not leave reported nothing") + } + if box.mode == control.ModeCharge { + t.Fatal("the box acted on a command whose ack never left") + } + + rec.mu.Lock() + rec.fail = nil + rec.mu.Unlock() + deliver(t, h, MsgCmd, nil, cmd) + rec.only(t, MsgCmdAck) + if res := body[CmdResult](t, rec.only(t, MsgCmdResult)); res.State != CmdApplied { + t.Fatalf("the retry got %q", res.State) + } +} diff --git a/go/internal/appproto/handler.go b/go/internal/appproto/handler.go index 31d395328..baa31e6e7 100644 --- a/go/internal/appproto/handler.go +++ b/go/internal/appproto/handler.go @@ -982,10 +982,17 @@ func (h *Handler) acceptCmd(cmd Cmd, uptimeMs int64) (took bool, err error) { LeaseID: h.cfg.NewLeaseID(), ExpiresAtMs: uptimeMs + LeaseMs, } - if rec, fresh := h.cfg.Commands.remember(h.cmdRoot(), cmd.CmdID, ack, uptimeMs); !fresh { + root := h.cmdRoot() + if rec, fresh := h.cfg.Commands.remember(root, cmd.CmdID, ack, uptimeMs); !fresh { return false, h.replayCmd(rec) } - return true, h.sendCmdAck(ack) + if err := h.sendCmdAck(ack); err != nil { + // The box has not acted, and the app never heard the ack. Keeping + // the id would answer its retry with an ack and no result. + h.cfg.Commands.forget(root, cmd.CmdID) + return false, err + } + return true, nil } // settleAndReport records the outcome, reports it, and pushes a fresh plan diff --git a/go/internal/appproto/protocol.go b/go/internal/appproto/protocol.go index 26783d477..662614475 100644 --- a/go/internal/appproto/protocol.go +++ b/go/internal/appproto/protocol.go @@ -224,6 +224,7 @@ const ( CapPriceSpot = protocol.CapPriceSpot CapApiPassthrough = protocol.CapApiPassthrough CapCloudLink = protocol.CapCloudLink + CapCloudTun = protocol.CapCloudTun CapCmdExchange = protocol.CapCmdExchange ScopeLiveRead = protocol.ScopeLiveRead ScopeHistoryRead = protocol.ScopeHistoryRead diff --git a/go/internal/appuplink/client.go b/go/internal/appuplink/client.go index cab90f236..771899302 100644 --- a/go/internal/appuplink/client.go +++ b/go/internal/appuplink/client.go @@ -189,6 +189,13 @@ type Uplink struct { // watch's session survives the hour turning under it. exchanges *exchanges + // slots holds each device's one session, across the relay's + // connections and the tunnel through FTW Cloud. + slots *slots + // tunnel is the table of sessions FTW Cloud carries. It runs whether or + // not the relay can be reached. + tunnel *Tunnel + // The dead man's switch bookkeeping: which row ids the relay holds, // so a removed subscription's row is deleted rather than left armed. deadmanMu sync.Mutex @@ -221,6 +228,7 @@ func (u *Uplink) DropSessionsByAppKey(pub []byte) int { if u.exchanges != nil { dropped += u.exchanges.dropByAppKey(pub) } + dropped += u.tunnel.dropByAppKey(pub) return dropped } @@ -279,6 +287,8 @@ func New(opts Options) (*Uplink, error) { if u.deadmanRefresh <= 0 { u.deadmanRefresh = deadmanRefreshInterval } + u.slots = &slots{} + u.tunnel = newTunnel(opts.Handler, opts.Enroll, u.slots, u.log) if opts.ExchangeHandler != nil { u.exchanges = &exchanges{ build: opts.ExchangeHandler, @@ -305,8 +315,11 @@ func productionDialer() *websocket.Dialer { } } -// Run keeps the uplink up until the context ends. +// Run keeps the uplink up until the context ends. The tunnel's table runs +// beside it, so a phone reaches the box through FTW Cloud while the relay +// cannot be reached. func (u *Uplink) Run(ctx context.Context) error { + go u.tunnel.Run(ctx) for { if ctx.Err() != nil { return ctx.Err() @@ -567,6 +580,7 @@ func (u *Uplink) serve(ctx context.Context, conn *websocket.Conn) (int, string, } live := &sessions{ + slots: u.slots, build: u.opts.Handler, enroll: u.opts.Enroll, static: u.opts.Enroll.StaticKey(), diff --git a/go/internal/appuplink/cloudtunnel_test.go b/go/internal/appuplink/cloudtunnel_test.go new file mode 100644 index 000000000..2b72b3fba --- /dev/null +++ b/go/internal/appuplink/cloudtunnel_test.go @@ -0,0 +1,406 @@ +package appuplink + +import ( + "bytes" + "context" + "crypto/ecdsa" + "crypto/elliptic" + "crypto/rand" + "crypto/tls" + "crypto/x509" + "log/slog" + "net" + "net/http" + "net/http/httptest" + "path/filepath" + "sync" + "sync/atomic" + "testing" + "time" + + "github.com/gorilla/websocket" + "github.com/srcfl/ftw/go/internal/apiauth" + "github.com/srcfl/ftw/go/internal/appenroll" + "github.com/srcfl/ftw/go/internal/appproto" + "github.com/srcfl/ftw/go/internal/cloudlink" + "github.com/srcfl/ftw/go/protocol/appwire" + "github.com/srcfl/ftw/go/protocol/cloudwire" +) + +// The box's half of the one door, end to end: the real cloud link, the real +// tunnel table and the real message handler, with the relay unreachable. The +// service here is FTW Cloud cut to what the tunnel needs, as srcfl/ftw-cloud +// runs it: the handshake, the roster, and tun stamped with the device id of +// the roster entry whose key the phone presented. + +type miniCloud struct { + t *testing.T + key *ecdsa.PrivateKey + srv *httptest.Server + + mu sync.Mutex + conn *websocket.Conn + home string + roster *cloudwire.Roster + got []string + toApp chan cloudwire.Tun + rostered chan struct{} + wmu sync.Mutex +} + +func newMiniCloud(t *testing.T) *miniCloud { + t.Helper() + key, _ := ecdsa.GenerateKey(elliptic.P256(), rand.Reader) + c := &miniCloud{t: t, key: key, toApp: make(chan cloudwire.Tun, 64), rostered: make(chan struct{}, 8)} + c.srv = httptest.NewUnstartedServer(http.HandlerFunc(c.serve)) + c.srv.TLS = &tls.Config{MinVersion: tls.VersionTLS13} + c.srv.StartTLS() + t.Cleanup(c.srv.Close) + return c +} + +func (c *miniCloud) endpoint() string { return "wss://" + c.srv.Listener.Addr().String() + "/v1/box" } + +func (c *miniCloud) roots() *x509.CertPool { + pool := x509.NewCertPool() + pool.AddCert(c.srv.Certificate()) + return pool +} + +func (c *miniCloud) write(conn *websocket.Conn, key *ecdsa.PrivateKey, home, t string, body any) error { + env, err := cloudwire.Seal(key, home, t, body) + if err != nil { + return err + } + raw, _ := env.Encode() + frame, _ := cloudwire.EncodeFrame(raw) + c.wmu.Lock() + defer c.wmu.Unlock() + return conn.WriteMessage(websocket.BinaryMessage, frame) +} + +func (c *miniCloud) serve(w http.ResponseWriter, r *http.Request) { + conn, err := (&websocket.Upgrader{}).Upgrade(w, r, nil) + if err != nil { + return + } + defer conn.Close() + conn.SetReadLimit(cloudwire.MaxTunFrame) + ekm, _ := r.TLS.ExportKeyingMaterial(cloudwire.EKMLabel, nil, cloudwire.EKMBytes) + nonce := bytes.Repeat([]byte{9}, cloudwire.NonceBytes) + if c.write(conn, nil, "", cloudwire.MsgChallenge, cloudwire.Challenge{V: 1, Nonce: nonce}) != nil { + return + } + _, msg, err := conn.ReadMessage() + if err != nil { + return + } + payload, _ := cloudwire.DecodeFrame(msg) + env, _ := cloudwire.DecodeEnvelope(payload) + var hello cloudwire.Hello + if env.T != cloudwire.MsgHello || cloudwire.Unmarshal(env.B, &hello) != nil { + c.t.Error("the box opened with no hello") + return + } + pub, err := cloudwire.ParsePublicKey(hello.PK) + home := cloudwire.HomeID(hello.PK) + if err != nil || cloudwire.Verify(pub, home, env) != nil || !bytes.Equal(hello.EKM, ekm) { + c.t.Error("hello did not check out") + return + } + if c.write(conn, c.key, home, cloudwire.MsgHelloOK, cloudwire.HelloOK{ + BoxNonce: hello.BoxNonce, EKM: ekm, Have: map[string]string{}}) != nil { + return + } + c.mu.Lock() + c.conn, c.home = conn, home + c.mu.Unlock() + for { + _, msg, err := conn.ReadMessage() + if err != nil { + return + } + payload, err := cloudwire.DecodeFrame(msg) + if err != nil { + c.t.Errorf("box frame: %v", err) + return + } + env, err := cloudwire.DecodeEnvelope(payload) + if err != nil { + c.t.Errorf("box envelope: %v", err) + return + } + c.mu.Lock() + c.got = append(c.got, env.T) + c.mu.Unlock() + switch env.T { + case cloudwire.MsgTun: + var t cloudwire.Tun + if cloudwire.Unmarshal(env.B, &t) == nil { + c.toApp <- t + } + case cloudwire.MsgRoster: + var ro cloudwire.Roster + if cloudwire.Verify(pub, home, env) == nil && cloudwire.Unmarshal(env.B, &ro) == nil { + c.mu.Lock() + c.roster = &ro + c.mu.Unlock() + c.rostered <- struct{}{} + } + } + } +} + +// fromApp is a phone's tun: the service finds the roster entry by the key +// the phone's own handshake with it proved, and names the device from it. +// A key the roster does not list with tun gets nothing forwarded. +func (c *miniCloud) fromApp(key []byte, sid uint32, d []byte) bool { + c.mu.Lock() + conn, home, ro := c.conn, c.home, c.roster + c.mu.Unlock() + if conn == nil || ro == nil { + return false + } + for _, dev := range ro.Devices { + if dev.Tun && bytes.Equal(dev.Key, key) { + return c.write(conn, nil, home, cloudwire.MsgTun, cloudwire.Tun{Dev: dev.ID, Sid: sid, D: d}) == nil + } + } + return false +} + +// forward sends a tun to the box as it is, with no roster check: a service +// that forwards what it should not. +func (c *miniCloud) forward(dev string, sid uint32, d []byte) { + c.mu.Lock() + conn, home := c.conn, c.home + c.mu.Unlock() + _ = c.write(conn, nil, home, cloudwire.MsgTun, cloudwire.Tun{Dev: dev, Sid: sid, D: d}) +} + +func (c *miniCloud) toAppNext(t *testing.T) cloudwire.Tun { + t.Helper() + select { + case m := <-c.toApp: + return m + case <-time.After(5 * time.Second): + t.Fatal("no tun from the box") + return cloudwire.Tun{} + } +} + +type quietSite struct{} + +func (quietSite) Live() cloudlink.Live { return cloudlink.Live{Fields: map[string]int64{"2": 1200}} } +func (quietSite) Loadpoints() (cloudwire.LP, bool) { return cloudwire.LP{}, false } +func (quietSite) Prices() (cloudwire.Prices, bool) { return cloudwire.Prices{}, false } + +// memState is the state store's two calls the link makes. +type memState struct { + mu sync.Mutex + m map[string]string +} + +func (k *memState) LoadConfig(key string) (string, bool) { + k.mu.Lock() + defer k.mu.Unlock() + v, ok := k.m[key] + return v, ok +} + +func (k *memState) SaveConfig(key, value string) error { + k.mu.Lock() + defer k.mu.Unlock() + k.m[key] = value + return nil +} + +// unreachable is an address nothing listens on: the relay, gone. +func unreachable(t *testing.T) string { + t.Helper() + l, err := net.Listen("tcp", "127.0.0.1:0") + if err != nil { + t.Fatal(err) + } + addr := l.Addr().String() + _ = l.Close() + return "ws://" + addr +} + +// With the relay unreachable, a phone the roster lists sends a command +// through FTW Cloud and gets the box's answer. The readable copy stays off: +// nothing but the roster and the tunnel leaves the box. +func TestACommandThroughFTWCloudWithTheRelayGone(t *testing.T) { + enroll, err := appenroll.LoadOrCreate(filepath.Join(t.TempDir(), "nova.key")) + if err != nil { + t.Fatal(err) + } + modes := &countingModes{} + cmds := appproto.NewCmdLog() + // The link is built after the uplink, as cmd/ftw builds it, and bound + // late, as there. + var bound atomic.Pointer[cloudlink.Link] + build := func(s appproto.Sender, caller apiauth.Caller, grants appproto.GrantReader) (*appproto.Handler, error) { + return appproto.New(appproto.Config{ + Clock: appproto.SystemClock{StartedAt: time.Now(), Source: "ntp"}, + Site: stubSite{}, Info: stubInfo{}, Modes: modes, Plans: stubPlans{}, Codec: codec{}, + Sender: s, Caller: caller, Grants: grants, Commands: cmds, + Cloud: func() *appproto.CloudSession { + if l := bound.Load(); l != nil { + return l.SessionInfo() + } + return nil + }, + SrcGrid: "meter", SrcPV: "meter", SrcBattery: "meter", Logger: slog.New(slog.DiscardHandler), + }) + } + relay := unreachable(t) + uplink, err := New(Options{Endpoint: relay, Enroll: enroll, Handler: build, Logger: slog.New(slog.DiscardHandler)}) + if err != nil { + t.Fatal(err) + } + cloud := newMiniCloud(t) + link, err := cloudlink.New(cloudlink.Options{ + Site: quietSite{}, Roster: CloudRoster{Enroll: enroll, Tunnel: true}, KV: &memState{m: map[string]string{}}, + KeyPath: filepath.Join(t.TempDir(), "cloud.key"), Build: "test", Tunnel: uplink.Tunnel(), + Endpoint: cloud.endpoint(), ServiceKeys: []*ecdsa.PublicKey{&cloud.key.PublicKey}, RootCAs: cloud.roots(), + Tick: 20 * time.Millisecond, Logger: slog.New(slog.DiscardHandler), + }) + if err != nil { + t.Fatal(err) + } + uplink.Tunnel().Attach(link) + bound.Store(link) + enroll.OnChange(link.RosterChanged) + enroll.OnPairingCode(link.RosterChanged) + ctx, cancel := context.WithCancel(context.Background()) + t.Cleanup(cancel) + go func() { _ = uplink.Run(ctx) }() + go func() { _ = link.Run(ctx) }() + + // The phone paired earlier; the box links for it. + code, _, err := enroll.MintPairingCode(apiauth.RoleOwner, appenroll.PairingTTL) + if err != nil { + t.Fatal(err) + } + app, err := newAppClient() + if err != nil { + t.Fatal(err) + } + if _, err := enroll.Authorise(app.staticPublic(), code); err != nil { + t.Fatal(err) + } + link.RosterChanged() + deadline := time.After(10 * time.Second) + for { + select { + case <-cloud.rostered: + case <-deadline: + t.Fatal("the box never sent a roster with the phone") + } + cloud.mu.Lock() + ok := cloud.roster != nil && len(cloud.roster.Devices) == 1 && cloud.roster.Devices[0].Tun + cloud.mu.Unlock() + if ok { + break + } + } + + dev := cloudwire.DeviceID(app.staticPublic()) + m1, err := app.message1(enroll.StaticKey().Public(), nil) + if err != nil { + t.Fatal(err) + } + if !cloud.fromApp(app.staticPublic(), 1, m1) { + t.Fatal("the service would not forward the phone's handshake") + } + reply := cloud.toAppNext(t) + if reply.Dev != dev || reply.Sid != 1 { + t.Fatalf("the handshake's answer is %+v", reply) + } + if err := app.readMessage2(reply.D); err != nil { + t.Fatal(err) + } + sendFrame := func(msgType string, body any) { + t.Helper() + raw, _ := appproto.Marshal(body) + frame, _ := appwire.EncodeFrame(appwire.Frame{Lane: appwire.LaneControl, Bucket: 512, + Envelope: appwire.Envelope{T: msgType, B: raw}}) + msg, _ := app.encrypt(frame) + if !cloud.fromApp(app.staticPublic(), 1, msg) { + t.Fatalf("the service would not forward %s", msgType) + } + } + receiveFrame := func(want string) appwire.Frame { + t.Helper() + for { + m := cloud.toAppNext(t) + plain, err := app.decrypt(m.D) + if err != nil { + t.Fatal(err) + } + f, _ := appwire.DecodeFrame(plain) + if f.Envelope.T == want { + return f + } + } + } + sendFrame(appproto.MsgHello, appproto.Hello{Proto: appproto.ProtoRange{Max: appproto.ProtoMax}, + App: appproto.AppInfo{Build: "test", UA: "ios"}}) + var ok appproto.HelloOK + if err := appproto.Unmarshal(receiveFrame(appproto.MsgHelloOK).Envelope.B, &ok); err != nil { + t.Fatal(err) + } + + start := time.Now() + sendFrame(appproto.MsgCmd, setModeCmd("0192f2a0-7c1e-7000-8000-00000000c10d")) + receiveFrame(appproto.MsgCmdAck) + var res appproto.CmdResult + if err := appproto.Unmarshal(receiveFrame(appproto.MsgCmdResult).Envelope.B, &res); err != nil { + t.Fatal(err) + } + t.Logf("cmd to result through the cloud link: %s", time.Since(start)) + if res.State != appproto.CmdApplied || modes.count() != 1 { + t.Fatalf("result %q after %d mode changes", res.State, modes.count()) + } + + // A stranger's handshake, forwarded anyway, as a broken or hostile + // service might: the box itself answers nothing and opens nothing. + stranger, _ := newAppClient() + sm1, _ := stranger.message1(enroll.StaticKey().Public(), nil) + strangerDev := cloudwire.DeviceID(stranger.staticPublic()) + cloud.forward(strangerDev, 1, sm1) + quiet := time.After(500 * time.Millisecond) +wait: + for { + select { + case m := <-cloud.toApp: + // The phone's own session goes on beside it. + if m.Dev == strangerDev { + t.Fatalf("the box answered a stranger: sid %d, %d bytes", m.Sid, len(m.D)) + } + case <-quiet: + break wait + } + } + if uplink.Tunnel().count() != 1 { + t.Fatal("a stranger's handshake changed the box's sessions") + } + + // The readable copy is off: the box sent the roster and the tunnel, + // and nothing to read. + cloud.mu.Lock() + got := append([]string(nil), cloud.got...) + cloud.mu.Unlock() + for _, typ := range got { + if typ != cloudwire.MsgRoster && typ != cloudwire.MsgTun { + t.Fatalf("%s left a box whose readable copy is off", typ) + } + } + if st := link.Status(); st.State != "off" || st.Link != "on" { + t.Fatalf("status %+v", st) + } + if ok.Cloud == nil || ok.Cloud.Home != link.Status().Home || len(ok.Cloud.Sources) != 0 { + t.Fatalf("hello_ok.cloud %+v", ok.Cloud) + } +} diff --git a/go/internal/appuplink/exchange_test.go b/go/internal/appuplink/exchange_test.go index bb8c7a4df..6fe9770d3 100644 --- a/go/internal/appuplink/exchange_test.go +++ b/go/internal/appuplink/exchange_test.go @@ -623,15 +623,11 @@ func TestHandshakesAreRationed(t *testing.T) { // Isolation: eight watch sessions leave four phone sessions running, and a // ninth watch ends the oldest watch's session, never a phone's. func TestWatchesNeverTakeAPhonesPlace(t *testing.T) { + // The relay lets four streams into a room. + const phones = 4 r := newXRig(t) - streams := &sessions{ - build: handlerWith(false, r.charger, r.cmds), - enroll: r.enroll, - static: r.enroll.StaticKey(), - write: func([]byte) error { return nil }, - log: slog.New(slog.DiscardHandler), - } - for i := 0; i < MaxSessions; i++ { + streams := relayTable(&slots{}, r.enroll) + for i := 0; i < phones; i++ { code, _, err := r.enroll.MintPairingCode(apiauth.RoleOwner, appenroll.PairingTTL) if err != nil { t.Fatal(err) @@ -640,26 +636,20 @@ func TestWatchesNeverTakeAPhonesPlace(t *testing.T) { if err != nil { t.Fatal(err) } - message1, err := app.message1(r.enroll.StaticKey().Public(), code) - if err != nil { - t.Fatal(err) - } - if err := streams.open(message1); err != nil { - t.Fatalf("phone %d: %v", i, err) - } + relayOpen(t, streams, r.enroll, app, code) } first, _ := r.addWatch(t) for i := 1; i < MaxExchangeSessions; i++ { r.addWatch(t) } - if streams.count() != MaxSessions || r.x.count() != MaxExchangeSessions { + if streams.count() != phones || r.x.count() != MaxExchangeSessions { t.Fatalf("%d phone and %d watch sessions, want %d and %d", - streams.count(), r.x.count(), MaxSessions, MaxExchangeSessions) + streams.count(), r.x.count(), phones, MaxExchangeSessions) } r.addWatch(t) - if streams.count() != MaxSessions { + if streams.count() != phones { t.Fatalf("a ninth watch left %d phone sessions", streams.count()) } if r.ask(t, first, appproto.MsgNow, struct{}{}) != nil { diff --git a/go/internal/appuplink/roster.go b/go/internal/appuplink/roster.go new file mode 100644 index 000000000..693477745 --- /dev/null +++ b/go/internal/appuplink/roster.go @@ -0,0 +1,39 @@ +package appuplink + +import ( + "time" + + "github.com/srcfl/ftw/go/internal/apiauth" + "github.com/srcfl/ftw/go/internal/appenroll" + "github.com/srcfl/ftw/go/internal/appproto" + "github.com/srcfl/ftw/go/internal/cloudlink" +) + +// CloudRoster is enrolment as FTW Cloud's roster reads it: every paired +// device with the scopes the box derives for it, a phone's role, or for a +// watch what its role and its phone's role share. With Tunnel set, a phone +// may open a session through FTW Cloud and the QR code on the screen is +// listed; a watch has no stream and reaches the box through its exchange. +type CloudRoster struct { + Enroll *appenroll.Identity + Tunnel bool +} + +// Devices lists the paired devices. +func (r CloudRoster) Devices() []cloudlink.RosterDevice { + var out []cloudlink.RosterDevice + for _, e := range r.Enroll.Roster() { + out = append(out, cloudlink.RosterDevice{ID: e.ID, Key: e.Key, Role: e.Role, Epoch: e.Epoch, + Scopes: appproto.GrantScopes(e.Role, e.ParentRole), Tun: r.Tunnel && e.Role != apiauth.RoleWatch}) + } + return out +} + +// Pending is the QR code on the screen, while the box takes the tunnel: a +// new phone can then pair through FTW Cloud. +func (r CloudRoster) Pending() ([]byte, time.Time, bool) { + if !r.Tunnel { + return nil, time.Time{}, false + } + return r.Enroll.PendingCode() +} diff --git a/go/internal/appuplink/session.go b/go/internal/appuplink/session.go index 5f9c6c544..40771eb7d 100644 --- a/go/internal/appuplink/session.go +++ b/go/internal/appuplink/session.go @@ -6,6 +6,7 @@ import ( "errors" "log/slog" "sync" + "time" "github.com/srcfl/ftw/go/internal/apiauth" "github.com/srcfl/ftw/go/internal/appenroll" @@ -13,13 +14,6 @@ import ( "github.com/srcfl/ftw/go/protocol/appwire" ) -// MaxSessions is how many phones may watch one house at once. -// -// It matches the relay's own per-handle stream limit; a fifth stream never -// reaches the box, so a larger number here would only describe sessions that -// cannot exist. -const MaxSessions = 4 - // session is one app's encrypted conversation with this box. // // A session belongs to one relay connection and never outlives it. The app @@ -32,6 +26,25 @@ type session struct { // appStatic is the app's static key as the handshake authenticated it, // never as it was claimed. Logged as a prefix and nothing more. appStatic []byte + // release frees the device's slot (see slots), if the session still + // holds it. Nil for a session that took none. Set through setRelease, + // because a newer session can end this one before its claim returns. + relMu sync.Mutex + release func() + closed bool +} + +// setRelease keeps the slot's release for close, or runs it at once when the +// session already ended. +func (s *session) setRelease(fn func()) { + s.relMu.Lock() + if s.closed { + s.relMu.Unlock() + fn() + return + } + s.release = fn + s.relMu.Unlock() } // close ends one session for good. @@ -44,8 +57,36 @@ type session struct { func (s *session) close() { s.handler.Close() s.transport.Close() + s.relMu.Lock() + s.closed = true + fn := s.release + s.release = nil + s.relMu.Unlock() + if fn != nil { + fn() + } +} + +// pending is a handshake the box answered whose session has not yet proved +// itself. A message 1 can be replayed by whoever carries it, and the box +// cannot tell a replay from a phone until the phone's first transport +// message decrypts. Until then it takes no device's place, stamps nothing +// and spends no code. +type pending struct { + transport *appwire.Transport + appStatic []byte + deviceID string + commit func() (appenroll.Grant, error) + at time.Time } +// pendingKept is how many unproved handshakes a table holds, and how long +// one waits for its first transport message. +const ( + pendingKept = 8 + pendingWait = 30 * time.Second +) + // sender encrypts one session's frames onto the shared socket. type sender struct { transport *appwire.Transport @@ -69,14 +110,21 @@ func (s sender) Send(frame []byte) error { // decide. A message nothing authenticates is either a new handshake or noise, // and noise is dropped. type sessions struct { - mu sync.Mutex - live []*session - build HandlerBuilder + mu sync.Mutex + live []*session + pending []*pending + build HandlerBuilder + // slots is the box's one slot per device, shared with the tunnel. Nil + // in a test that builds a table alone. + slots *slots enroll *appenroll.Identity static appwire.KeyPair write func([]byte) error log *slog.Logger + + // testOut is where a test's write puts what the table sends. + testOut chan []byte } // prologue binds a session to the box it was enrolled against, so a captured @@ -101,12 +149,31 @@ func (s *sessions) deliver(ctx context.Context, message []byte) { } } -// route hands the message to the session that can decrypt it, if any. +// route hands the message to the session that can decrypt it, if any. A +// message that proves a pending handshake makes its session live first. func (s *sessions) route(ctx context.Context, message []byte) bool { s.mu.Lock() live := append([]*session(nil), s.live...) + waiting := append([]*pending(nil), s.pending...) s.mu.Unlock() + for _, p := range waiting { + frame, err := p.transport.Decrypt(message) + if err != nil { + continue + } + sess, err := s.promote(p) + if err != nil { + s.log.Debug("a proved handshake was not admitted", "err", err) + return true + } + if err := sess.handler.Handle(ctx, frame); err != nil { + s.log.Warn("app session failed on a frame", "err", err) + s.drop(sess) + } + return true + } + for _, sess := range live { frame, err := sess.transport.Decrypt(message) if err != nil { @@ -159,13 +226,58 @@ func (s *sessions) open(message []byte) error { // The payload is the pairing code from the QR. A device that has already // paired presents nothing and is recognised by its static key, which is - // what makes the code single-use without breaking reconnects. - grant, err := s.enroll.Authorise(noise.RemoteStatic, payload) + // what makes the code single-use without breaking reconnects. Checked + // now, committed when the session proves itself. + grant, commit, err := s.enroll.Check(noise.RemoteStatic, payload) if err != nil { transport.Close() return err } + s.mu.Lock() + now := time.Now() + kept := s.pending[:0] + for _, p := range s.pending { + if now.Sub(p.at) < pendingWait { + kept = append(kept, p) + } else { + p.transport.Close() + } + } + s.pending = kept + if len(s.pending) >= pendingKept { + s.pending[0].transport.Close() + s.pending = s.pending[1:] + } + s.pending = append(s.pending, &pending{transport: transport, appStatic: noise.RemoteStatic, + deviceID: grant.DeviceID, commit: commit, at: now}) + s.mu.Unlock() + + if err := s.write(reply); err != nil { + return err + } + return nil +} + +// promote makes a proved handshake a live session: it commits the pairing or +// the reconnect stamp, builds the handler from the grant as it stands now, +// and takes the device's one slot. +func (s *sessions) promote(p *pending) (*session, error) { + s.mu.Lock() + for i, q := range s.pending { + if q == p { + s.pending = append(s.pending[:i], s.pending[i+1:]...) + break + } + } + s.mu.Unlock() + + grant, err := p.commit() + if err != nil { + p.transport.Close() + return nil, err + } + // The grant becomes a caller here, and this is the only place it is // built. Everything the box later decides about this phone — which // commands it may send, which routes it may reach — is decided from this @@ -177,45 +289,33 @@ func (s *sessions) open(message []byte) error { Scopes: appproto.ScopesForGrant(grant.Role, grant.ParentRole), Epoch: grant.Epoch, } - handler, err := s.build( - sender{transport: transport, write: s.write}, + sender{transport: p.transport, write: s.write}, caller, liveGrant{enroll: s.enroll, deviceID: grant.DeviceID}, ) if err != nil { - transport.Close() - return err + p.transport.Close() + return nil, err } - sess := &session{transport: transport, handler: handler, appStatic: noise.RemoteStatic} + sess := &session{transport: p.transport, handler: handler, appStatic: p.appStatic} s.admit(sess) - - // Written after the session is registered, so a fast app whose first - // frame arrives immediately meets a session that exists. - if err := s.write(reply); err != nil { - s.drop(sess) - return err - } + // The device's one slot: a session it held before, here or through FTW + // Cloud, ends now. + sess.setRelease(s.slots.claim(grant.DeviceID, func() { s.drop(sess) })) s.log.Info("app session open", "sessions", s.count()) - return nil + return sess, nil } -// admit adds a session, evicting the oldest when the room is full. -// -// Evicting rather than refusing: the newcomer is the phone somebody is holding -// right now, and the session it displaces is one whose socket the relay has -// probably already reaped. +// admit adds a session. There is no count to evict by: each device holds one +// slot (see slots), so a phone that reconnects replaces its own session and +// never another device's. The relay lets four streams into a room. func (s *sessions) admit(sess *session) { s.mu.Lock() defer s.mu.Unlock() - s.live = append(s.live, sess) - for len(s.live) > MaxSessions { - s.live[0].close() - s.live = s.live[1:] - } } // dropByAppKey terminates and forgets every session a revoked key is @@ -226,6 +326,16 @@ func (s *sessions) dropByAppKey(pub []byte) int { s.mu.Lock() defer s.mu.Unlock() + waiting := s.pending[:0] + for _, p := range s.pending { + if bytes.Equal(p.appStatic, pub) { + p.transport.Close() + continue + } + waiting = append(waiting, p) + } + s.pending = waiting + dropped := 0 kept := s.live[:0] for _, live := range s.live { @@ -263,6 +373,10 @@ func (s *sessions) closeAll() { s.mu.Lock() live := s.live s.live = nil + for _, p := range s.pending { + p.transport.Close() + } + s.pending = nil s.mu.Unlock() for _, sess := range live { diff --git a/go/internal/appuplink/tunnel.go b/go/internal/appuplink/tunnel.go new file mode 100644 index 000000000..d2a02ce80 --- /dev/null +++ b/go/internal/appuplink/tunnel.go @@ -0,0 +1,603 @@ +package appuplink + +import ( + "bytes" + "context" + "errors" + "log/slog" + "sync" + "time" + + "github.com/srcfl/ftw/go/internal/apiauth" + "github.com/srcfl/ftw/go/internal/appenroll" + "github.com/srcfl/ftw/go/internal/appproto" + "github.com/srcfl/ftw/go/protocol/appwire" + "github.com/srcfl/ftw/go/protocol/cloudwire" +) + +// The tunnel: the phones' sessions that FTW Cloud carries +// (docs/protocol-cloud.md, "Tunnel"). +// +// A phone's session through the service is the same ftw.session.v1 session +// it runs over the relay, byte for byte. Only the routing differs. The relay +// cannot say whose a message is, so the box lets each session's AEAD try. The +// service says which device and which session, tun {dev, sid, d}, so the box +// looks the session up. It still trusts nothing the service says: the +// handshake proves the phone's key and the box admits it by that key, exactly +// as over the relay. dev only picks the slot, and a handshake whose key is +// not the device dev names is dropped. +// +// A new session waits, pending, until its first transport message decrypts. +// The service sees every message 1 and can replay one under a new sid; the +// box answers it, but the replay never proves itself, so it takes no +// device's place, stamps nothing and spends no code. Each new session runs +// its handshake on a goroutine of its own, so a slow disk under the +// enrolment write never holds up routing or the one-second tick. + +// TunSender sends one tun up the box's link to FTW Cloud. cloudlink.Link is +// one. bulk says the message carries a lane 1 frame, which the link paces; +// everything else goes first. An error whose Dropped method says true lost +// that one message and counted it; any other error means no message will +// leave until the box reconnects. +type TunSender interface { + SendTun(t cloudwire.Tun, bulk bool) error +} + +// dropped reports whether a TunSender's error lost one message only. +func dropped(err error) bool { + var d interface{ Dropped() bool } + return errors.As(err, &d) && d.Dropped() +} + +const ( + // tunInbox is how many tun messages wait for the table. The link's + // reader never waits on it; past it a message is dropped, which Noise + // transport survives at the cost of one frame. + tunInbox = 256 + // tunSessionInbox is the same, per session, so one phone's slow + // command never holds up another phone. + tunSessionInbox = 64 + // tunOpensPerDevice bounds one device's handshakes through the service + // a minute, so one phone or one pairing cannot starve the others. + // tunOpensPerMinute bounds them all, each of which costs the box up to + // four X25519 operations. + tunOpensPerDevice = 12 + tunOpensPerMinute = 240 + // tunPendingPerDevice is how many unproved sessions a device may hold, + // and pendingWait (session.go) how long one waits to prove itself. + tunPendingPerDevice = 4 +) + +// Tunnel is the box's table of sessions that FTW Cloud carries: one live per +// device, in the slot that device holds across every carrier, and the +// pending ones that have not yet proved themselves. +type Tunnel struct { + build HandlerBuilder + enroll *appenroll.Identity + static appwire.KeyPair + slots *slots + log *slog.Logger + now func() time.Time + + inbox chan tunIn + + mu sync.Mutex + sender TunSender + // gen moves on every Drop, so a message queued from an uplink that has + // ended opens nothing on the next. + gen uint64 + live map[string]*tunSession + pending map[tunKey]*tunSession + opens map[string][]time.Time +} + +type tunKey struct { + dev string + sid uint32 +} + +type tunIn struct { + t cloudwire.Tun + gen uint64 +} + +// tunSession is one phone's session through the service. Its session is nil +// until it proves itself; its goroutine owns everything else. +type tunSession struct { + dev string + sid uint32 + gen uint64 + session *session + // appStatic is the key the handshake proved, once it has run. + appStatic []byte + in chan []byte + stop chan struct{} + once sync.Once +} + +// Tunnel returns the uplink's tunnel table, for the box's link to FTW Cloud +// to deliver to. Its sessions take the same slots as the relay's. +func (u *Uplink) Tunnel() *Tunnel { return u.tunnel } + +func newTunnel(build HandlerBuilder, enroll *appenroll.Identity, slots *slots, log *slog.Logger) *Tunnel { + return &Tunnel{ + build: build, enroll: enroll, static: enroll.StaticKey(), slots: slots, log: log, now: time.Now, + inbox: make(chan tunIn, tunInbox), live: map[string]*tunSession{}, pending: map[tunKey]*tunSession{}, + opens: map[string][]time.Time{}, + } +} + +// Attach names where answers go. The box's link calls nothing until then. +func (t *Tunnel) Attach(s TunSender) { + t.mu.Lock() + defer t.mu.Unlock() + t.sender = s +} + +// Deliver takes one tun from the service. It never blocks. +func (t *Tunnel) Deliver(m cloudwire.Tun) { + t.mu.Lock() + gen := t.gen + t.mu.Unlock() + select { + case t.inbox <- tunIn{t: m, gen: gen}: + default: + t.log.Debug("tunnel inbox full; a message was dropped") + } +} + +// Drop ends every session, when the uplink that carried them ends. The +// phones start again on the next. +func (t *Tunnel) Drop() { + t.mu.Lock() + t.gen++ + all := make([]*tunSession, 0, len(t.live)+len(t.pending)) + for _, s := range t.live { + all = append(all, s) + } + for _, s := range t.pending { + all = append(all, s) + } + t.live, t.pending = map[string]*tunSession{}, map[tunKey]*tunSession{} + t.mu.Unlock() + for _, s := range all { + s.end() + } +} + +// Run serves the table until ctx ends: the messages the service delivers, +// and the one-second pulse lane 0 needs. +func (t *Tunnel) Run(ctx context.Context) { + tick := time.NewTicker(tickInterval) + defer tick.Stop() + defer t.Drop() + for { + select { + case <-ctx.Done(): + return + case m := <-t.inbox: + t.route(ctx, m) + case <-tick.C: + t.tick() + } + } +} + +// count is how many live sessions the table holds. +func (t *Tunnel) count() int { + t.mu.Lock() + defer t.mu.Unlock() + return len(t.live) +} + +// route hands one message to its session, live or pending, or starts a +// pending one. It never waits: each session works on its own goroutine. +func (t *Tunnel) route(ctx context.Context, m tunIn) { + key := tunKey{m.t.Dev, m.t.Sid} + t.mu.Lock() + if m.gen != t.gen { + t.mu.Unlock() + return + } + s := t.pending[key] + if cur := t.live[m.t.Dev]; cur != nil && cur.sid == m.t.Sid { + s = cur + } + if s == nil && !m.t.Ended() { + s = t.startLocked(ctx, key, m.gen) + } + t.mu.Unlock() + + switch { + case s == nil: + // The end of a session the box no longer holds, or a handshake + // over the device's allowance. + case m.t.Ended(): + t.remove(s, false) + default: + select { + case s.in <- m.t.D: + default: + t.log.Debug("tunnel session inbox full; a message was dropped") + } + } +} + +// startLocked makes a pending session for a new (dev, sid), within the +// device's allowance. The caller holds t.mu. +func (t *Tunnel) startLocked(ctx context.Context, key tunKey, gen uint64) *tunSession { + if !t.allowOpenLocked(key.dev) { + return nil + } + held := 0 + var oldest *tunSession + for k, p := range t.pending { + if k.dev == key.dev { + held++ + if oldest == nil || p.sid != key.sid { + oldest = p + } + } + } + if held >= tunPendingPerDevice && oldest != nil { + delete(t.pending, tunKey{oldest.dev, oldest.sid}) + go oldest.end() + } + s := &tunSession{dev: key.dev, sid: key.sid, gen: gen, + in: make(chan []byte, tunSessionInbox), stop: make(chan struct{})} + t.pending[key] = s + go s.run(ctx, t) + return s +} + +var ( + errTunDevice = errors.New("appuplink: the handshake's key is not the device the service named") + errTunShort = errors.New("appuplink: too short to be handshake message 1") + errTunGone = errors.New("appuplink: the tunnel's uplink is gone") +) + +// run is one session's life: the handshake on its first message, then each +// message in order. Until a transport message decrypts, the session is +// pending and has no handler; the first that does makes it live. +func (s *tunSession) run(ctx context.Context, t *Tunnel) { + var first []byte + select { + case <-ctx.Done(): + return + case <-s.stop: + return + case first = <-s.in: + } + transport, commit, err := t.handshake(s, first) + if err != nil { + t.log.Debug("tun opened no session", "err", err) + t.remove(s, false) + return + } + defer transport.Close() + + wait := time.NewTimer(pendingWait) + defer wait.Stop() + for { + select { + case <-ctx.Done(): + return + case <-s.stop: + return + case <-wait.C: + if s.session == nil { + // Never proved: a replay, or a phone that went away. + t.remove(s, false) + return + } + case d := <-s.in: + frame, err := transport.Decrypt(d) + if err != nil { + // A replay, or bytes that are not this session's. The + // service can delay and drop; it cannot make the box act. + continue + } + if s.session == nil { + if err := t.promote(s, transport, commit); err != nil { + t.log.Debug("a proved tunnel session was not admitted", "err", err) + t.remove(s, true) + return + } + } + if err := s.session.handler.Handle(ctx, frame); err != nil { + t.log.Warn("app session failed on a frame", "err", err) + t.remove(s, true) + return + } + } + } +} + +// handshake runs the responder half for a new session and answers it. A +// message that is not a message 1 gets an end back, so the phone starts +// again; a key the box does not admit, or one the service put under another +// device, gets nothing. +func (t *Tunnel) handshake(s *tunSession, message1 []byte) (*appwire.Transport, func() (appenroll.Grant, error), error) { + end := cloudwire.Tun{Dev: s.dev, Sid: s.sid} + if len(message1) < appwire.Message1Overhead { + t.send(end) + return nil, nil, errTunShort + } + hs, err := appwire.NewResponder(appwire.HandshakeOptions{ + StaticKey: t.static, + Prologue: prologue(t.static.Public()), + }) + if err != nil { + return nil, nil, err + } + payload, err := hs.ReadMessage(message1) + if err != nil { + // Not message 1 of this box's session: most often a frame of a + // session the box dropped. The phone starts again. + t.send(end) + return nil, nil, err + } + // The service names the device; the handshake proves it. A key that is + // not the named device's is the service, or a bug, putting one phone's + // session under another's slot. + if cloudwire.DeviceID(hs.RemoteStatic()) != s.dev { + return nil, nil, errTunDevice + } + // Built before the caller is checked and sent only after, as over the + // relay. The check changes nothing yet; commit does, once the session + // proves itself. + reply, err := hs.WriteMessage(nil) + if err != nil { + return nil, nil, err + } + noise, err := hs.Split() + if err != nil { + return nil, nil, err + } + transport := appwire.NewTransport(noise) + if _, commit, err := t.enroll.Check(noise.RemoteStatic, payload); err != nil { + transport.Close() + return nil, nil, err + } else { + t.mu.Lock() + s.appStatic = noise.RemoteStatic + t.mu.Unlock() + if !t.send(cloudwire.Tun{Dev: s.dev, Sid: s.sid, D: reply}) { + transport.Close() + return nil, nil, errTunGone + } + return transport, commit, nil + } +} + +// promote makes a proved session live: it commits the pairing or the +// reconnect stamp, builds the handler from the grant as it stands now, takes +// the device's one slot and the table's live place. +func (t *Tunnel) promote(s *tunSession, transport *appwire.Transport, commit func() (appenroll.Grant, error)) error { + grant, err := commit() + if err != nil { + return err + } + caller := apiauth.Caller{ + Subject: subjectOf(grant.DeviceID), + Kind: apiauth.KindApp, + Role: grant.Role, + Scopes: appproto.ScopesForGrant(grant.Role, grant.ParentRole), + Epoch: grant.Epoch, + } + send := func(msg []byte, bulk bool) error { + return t.sendTun(cloudwire.Tun{Dev: s.dev, Sid: s.sid, D: msg}, bulk) + } + handler, err := t.build(tunSender{transport: transport, send: send}, caller, + liveGrant{enroll: t.enroll, deviceID: grant.DeviceID}) + if err != nil { + return err + } + sess := &session{transport: transport, handler: handler, appStatic: s.appStatic} + + t.mu.Lock() + if s.gen != t.gen { + // The uplink ended while the session proved itself. + t.mu.Unlock() + sess.close() + return errTunGone + } + delete(t.pending, tunKey{s.dev, s.sid}) + old := t.live[s.dev] + s.session = sess + t.live[s.dev] = s + t.mu.Unlock() + if old != nil && old != s { + // The phone hears that its older session ended. + t.remove(old, true) + } + // The device's one slot, whichever carrier held it before. Claiming it + // ends the older session, through the relay or through here. + sess.setRelease(t.slots.claim(grant.DeviceID, func() { t.remove(s, true) })) + t.log.Info("app session open through FTW Cloud") + return nil +} + +// tunSender encrypts one session's frames for the tunnel and says which are +// bulk, so the link paces those and sends lane 0 (commands, their answers +// and the ticks) first. A message the link dropped and counted is gone like +// any lost frame, and the session goes on; only a lost uplink fails it. +type tunSender struct { + transport *appwire.Transport + send func(msg []byte, bulk bool) error +} + +func (s tunSender) Send(frame []byte) error { + msg, err := s.transport.Encrypt(frame) + if err != nil { + return err + } + return s.send(msg, len(frame) > 1 && frame[1] == appwire.LaneBulk) +} + +// end closes the session once and frees its slot. It reports whether this +// call ended it. +func (s *tunSession) end() bool { + ended := false + s.once.Do(func() { + ended = true + close(s.stop) + if s.session != nil { + s.session.close() + } + }) + return ended +} + +// remove takes a session out of the table and ends it. tell sends the phone +// an end, when the uplink that carried it is still up. +func (t *Tunnel) remove(s *tunSession, tell bool) { + t.mu.Lock() + if t.live[s.dev] == s { + delete(t.live, s.dev) + } + if t.pending[tunKey{s.dev, s.sid}] == s { + delete(t.pending, tunKey{s.dev, s.sid}) + } + current := s.gen == t.gen + t.mu.Unlock() + if s.end() && tell && current { + t.send(cloudwire.Tun{Dev: s.dev, Sid: s.sid}) + } +} + +// send hands one tun to the link, ahead of bulk: a handshake's answer and +// an end. False means it did not leave. +func (t *Tunnel) send(m cloudwire.Tun) bool { return t.sendTun(m, false) == nil } + +// sendTun hands a session's message to the link. A counted drop is not an +// error: Noise transport survives it, and failing the session would lose a +// command id it holds. A lost uplink is. +func (t *Tunnel) sendTun(m cloudwire.Tun, bulk bool) error { + t.mu.Lock() + s := t.sender + t.mu.Unlock() + if s == nil { + return errTunGone + } + err := s.SendTun(m, bulk) + if dropped(err) { + return nil + } + return err +} + +// allowOpenLocked counts a handshake against its device's allowance for the +// minute, and against the box's. The caller holds t.mu. +func (t *Tunnel) allowOpenLocked(dev string) bool { + now := t.now() + total := 0 + for d, list := range t.opens { + kept := list[:0] + for _, at := range list { + if now.Sub(at) < time.Minute { + kept = append(kept, at) + } + } + if len(kept) == 0 { + delete(t.opens, d) + continue + } + t.opens[d] = kept + total += len(kept) + } + if total >= tunOpensPerMinute || len(t.opens[dev]) >= tunOpensPerDevice { + return false + } + t.opens[dev] = append(t.opens[dev], now) + return true +} + +// tick sends the one-second pulse to every live session. +func (t *Tunnel) tick() { + t.mu.Lock() + live := make([]*tunSession, 0, len(t.live)) + for _, s := range t.live { + live = append(live, s) + } + t.mu.Unlock() + for _, s := range live { + if err := s.session.handler.Tick(); err != nil { + t.log.Warn("app session failed on a tick", "err", err) + t.remove(s, true) + } + } +} + +// dropByAppKey ends every session a revoked key runs, live or pending, as +// the relay's table does. +func (t *Tunnel) dropByAppKey(pub []byte) int { + t.mu.Lock() + var gone []*tunSession + for _, s := range t.live { + if bytes.Equal(s.appStatic, pub) { + gone = append(gone, s) + } + } + for _, s := range t.pending { + if bytes.Equal(s.appStatic, pub) { + gone = append(gone, s) + } + } + t.mu.Unlock() + dropped := 0 + for _, s := range gone { + if s.session != nil { + _ = s.session.handler.Terminate(appproto.TerminateRevoked) + dropped++ + } + t.remove(s, true) + } + return dropped +} + +// -------------------------------------------------------------------------- +// One slot per device +// -------------------------------------------------------------------------- + +// slots holds each device's one session, whichever carrier it rides. A new +// session from a device ends its older one, so a phone that reconnects, a +// watch or another phone never pushes out a different device's session. +type slots struct { + mu sync.Mutex + n uint64 + held map[string]slot +} + +type slot struct { + id uint64 + end func() +} + +// claim gives dev's slot to a new session, whose end func closes it, and +// ends the session that held it. The returned func frees the slot if this +// session still holds it. A nil table claims nothing. +func (s *slots) claim(dev string, end func()) func() { + if s == nil { + return func() {} + } + s.mu.Lock() + if s.held == nil { + s.held = map[string]slot{} + } + s.n++ + id := s.n + old, had := s.held[dev] + s.held[dev] = slot{id: id, end: end} + s.mu.Unlock() + if had { + old.end() + } + return func() { + s.mu.Lock() + defer s.mu.Unlock() + if cur, ok := s.held[dev]; ok && cur.id == id { + delete(s.held, dev) + } + } +} diff --git a/go/internal/appuplink/tunnel_test.go b/go/internal/appuplink/tunnel_test.go new file mode 100644 index 000000000..2dc7e1c72 --- /dev/null +++ b/go/internal/appuplink/tunnel_test.go @@ -0,0 +1,590 @@ +package appuplink + +import ( + "bytes" + "context" + "log/slog" + "path/filepath" + "testing" + "time" + + "github.com/srcfl/ftw/go/internal/apiauth" + "github.com/srcfl/ftw/go/internal/appenroll" + "github.com/srcfl/ftw/go/internal/appproto" + "github.com/srcfl/ftw/go/internal/control" + "github.com/srcfl/ftw/go/protocol/appwire" + "github.com/srcfl/ftw/go/protocol/cloudwire" +) + +// capturedTun is the box's link to FTW Cloud, as the tunnel sees it. +type capturedTun struct { + ch chan cloudwire.Tun +} + +func (c *capturedTun) SendTun(t cloudwire.Tun, _ bool) error { + select { + case c.ch <- t: + return nil + default: + return errTunGone + } +} + +type tunRig struct { + enroll *appenroll.Identity + tunnel *Tunnel + slots *slots + out *capturedTun + modes *countingModes +} + +func newTunRig(t *testing.T) *tunRig { + t.Helper() + enroll, err := appenroll.LoadOrCreate(filepath.Join(t.TempDir(), "nova.key")) + if err != nil { + t.Fatal(err) + } + r := &tunRig{enroll: enroll, slots: &slots{}, out: &capturedTun{ch: make(chan cloudwire.Tun, 64)}, + modes: &countingModes{}} + r.tunnel = newTunnel(handlerOver(r.modes, appproto.NewCmdLog()), enroll, r.slots, slog.New(slog.DiscardHandler)) + r.tunnel.Attach(r.out) + ctx, cancel := context.WithCancel(context.Background()) + t.Cleanup(cancel) + go r.tunnel.Run(ctx) + return r +} + +// pair enrols a phone directly, as a pairing over any carrier would. +func (r *tunRig) pair(t *testing.T) *appClient { + t.Helper() + code, _, err := r.enroll.MintPairingCode(apiauth.RoleOwner, appenroll.PairingTTL) + if err != nil { + t.Fatal(err) + } + app, err := newAppClient() + if err != nil { + t.Fatal(err) + } + if _, err := r.enroll.Authorise(app.staticPublic(), code); err != nil { + t.Fatal(err) + } + return app +} + +// next is the next tun the box sent, failing after d. +func (r *tunRig) next(t *testing.T, d time.Duration) cloudwire.Tun { + t.Helper() + select { + case m := <-r.out.ch: + return m + case <-time.After(d): + t.Fatalf("the box sent no tun within %s", d) + return cloudwire.Tun{} + } +} + +// silent fails if the box sends a tun within d. +func (r *tunRig) silent(t *testing.T, d time.Duration) { + t.Helper() + select { + case m := <-r.out.ch: + t.Fatalf("the box sent %+v", m) + case <-time.After(d): + } +} + +// open runs the handshake for app under dev and sid, through the tunnel. +func (r *tunRig) open(t *testing.T, app *appClient, dev string, sid uint32, code []byte) { + t.Helper() + m1, err := app.message1(r.enroll.StaticKey().Public(), code) + if err != nil { + t.Fatal(err) + } + r.tunnel.Deliver(cloudwire.Tun{Dev: dev, Sid: sid, D: m1}) + reply := r.next(t, 3*time.Second) + if reply.Dev != dev || reply.Sid != sid || reply.Ended() { + t.Fatalf("the handshake's answer is %+v", reply) + } + if err := app.readMessage2(reply.D); err != nil { + t.Fatal(err) + } +} + +func (r *tunRig) send(t *testing.T, app *appClient, dev string, sid uint32, msgType string, body any) { + t.Helper() + raw, err := appproto.Marshal(body) + if err != nil { + t.Fatal(err) + } + frame, err := appwire.EncodeFrame(appwire.Frame{Lane: appwire.LaneControl, Bucket: 512, + Envelope: appwire.Envelope{T: msgType, B: raw}}) + if err != nil { + t.Fatal(err) + } + msg, err := app.encrypt(frame) + if err != nil { + t.Fatal(err) + } + r.tunnel.Deliver(cloudwire.Tun{Dev: dev, Sid: sid, D: msg}) +} + +// receive is the next frame of type want for app, skipping lane 0 ticks. +func (r *tunRig) receive(t *testing.T, app *appClient, want string) appwire.Frame { + t.Helper() + deadline := time.Now().Add(5 * time.Second) + for time.Now().Before(deadline) { + m := r.next(t, time.Until(deadline)) + if m.Ended() { + t.Fatalf("the box ended the session while waiting for %s", want) + } + plain, err := app.decrypt(m.D) + if err != nil { + t.Fatal(err) + } + f, err := appwire.DecodeFrame(plain) + if err != nil { + t.Fatal(err) + } + if f.Envelope.T == want { + return f + } + } + t.Fatalf("no %s", want) + return appwire.Frame{} +} + +func (r *tunRig) hello(t *testing.T, app *appClient, dev string, sid uint32) { + t.Helper() + r.send(t, app, dev, sid, appproto.MsgHello, appproto.Hello{ + Proto: appproto.ProtoRange{Min: 0, Max: appproto.ProtoMax}, + App: appproto.AppInfo{Build: "test", UA: "ios"}, + }) + r.receive(t, app, appproto.MsgHelloOK) +} + +func setModeCmd(id string) appproto.Cmd { + return appproto.Cmd{ + CmdID: id, Op: appproto.OpSetMode, + Args: map[string]any{"mode": string(control.ModeSelfConsumption)}, + NotValidAfterMs: 10 * 60 * 1000, Expect: appproto.Expect{Rev: 1}, + } +} + +// A paired phone's session runs through the tunnel as over the relay: the +// same handshake, the same hello and a command the box acts on and answers. +func TestATunnelSessionAnswersACommand(t *testing.T) { + r := newTunRig(t) + app := r.pair(t) + dev := deviceIDOf(app.staticPublic()) + r.open(t, app, dev, 7, nil) + r.hello(t, app, dev, 7) + + r.send(t, app, dev, 7, appproto.MsgCmd, setModeCmd("0192f2a0-7c1e-7000-8000-00000000a001")) + r.receive(t, app, appproto.MsgCmdAck) + var res appproto.CmdResult + if err := appproto.Unmarshal(r.receive(t, app, appproto.MsgCmdResult).Envelope.B, &res); err != nil { + t.Fatal(err) + } + if res.State != appproto.CmdApplied || r.modes.count() != 1 { + t.Fatalf("result %q after %d mode changes", res.State, r.modes.count()) + } +} + +// The service names the device, and the box checks it against the key the +// handshake proved. A session put under another device's id opens nothing +// and gets no answer. +func TestAForgedDevGetsNothing(t *testing.T) { + r := newTunRig(t) + app := r.pair(t) + other := r.pair(t) + m1, err := app.message1(r.enroll.StaticKey().Public(), nil) + if err != nil { + t.Fatal(err) + } + r.tunnel.Deliver(cloudwire.Tun{Dev: deviceIDOf(other.staticPublic()), Sid: 1, D: m1}) + r.silent(t, 500*time.Millisecond) + if r.tunnel.count() != 0 { + t.Fatal("a session opened under another device's id") + } +} + +// A key the box never paired gets nothing, through the tunnel as over the +// relay, and neither does a wrong pairing code. +func TestAStrangerThroughTheTunnelGetsNothing(t *testing.T) { + r := newTunRig(t) + if _, _, err := r.enroll.MintPairingCode(apiauth.RoleOwner, appenroll.PairingTTL); err != nil { + t.Fatal(err) + } + for _, code := range [][]byte{nil, make([]byte, appenroll.PairingCodeBytes)} { + app, err := newAppClient() + if err != nil { + t.Fatal(err) + } + m1, err := app.message1(r.enroll.StaticKey().Public(), code) + if err != nil { + t.Fatal(err) + } + r.tunnel.Deliver(cloudwire.Tun{Dev: deviceIDOf(app.staticPublic()), Sid: 1, D: m1}) + r.silent(t, 400*time.Millisecond) + } + if r.tunnel.count() != 0 || r.enroll.AuthorisedCount() != 0 { + t.Fatal("a stranger got a session or a pairing") + } +} + +// A new phone pairs through the tunnel with the code from the QR, inside its +// handshake, exactly as over the relay. +func TestPairingThroughTheTunnel(t *testing.T) { + r := newTunRig(t) + code, _, err := r.enroll.MintPairingCode(apiauth.RoleViewer, appenroll.PairingTTL) + if err != nil { + t.Fatal(err) + } + app, err := newAppClient() + if err != nil { + t.Fatal(err) + } + dev := cloudwire.DeviceID(app.staticPublic()) + r.open(t, app, dev, 3, code) + r.hello(t, app, dev, 3) + if g, ok := r.enroll.GrantFor(dev); !ok || g.Role != apiauth.RoleOwner { + t.Fatalf("the first phone through the tunnel holds %+v", g) + } +} + +// One slot per device: a new session from the same device ends its older one +// and tells the phone, and a session of another device stays. +func TestOneSessionPerDevice(t *testing.T) { + r := newTunRig(t) + app := r.pair(t) + other := r.pair(t) + dev, otherDev := deviceIDOf(app.staticPublic()), deviceIDOf(other.staticPublic()) + r.open(t, other, otherDev, 1, nil) + r.hello(t, other, otherDev, 1) + r.open(t, app, dev, 1, nil) + r.hello(t, app, dev, 1) + + again := &appClient{static: app.static} + m1, err := again.message1(r.enroll.StaticKey().Public(), nil) + if err != nil { + t.Fatal(err) + } + r.tunnel.Deliver(cloudwire.Tun{Dev: dev, Sid: 2, D: m1}) + reply := r.next(t, 3*time.Second) + if reply.Sid != 2 || reply.Ended() { + t.Fatalf("got %+v", reply) + } + if err := again.readMessage2(reply.D); err != nil { + t.Fatal(err) + } + // The new session proves itself with its first message, and only then + // takes the device's place. + r.send(t, again, dev, 2, appproto.MsgHello, appproto.Hello{Proto: appproto.ProtoRange{Max: appproto.ProtoMax}, + App: appproto.AppInfo{Build: "test", UA: "ios"}}) + sawEnd := false + for !sawEnd { + m := r.next(t, 3*time.Second) + sawEnd = m.Dev == dev && m.Sid == 1 && m.Ended() + } + if r.tunnel.count() != 2 { + t.Fatalf("%d sessions, want the new one and the other device's", r.tunnel.count()) + } +} + +// A relay session and a tunnel session of one device share its slot: the +// newer ends the older, whichever carrier either rides. +func TestTheRelayAndTheTunnelShareADevicesSlot(t *testing.T) { + r := newTunRig(t) + app := r.pair(t) + dev := deviceIDOf(app.staticPublic()) + streams := relayTable(r.slots, r.enroll) + relayOpen(t, streams, r.enroll, &appClient{static: app.static}, nil) + r.open(t, app, dev, 9, nil) + r.hello(t, app, dev, 9) + if streams.count() != 0 { + t.Fatal("the relay session outlived the device's tunnel session") + } + + relayOpen(t, streams, r.enroll, &appClient{static: app.static}, nil) + if end := r.next(t, 3*time.Second); end.Dev != dev || end.Sid != 9 || !end.Ended() { + t.Fatalf("the tunnel session ended with %+v", end) + } + if r.tunnel.count() != 0 || streams.count() != 1 { + t.Fatalf("%d tunnel and %d relay sessions", r.tunnel.count(), streams.count()) + } +} + +// A message for a session the box does not hold gets an end, so the phone +// starts again at once instead of waiting. +func TestAnUnknownSessionGetsAnEnd(t *testing.T) { + r := newTunRig(t) + app := r.pair(t) + dev := deviceIDOf(app.staticPublic()) + r.tunnel.Deliver(cloudwire.Tun{Dev: dev, Sid: 4, D: bytes.Repeat([]byte{1}, 600)}) + if m := r.next(t, 3*time.Second); m.Dev != dev || m.Sid != 4 || !m.Ended() { + t.Fatalf("answer %+v", m) + } +} + +// The uplink ending ends every session, and a message queued from it opens +// nothing afterwards. +func TestDropEndsEverySession(t *testing.T) { + r := newTunRig(t) + app := r.pair(t) + dev := deviceIDOf(app.staticPublic()) + r.open(t, app, dev, 1, nil) + r.tunnel.Drop() + if r.tunnel.count() != 0 { + t.Fatal("a session outlived its uplink") + } + r.silent(t, 300*time.Millisecond) +} + +// The phone ending its session, with an empty d, ends it at the box. +func TestAnEmptyDEndsTheSession(t *testing.T) { + r := newTunRig(t) + app := r.pair(t) + dev := deviceIDOf(app.staticPublic()) + r.open(t, app, dev, 1, nil) + r.tunnel.Deliver(cloudwire.Tun{Dev: dev, Sid: 1}) + deadline := time.Now().Add(3 * time.Second) + for r.tunnel.count() != 0 { + if time.Now().After(deadline) { + t.Fatal("the phone's end left the session open") + } + time.Sleep(10 * time.Millisecond) + } +} + +// Revoking a phone ends its tunnel session too. +func TestARevokeEndsATunnelSession(t *testing.T) { + r := newTunRig(t) + enrolOwner(t, r.enroll) + app := r.pair(t) + dev := deviceIDOf(app.staticPublic()) + r.open(t, app, dev, 1, nil) + r.hello(t, app, dev, 1) + removed, err := r.enroll.Revoke(dev, appenroll.AtTheBox) + if err != nil { + t.Fatal(err) + } + if n := r.tunnel.dropByAppKey(removed[0].Key); n != 1 || r.tunnel.count() != 0 { + t.Fatalf("dropped %d, %d left", n, r.tunnel.count()) + } +} + +// One device cannot spend the handshakes every other phone needs. +func TestOneDeviceCannotStarveTheOthersOfHandshakes(t *testing.T) { + r := newTunRig(t) + busy := r.pair(t) + busyDev := deviceIDOf(busy.staticPublic()) + for i := range tunOpensPerDevice + 5 { + again := &appClient{static: busy.static} + m1, _ := again.message1(r.enroll.StaticKey().Public(), nil) + r.tunnel.Deliver(cloudwire.Tun{Dev: busyDev, Sid: uint32(100 + i), D: m1}) + } + // Drain what the busy device got: its answers, and the ends of the + // sessions each new one replaced. + deadline := time.After(2 * time.Second) + answers := 0 +drain: + for { + select { + case m := <-r.out.ch: + if !m.Ended() { + answers++ + } + case <-deadline: + break drain + } + } + if answers != tunOpensPerDevice { + t.Fatalf("the busy device got %d handshakes answered, want %d", answers, tunOpensPerDevice) + } + other := r.pair(t) + r.open(t, other, deviceIDOf(other.staticPublic()), 1, nil) +} + +// relayTable is a relay connection's session table whose writes a test can +// read back, one message at a time. +func relayTable(sl *slots, enroll *appenroll.Identity) *sessions { + out := make(chan []byte, 16) + return &sessions{ + slots: sl, build: newHandler, enroll: enroll, static: enroll.StaticKey(), log: slog.New(slog.DiscardHandler), + write: func(m []byte) error { + select { + case out <- append([]byte(nil), m...): + default: + } + return nil + }, + testOut: out, + } +} + +// relayOpen runs a phone's handshake on a relay table and proves it with a +// hello, so the session is live. +func relayOpen(t *testing.T, streams *sessions, enroll *appenroll.Identity, app *appClient, code []byte) { + t.Helper() + m1, err := app.message1(enroll.StaticKey().Public(), code) + if err != nil { + t.Fatal(err) + } + if err := streams.open(m1); err != nil { + t.Fatal(err) + } + if err := app.readMessage2(<-streams.testOut); err != nil { + t.Fatal(err) + } + raw, _ := appproto.Marshal(appproto.Hello{Proto: appproto.ProtoRange{Max: appproto.ProtoMax}, App: appproto.AppInfo{Build: "t", UA: "pwa"}}) + frame, _ := appwire.EncodeFrame(appwire.Frame{Lane: appwire.LaneControl, Bucket: 512, Envelope: appwire.Envelope{T: appproto.MsgHello, B: raw}}) + msg, _ := app.encrypt(frame) + streams.deliver(context.Background(), msg) + // The hello_ok. + <-streams.testOut +} + +// A message 1 replayed under a new sid, as the service could, gets an +// answer and nothing else: the live session goes on, and no stamp or code +// moves. +func TestAReplayedMessage1LeavesTheLiveSession(t *testing.T) { + r := newTunRig(t) + app := r.pair(t) + dev := deviceIDOf(app.staticPublic()) + m1, err := app.message1(r.enroll.StaticKey().Public(), nil) + if err != nil { + t.Fatal(err) + } + r.tunnel.Deliver(cloudwire.Tun{Dev: dev, Sid: 1, D: m1}) + reply := r.next(t, 3*time.Second) + if err := app.readMessage2(reply.D); err != nil { + t.Fatal(err) + } + r.hello(t, app, dev, 1) + seen := r.enroll.Devices()[0].LastSeenMs + + // The replay, under a sid of the replayer's choosing. + r.tunnel.Deliver(cloudwire.Tun{Dev: dev, Sid: 99, D: m1}) + if m := r.next(t, 3*time.Second); m.Sid != 99 || m.Ended() { + t.Fatalf("the replay got %+v", m) + } + r.silent(t, 300*time.Millisecond) + if r.tunnel.count() != 1 || r.enroll.Devices()[0].LastSeenMs != seen { + t.Fatal("a replayed message 1 changed the live session or the stamp") + } + r.send(t, app, dev, 1, appproto.MsgCmd, setModeCmd("0192f2a0-7c1e-7000-8000-00000000d001")) + r.receive(t, app, appproto.MsgCmdAck) +} + +// The same on the relay: a replayed message 1 takes no device's place. +func TestAReplayedMessage1LeavesTheLiveRelaySession(t *testing.T) { + r := newTunRig(t) + app := r.pair(t) + streams := relayTable(r.slots, r.enroll) + m1, err := app.message1(r.enroll.StaticKey().Public(), nil) + if err != nil { + t.Fatal(err) + } + if err := streams.open(m1); err != nil { + t.Fatal(err) + } + if err := app.readMessage2(<-streams.testOut); err != nil { + t.Fatal(err) + } + raw, _ := appproto.Marshal(appproto.Hello{Proto: appproto.ProtoRange{Max: appproto.ProtoMax}, App: appproto.AppInfo{Build: "t", UA: "pwa"}}) + frame, _ := appwire.EncodeFrame(appwire.Frame{Lane: appwire.LaneControl, Bucket: 512, Envelope: appwire.Envelope{T: appproto.MsgHello, B: raw}}) + msg, _ := app.encrypt(frame) + streams.deliver(context.Background(), msg) + <-streams.testOut + if streams.count() != 1 { + t.Fatal("the phone's session is not live") + } + // Replayed, twice. + for range 2 { + if err := streams.open(m1); err != nil { + t.Fatal(err) + } + <-streams.testOut + } + if streams.count() != 1 { + t.Fatalf("%d live sessions after replays", streams.count()) + } + frame, _ = appwire.EncodeFrame(appwire.Frame{Lane: appwire.LaneControl, Bucket: 512, Envelope: appwire.Envelope{T: appproto.MsgHello, B: raw}}) + msg, _ = app.encrypt(frame) + streams.deliver(context.Background(), msg) + select { + case <-streams.testOut: + case <-time.After(3 * time.Second): + t.Fatal("the live relay session stopped answering after a replay") + } +} + +// A dropped tun does not end the session: the next command still gets its +// ack and its result. +func TestADroppedTunDoesNotEndTheSession(t *testing.T) { + r := newTunRig(t) + app := r.pair(t) + dev := deviceIDOf(app.staticPublic()) + r.open(t, app, dev, 1, nil) + r.hello(t, app, dev, 1) + drops := &dropOnce{inner: r.out} + r.tunnel.Attach(drops) + r.send(t, app, dev, 1, appproto.MsgCmd, setModeCmd("0192f2a0-7c1e-7000-8000-00000000d101")) + // The ack was dropped; the result comes. + r.receive(t, app, appproto.MsgCmdResult) + if r.tunnel.count() != 1 { + t.Fatal("a dropped frame ended the session") + } + r.send(t, app, dev, 1, appproto.MsgCmd, setModeCmd("0192f2a0-7c1e-7000-8000-00000000d102")) + r.receive(t, app, appproto.MsgCmdAck) + r.receive(t, app, appproto.MsgCmdResult) +} + +// dropOnce loses the first message, as a full queue would, and counts it. +type dropOnce struct { + inner *capturedTun + done bool +} + +type countedDrop struct{} + +func (countedDrop) Error() string { return "dropped" } +func (countedDrop) Dropped() bool { return true } + +func (d *dropOnce) SendTun(t cloudwire.Tun, bulk bool) error { + if !d.done { + d.done = true + return countedDrop{} + } + return d.inner.SendTun(t, bulk) +} + +// Revoking a phone through the uplink reaches its tunnel session. +func TestDropSessionsByAppKeyReachesTheTunnel(t *testing.T) { + enroll, err := appenroll.LoadOrCreate(filepath.Join(t.TempDir(), "nova.key")) + if err != nil { + t.Fatal(err) + } + u, err := New(Options{Endpoint: "ws://127.0.0.1:1", Enroll: enroll, Handler: newHandler, Logger: slog.New(slog.DiscardHandler)}) + if err != nil { + t.Fatal(err) + } + out := &capturedTun{ch: make(chan cloudwire.Tun, 16)} + u.Tunnel().Attach(out) + ctx, cancel := context.WithCancel(context.Background()) + defer cancel() + go u.Tunnel().Run(ctx) + enrolOwner(t, enroll) + r := &tunRig{enroll: enroll, tunnel: u.Tunnel(), out: out} + app := r.pair(t) + dev := deviceIDOf(app.staticPublic()) + r.open(t, app, dev, 1, nil) + r.hello(t, app, dev, 1) + removed, err := enroll.Revoke(dev, appenroll.AtTheBox) + if err != nil { + t.Fatal(err) + } + if n := u.DropSessionsByAppKey(removed[0].Key); n != 1 { + t.Fatalf("dropped %d", n) + } +} diff --git a/go/internal/appuplink/watch_test.go b/go/internal/appuplink/watch_test.go index de6ae5dd8..7303ad0f2 100644 --- a/go/internal/appuplink/watch_test.go +++ b/go/internal/appuplink/watch_test.go @@ -10,7 +10,7 @@ import ( ) // A watch reaches the box through the exchange and never through the stream, -// so it can never take one of a phone's four places. Its key on the stream +// so it can never take a phone's place. Its key on the stream // gets the silence a stranger gets. func TestAWatchKeyGetsNoStream(t *testing.T) { r := newRig(t, 481234) diff --git a/go/internal/cloudlink/conn.go b/go/internal/cloudlink/conn.go index 4559c59a6..a27e7238c 100644 --- a/go/internal/cloudlink/conn.go +++ b/go/internal/cloudlink/conn.go @@ -69,7 +69,7 @@ func (l *Link) once(ctx context.Context, snap grantSnapshot) (time.Duration, err l.attempt = 0 } l.mu.Unlock() - if errors.Is(err, errPurged) || errors.Is(err, errClosed) { + if errors.Is(err, errLinkEnded) || errors.Is(err, errClosed) { return 0, nil } return l.backoff(), err @@ -88,8 +88,22 @@ func (l *Link) backoff() time.Duration { return time.Duration(n.Int64()) } -// errPurged ends a connection after the service confirmed a deletion. -var errPurged = errors.New("cloudlink: purge confirmed") +// errLinkEnded ends a connection after the service confirmed that the home +// is gone. +var errLinkEnded = errors.New("cloudlink: the end of the link is confirmed") + +// The tunnel's two queues, apart from the readable copy. Lane 0, a phone's +// commands, their answers and the one-second ticks, goes first and is never +// held. Bulk is paced under the service's limits and waits its turn. When a +// queue is full the newest message is dropped: Noise transport carries its +// own counter, so a lost message costs the phone one frame. +const ( + tunQueue = 64 + tunBulkQueue = 256 + // tunPace is the share of the service's limits bulk is paced to, so + // the service never drops for a box that keeps to it. + tunPace = 0.9 +) // serve runs one connection: the handshake, then the reader here and the // sender on its own goroutine. @@ -105,7 +119,7 @@ func (l *Link) serve(ctx context.Context, conn *websocket.Conn, snap grantSnapsh if err != nil { return err } - conn.SetReadLimit(cloudwire.MaxServiceFrame) + conn.SetReadLimit(cloudwire.MaxTunFrame) // 1. challenge _ = conn.SetReadDeadline(time.Now().Add(handshakeTimeout)) @@ -129,7 +143,7 @@ func (l *Link) serve(ctx context.Context, conn *websocket.Conn, snap grantSnapsh hello := cloudwire.Hello{ Header: l.header(), V: cloudwire.Version, PK: snap.pk, Nonce: ch.Nonce, EKM: ekm, BoxNonce: boxNonce, Build: l.opts.Build, Clock: l.clockName(), - Caps: snap.caps(), RosterSeq: rosterSeq, GrantID: snap.g.ID, + Caps: l.caps(snap), RosterSeq: rosterSeq, GrantID: snap.grantID(), } l.mu.Unlock() w := &writer{conn: conn, link: l} @@ -155,10 +169,24 @@ func (l *Link) serve(ctx context.Context, conn *websocket.Conn, snap grantSnapsh l.bumpRoster(ok.RosterSeq) } + tunOut, tunBulk := make(chan []byte, tunQueue), make(chan []byte, tunBulkQueue) l.mu.Lock() l.connected = true + if l.opts.Tunnel != nil { + l.tunOut, l.tunBulk = tunOut, tunBulk + } l.mu.Unlock() l.log.Info("FTW Cloud uplink connected", "home", snap.home) + defer func() { + l.mu.Lock() + l.tunOut, l.tunBulk = nil, nil + l.mu.Unlock() + // Every session the uplink carried ends with it. The phone starts + // again on the next one. + if l.opts.Tunnel != nil { + l.opts.Tunnel.Drop() + } + }() events := make(chan cloudwire.Envelope, 4) readErr := make(chan error, 1) @@ -169,17 +197,27 @@ func (l *Link) serve(ctx context.Context, conn *websocket.Conn, snap grantSnapsh go func() { sendErr <- w.run(ctx, events) }() + tunErr := make(chan error, 1) + go func() { + tunErr <- w.tunnel(ctx, tunOut, tunBulk) + }() + // The first to end ends the others: cancel closes the socket under the + // reader, and the writers stop on the context. + pending := []chan error{readErr, sendErr, tunErr} select { - case err := <-readErr: - cancel() - <-sendErr - return err - case err := <-sendErr: - cancel() - <-readErr - return err + case err = <-readErr: + pending = pending[1:] + case err = <-sendErr: + pending = []chan error{readErr, tunErr} + case err = <-tunErr: + pending = pending[:2] + } + cancel() + for _, ch := range pending { + <-ch } + return err } // exporterValue is the TLS 1.3 exporter value both ends bind the handshake to. @@ -211,8 +249,9 @@ func (l *Link) clockName() string { return cloudwire.ClockNone } -// readEnvelope reads one binary frame. A text message, a lane 0 frame or a -// frame over 4 KiB is the service breaking the protocol. +// readEnvelope reads one binary frame. A text message, a lane 0 frame, a tun +// frame over MaxTunFrame or any other over 4 KiB is the service breaking the +// protocol. func readEnvelope(conn *websocket.Conn) (cloudwire.Envelope, error) { kind, msg, err := conn.ReadMessage() if err != nil { @@ -225,7 +264,7 @@ func parseDownlink(kind int, msg []byte) (cloudwire.Envelope, error) { if kind != websocket.BinaryMessage { return cloudwire.Envelope{}, fmt.Errorf("%w: a text message", errService) } - if len(msg) > cloudwire.MaxServiceFrame { + if len(msg) > cloudwire.MaxTunFrame { return cloudwire.Envelope{}, fmt.Errorf("%w: %d bytes", errService, len(msg)) } payload, err := cloudwire.DecodeFrame(msg) @@ -239,14 +278,17 @@ func parseDownlink(kind int, msg []byte) (cloudwire.Envelope, error) { if !slices.Contains(cloudwire.Downlink, env.T) { return cloudwire.Envelope{}, fmt.Errorf("%w: %q is not a message the service may send", errService, env.T) } + if env.T != cloudwire.MsgTun && len(msg) > cloudwire.MaxServiceFrame { + return cloudwire.Envelope{}, fmt.Errorf("%w: a %d-byte %s", errService, len(msg), env.T) + } return env, nil } // read is the downlink. After the handshake the service may send watch, -// resync and purged; challenge and hello_ok a second time are refused like -// any stranger. watch and resync go to the sender, which owns the cadence. -// A purge receipt is taken here, before the service hangs up, so the box -// never misses one it was sent. +// resync, purged and tun; challenge and hello_ok a second time are refused +// like any stranger. watch and resync go to the sender, which owns the +// cadence. A purge receipt is taken here, before the service hangs up, so the +// box never misses one it was sent. A tun goes to the Tunnel unread. func (l *Link) read(conn *websocket.Conn, snap grantSnapshot, events chan<- cloudwire.Envelope) error { _ = conn.SetReadDeadline(time.Now().Add(readTimeout)) conn.SetPingHandler(func(data string) error { @@ -262,7 +304,15 @@ func (l *Link) read(conn *websocket.Conn, snap grantSnapshot, events chan<- clou switch env.T { case cloudwire.MsgPurged: if l.acceptPurged(snap, env) { - return errPurged + return errLinkEnded + } + case cloudwire.MsgTun: + var t cloudwire.Tun + if len(env.Sig) != 0 || cloudwire.Unmarshal(env.B, &t) != nil || len(t.D) > cloudwire.TunDataMax { + return fmt.Errorf("%w: a tun the box cannot read", errService) + } + if l.opts.Tunnel != nil { + l.opts.Tunnel.Deliver(t) } case cloudwire.MsgWatch, cloudwire.MsgResync: events <- env @@ -299,12 +349,7 @@ func (w *writer) send(snap grantSnapshot, t string, body any) error { return err } - w.mu.Lock() - defer w.mu.Unlock() - if err := w.conn.SetWriteDeadline(time.Now().Add(writeTimeout)); err != nil { - return err - } - if err := w.conn.WriteMessage(websocket.BinaryMessage, frame); err != nil { + if err := w.write(frame); err != nil { return err } l := w.link @@ -315,3 +360,154 @@ func (w *writer) send(snap grantSnapshot, t string, body any) error { l.mu.Unlock() return nil } + +// Why a tun did not leave. ErrNoUplink means none will until the box +// reconnects; ErrTunDropped means this one was lost, to a full queue or the +// tunnel's own budget, and was counted. Noise transport survives a lost +// message, so a session goes on after ErrTunDropped. +var ( + ErrNoUplink = errors.New("cloudlink: no uplink to FTW Cloud") + ErrTunDropped error = tunDropped{} +) + +// tunDropped says, through Dropped, that one message was lost and counted, +// so a caller that may not import this package can tell it from a lost +// uplink. +type tunDropped struct{} + +func (tunDropped) Error() string { return "cloudlink: a tun was dropped" } +func (tunDropped) Dropped() bool { return true } + +// SendTun queues one tun for the uplink. bulk says it carries a lane 1 frame, +// which is paced; anything else goes first. It never blocks. nil means the +// message is queued. +func (l *Link) SendTun(t cloudwire.Tun, bulk bool) error { + frame, err := tunFrame(t) + l.mu.Lock() + defer l.mu.Unlock() + q := l.tunOut + if bulk { + q = l.tunBulk + } + switch { + case q == nil: + l.tunDropped++ + return ErrNoUplink + case err != nil || l.tunBudget.total(l.uptimeMs()) >= tunBudgetBytes: + l.tunDropped++ + return ErrTunDropped + } + select { + case q <- frame: + return nil + default: + l.tunDropped++ + return ErrTunDropped + } +} + +func tunFrame(t cloudwire.Tun) ([]byte, error) { + if len(t.D) > cloudwire.TunDataMax { + return nil, ErrTunDropped + } + body, err := cloudwire.Marshal(t) + if err != nil { + return nil, err + } + raw, err := cloudwire.Envelope{T: cloudwire.MsgTun, B: body}.Encode() + if err != nil { + return nil, err + } + return cloudwire.EncodeFrame(raw) +} + +// pacer keeps bulk under the service's tun limits: a message bucket and a +// byte bucket at tunPace of them. Taking may leave either in debt; the wait +// is how long until it is paid. +type pacer struct { + msgs, bytes float64 + at time.Time +} + +func newPacer(now time.Time) *pacer { + return &pacer{msgs: tunPace * cloudwire.TunMsgBurst, bytes: tunPace * cloudwire.TunByteBurst, at: now} +} + +func (p *pacer) take(n int, now time.Time) time.Duration { + dt := now.Sub(p.at).Seconds() + p.at = now + p.msgs = min(tunPace*cloudwire.TunMsgBurst, p.msgs+dt*tunPace*cloudwire.TunMsgsPerSecond) - 1 + p.bytes = min(tunPace*cloudwire.TunByteBurst, p.bytes+dt*tunPace*cloudwire.TunBytesPerSecond) - float64(n) + wait := max(-p.msgs/(tunPace*cloudwire.TunMsgsPerSecond), -p.bytes/(tunPace*cloudwire.TunBytesPerSecond), 0) + return time.Duration(wait * float64(time.Second)) +} + +// tunnel writes queued tun frames as they come, beside the readable copy, +// so a phone's session never waits for the copy's cadence. Lane 0 goes +// first and at once, even while bulk waits for the pacer. It counts both +// against the tunnel's budget, never the copy's. +func (w *writer) tunnel(ctx context.Context, out, bulk <-chan []byte) error { + l := w.link + pace := newPacer(time.Now()) + send := func(frame []byte) error { + if err := w.write(frame); err != nil { + return err + } + l.mu.Lock() + l.tunBudget.add(l.uptimeMs(), int64(len(frame))) + l.mu.Unlock() + return nil + } + for { + select { + case frame := <-out: + pace.take(len(frame), time.Now()) + if err := send(frame); err != nil { + return err + } + continue + default: + } + select { + case <-ctx.Done(): + return ctx.Err() + case frame := <-out: + pace.take(len(frame), time.Now()) + if err := send(frame); err != nil { + return err + } + case frame := <-bulk: + if wait := pace.take(len(frame), time.Now()); wait > 0 { + timer := time.NewTimer(wait) + held: + for { + select { + case <-ctx.Done(): + timer.Stop() + return ctx.Err() + case urgent := <-out: + if err := send(urgent); err != nil { + timer.Stop() + return err + } + case <-timer.C: + break held + } + } + } + if err := send(frame); err != nil { + return err + } + } + } +} + +// write sends one frame under the socket's one write lock. +func (w *writer) write(frame []byte) error { + w.mu.Lock() + defer w.mu.Unlock() + if err := w.conn.SetWriteDeadline(time.Now().Add(writeTimeout)); err != nil { + return err + } + return w.conn.WriteMessage(websocket.BinaryMessage, frame) +} diff --git a/go/internal/cloudlink/downlink_test.go b/go/internal/cloudlink/downlink_test.go index c99447552..ba43c2f45 100644 --- a/go/internal/cloudlink/downlink_test.go +++ b/go/internal/cloudlink/downlink_test.go @@ -40,6 +40,15 @@ func TestDownlinkIsAClosedSet(t *testing.T) { if _, err := parseDownlink(websocket.BinaryMessage, big); err == nil { t.Error("a frame over 4 KiB was accepted") } + // A tun may carry the largest bulk frame, and no more. + full := frameOf(t, cloudwire.MsgTun, cloudwire.Tun{Dev: "q5Lw0Zb1", Sid: 1, D: make([]byte, cloudwire.TunDataMax)}) + if _, err := parseDownlink(websocket.BinaryMessage, full); err != nil { + t.Errorf("a full tun was refused: %v", err) + } + over := frameOf(t, cloudwire.MsgTun, cloudwire.Tun{Dev: "q5Lw0Zb1", Sid: 1, D: make([]byte, cloudwire.MaxTunFrame)}) + if _, err := parseDownlink(websocket.BinaryMessage, over); err == nil { + t.Error("a tun over MaxTunFrame was accepted") + } lane0 := frameOf(t, cloudwire.MsgWatch, cloudwire.Watch{Hz: 1}) lane0[1] = 0 if _, err := parseDownlink(websocket.BinaryMessage, lane0); err == nil { @@ -48,11 +57,12 @@ func TestDownlinkIsAClosedSet(t *testing.T) { } // Whatever the service sends, the box either refuses it or reads one of the -// five types, and never panics. +// six types within its size, and never panics. func FuzzDownlink(f *testing.F) { for _, typ := range append(slices.Clone(cloudwire.Downlink), "cmd") { f.Add(true, frameOf(f, typ, map[string]any{"hz": 1, "what": []string{"full"}})) } + f.Add(true, frameOf(f, cloudwire.MsgTun, cloudwire.Tun{Dev: "q5Lw0Zb1", Sid: 7, D: []byte{1, 2, 3}})) f.Add(false, []byte("ready")) f.Fuzz(func(t *testing.T, binary bool, msg []byte) { kind := websocket.TextMessage @@ -63,12 +73,17 @@ func FuzzDownlink(f *testing.F) { if err != nil { return } - if !slices.Contains(cloudwire.Downlink, env.T) || len(msg) > cloudwire.MaxServiceFrame { + limit := cloudwire.MaxServiceFrame + if env.T == cloudwire.MsgTun { + limit = cloudwire.MaxTunFrame + } + if !slices.Contains(cloudwire.Downlink, env.T) || len(msg) > limit { t.Fatalf("accepted %q in %d bytes", env.T, len(msg)) } - // The bodies the sender reads must not panic either. + // The bodies the sender and the tunnel read must not panic either. _ = cloudwire.Unmarshal(env.B, &cloudwire.Watch{}) _ = cloudwire.Unmarshal(env.B, &cloudwire.Resync{}) _ = cloudwire.Unmarshal(env.B, &cloudwire.Purged{}) + _ = cloudwire.Unmarshal(env.B, &cloudwire.Tun{}) }) } diff --git a/go/internal/cloudlink/grant.go b/go/internal/cloudlink/grant.go index fcff0073e..e5313602e 100644 --- a/go/internal/cloudlink/grant.go +++ b/go/internal/cloudlink/grant.go @@ -18,14 +18,28 @@ import ( "github.com/srcfl/ftw/go/protocol/cloudwire" ) -// The grant lives in the box's state store, not in the config document, so a -// whole-document config write cannot reset it. A missing record reads as off. +// The link and the grant live in the box's state store, not in the config +// document, so a whole-document config write cannot reset them. A missing +// record reads as off. const ( + linkKey = "cloud/link_v1" grantKey = "cloud/grant_v1" rosterKey = "cloud/roster_v1" purgedKey = "cloud/purged_v1" ) +// linkRecord is the box's link to FTW Cloud: which cloud.key it holds. The +// link carries the roster and the phones' tunnels, and a grant adds the +// readable copy on top. +type linkRecord struct { + // KeyFp is SHA-256 of cloud.key's public point. A record whose + // fingerprint does not match the key on disk reads as off. + KeyFp string `json:"keyFp"` + // Ending means the box asked the service to delete the home and waits + // for the receipt. The key stays until it arrives. + Ending bool `json:"ending,omitempty"` +} + // KV is the part of the state store the grant needs. type KV interface { LoadConfig(key string) (string, bool) @@ -54,8 +68,9 @@ type grantRecord struct { StatsID []byte `json:"statsId,omitempty"` // StatsPolicy names the consent text the owner read for statistics. StatsPolicy string `json:"statsPolicy,omitempty"` - // Revoked means the owner turned FTW Cloud off and the service has not - // yet confirmed the deletion. The key stays until it has. + // Revoked means the owner turned the readable copy off and the service + // has not yet confirmed that the copy is gone. The record stays until it + // has; the link and its key stay after. Revoked bool `json:"revoked,omitempty"` RevokedAtMs int64 `json:"revokedAtMs,omitempty"` } @@ -67,6 +82,8 @@ func (g *grantRecord) has(scope string) bool { // receipt is the service's purge receipt, kept so the box can say when the // copy was erased. type receipt struct { + // What is the receipt's: copy for a grant, home for the link. + What string `json:"what,omitempty"` AtMs int64 `json:"atMs"` GrantID []byte `json:"grantId"` } @@ -132,8 +149,8 @@ func checkScopes(scopes []string) ([]string, error) { // -------------------------------------------------------------------------- // The key file is PEM SEC 1, 0600, beside state.db. Backups skip it: a box -// restored onto other hardware has FTW Cloud off until an owner turns it on -// again, under a new home id. +// restored onto other hardware links again under a new home id, and has the +// readable copy off until an owner turns it on again. func loadKey(path string) (*ecdsa.PrivateKey, error) { raw, err := os.ReadFile(path) @@ -154,6 +171,10 @@ func loadKey(path string) (*ecdsa.PrivateKey, error) { return key, nil } +// createKey makes cloud.key, and never replaces one: a key on disk is a +// home id phones may already dial, and a silent new one would strand them. +// os.Link fails when the file exists, so a race with another writer cannot +// replace it either. func createKey(path string) (*ecdsa.PrivateKey, error) { key, err := ecdsa.GenerateKey(elliptic.P256(), rand.Reader) if err != nil { @@ -167,9 +188,9 @@ func createKey(path string) (*ecdsa.PrivateKey, error) { if err := os.WriteFile(tmp, pem.EncodeToMemory(&pem.Block{Type: "EC PRIVATE KEY", Bytes: der}), 0o600); err != nil { return nil, fmt.Errorf("cloudlink: writing %s: %w", tmp, err) } - if err := os.Rename(tmp, path); err != nil { - _ = os.Remove(tmp) - return nil, fmt.Errorf("cloudlink: replacing %s: %w", path, err) + defer os.Remove(tmp) + if err := os.Link(tmp, path); err != nil { + return nil, fmt.Errorf("cloudlink: %s: %w", path, err) } return key, nil } diff --git a/go/internal/cloudlink/link.go b/go/internal/cloudlink/link.go index 01cdb7fa0..7329e2d77 100644 --- a/go/internal/cloudlink/link.go +++ b/go/internal/cloudlink/link.go @@ -1,16 +1,20 @@ -// Package cloudlink streams a signed, readable copy of the box's state to FTW -// Cloud, when an owner has turned it on. docs/protocol-cloud.md is the wire -// format and docs/adr/0008-ftw-cloud.md the decision. +// Package cloudlink is the box's one link to FTW Cloud. It carries the +// phones' own sessions with the box (the tunnel) and, when an owner has +// turned it on, a signed, readable copy of the box's state. +// docs/protocol-cloud.md is the wire format and docs/adr/0008-ftw-cloud.md +// the decision. // -// FTW Cloud reads and never controls. This package holds that line in its -// build: it reaches the box only through the read-only ports below and may -// import nothing from the box (imports_test.go and CI check it with go list -// -deps). What the service may send back is a closed set of five messages, -// and none of them asks the box to do anything but send sooner, send again or -// forget a key. +// FTW Cloud never chooses an action. This package holds that line in its +// build: it reaches the box only through the read-only ports below and the +// Tunnel port, and may import nothing from the box (CI checks it with go list +// -deps). What the service may send back is a closed set of six messages. +// Five ask the box for nothing but to send sooner, send again or forget a +// key. The sixth, tun, carries a phone's Noise session, which this package +// hands to the Tunnel unread. // -// The link is off until an owner grants it. A missing grant, or one whose key -// fingerprint does not match cloud.key, reads as off. +// The box links when it first shows a pairing code, has a phone that may +// open a tunnel, or an owner grants the readable copy. A link record or a +// grant whose key fingerprint does not match cloud.key reads as off. package cloudlink import ( @@ -65,6 +69,10 @@ var ErrPurgePending = errors.New("cloudlink: FTW Cloud is still deleting the pre // ErrNotGranted is statistics asked for while FTW Cloud is off. var ErrNotGranted = errors.New("cloudlink: turn FTW Cloud on first") +// ErrLinkEnding is a grant asked for while the service deletes the last +// link's home. The next link comes once the receipt is in. +var ErrLinkEnding = errors.New("cloudlink: FTW Cloud is still deleting the previous link") + // statsIDBytes is the length of a statsId, which the protocol fixes at 16. const statsIDBytes = 16 @@ -112,11 +120,28 @@ type RosterDevice struct { Epoch uint64 // Scopes are what the role carries, expanded by the box. Scopes []string + // Tun says the device may open a session to the box through FTW Cloud. + Tun bool } -// Roster lists the paired phones. +// Roster lists the paired phones and the pairing code on the box's screen. type Roster interface { Devices() []RosterDevice + // Pending is the QR pairing code the box shows now and when it stops + // working. ok is false when none is shown. A spoken code is never + // listed: its hash would let the service find it. + Pending() (code []byte, expires time.Time, ok bool) +} + +// Tunnel serves the phones' sessions that FTW Cloud carries. The box's app +// session table implements it; the link reaches it only through this port, +// and hands it each tun from the service unread. +type Tunnel interface { + // Deliver takes one tun from the service. It must not block: the link + // reads the service on the same goroutine. + Deliver(t cloudwire.Tun) + // Drop ends every session the uplink carried, when the uplink ends. + Drop() } // Options wires a link. Site, Roster, KV and KeyPath are required. @@ -134,6 +159,9 @@ type Options struct { // Notify tells owners the grant changed, with protocol.PushCloudGranted // or PushCloudRevoked. Nil sends nothing. Notify func(kind string) + // Tunnel serves the phones' sessions. Nil takes no tun: the box does not + // list tun in hello.caps, and the service sends it none. + Tunnel Tunnel Logger *slog.Logger // Seams for tests. Production leaves them zero. @@ -158,21 +186,29 @@ type Link struct { key *ecdsa.PrivateKey pk []byte home string + link *linkRecord grant *grantRecord notice string connected bool lastSend time.Time budget budget - sent map[string]sentFrame - srcRev uint64 - table map[string]cloudwire.Source - tableKey string - liveSeq uint64 - lpSeq uint64 - resyncAt time.Time - attempt int - purgedAt int64 - rosterDue bool + // tunOut and tunBulk carry framed tun messages, lane 0 and bulk, to + // the connection's tun writer. Nil while no uplink is up. tunBudget counts what left, apart from + // the readable copy's budget, and tunDropped what never did. + tunOut chan []byte + tunBulk chan []byte + tunBudget budget + tunDropped int64 + sent map[string]sentFrame + srcRev uint64 + table map[string]cloudwire.Source + tableKey string + liveSeq uint64 + lpSeq uint64 + resyncAt time.Time + attempt int + purgedAt int64 + rosterDue bool // events wait for the writer; eventsDropped counts those that never // left. closing is set by Bye, after which the link does not dial @@ -259,37 +295,68 @@ func parseServiceKeys(list string) ([]*ecdsa.PublicKey, error) { return keys, nil } -// load reads the grant and the key. A grant without its key, or with another -// key, reads as off and leaves a notice. +// load reads the link, the grant and the key. A record without its key, or +// with another key, reads as off and leaves a notice. func (l *Link) load() error { l.mu.Lock() defer l.mu.Unlock() - l.grant, l.key, l.pk, l.home, l.notice = nil, nil, nil, "", "" + l.link, l.grant, l.key, l.pk, l.home, l.notice = nil, nil, nil, nil, "", "" if r, err := loadJSON[receipt](l.opts.KV, purgedKey); err == nil && r != nil { l.purgedAt = r.AtMs } + link, err := loadJSON[linkRecord](l.opts.KV, linkKey) + if err != nil { + l.notice = "link_unreadable" + l.log.Warn("FTW Cloud link unreadable; treating it as off", "err", err) + link = nil + } g, err := loadJSON[grantRecord](l.opts.KV, grantKey) if err != nil { l.notice = "grant_unreadable" l.log.Warn("FTW Cloud grant unreadable; treating it as off", "err", err) - return nil + g = nil } - if g == nil { + if link == nil && g == nil { return nil } key, err := loadKey(l.opts.KeyPath) if err != nil { l.notice = "key_missing" - l.log.Warn("FTW Cloud is off: the grant has no key on this box", "err", err) + l.log.Warn("FTW Cloud is off: the link has no key on this box", "err", err) return nil } fp, pk, err := fingerprint(key) - if err != nil || fp != g.KeyFp { + if err != nil { + l.notice = "key_mismatch" + return nil + } + if link == nil { + // A grant made before the link existed holds the key it was made + // with. That key becomes the link's, so the home id stays. + if g.KeyFp != fp { + l.notice = "key_mismatch" + l.log.Warn("FTW Cloud is off: cloud.key is not the key the grant was made with") + return nil + } + link = &linkRecord{KeyFp: fp} + if err := saveJSON(l.opts.KV, linkKey, link); err != nil { + return err + } + } + if link.KeyFp != fp { l.notice = "key_mismatch" - l.log.Warn("FTW Cloud is off: cloud.key is not the key the grant was made with") + l.log.Warn("FTW Cloud is off: cloud.key is not the key the link was made with") return nil } - l.grant, l.key, l.pk, l.home = g, key, pk, cloudwire.HomeID(pk) + l.link, l.key, l.pk, l.home = link, key, pk, cloudwire.HomeID(pk) + switch { + case g == nil: + case g.KeyFp != fp: + l.notice = "key_mismatch" + l.log.Warn("the FTW Cloud grant was made with another key; the readable copy is off") + default: + l.grant = g + } return nil } @@ -301,14 +368,104 @@ func (l *Link) signal() { } } -// RosterChanged is called by enrolment after a pair, role change or revoke. +// RosterChanged is called by enrolment after a pair, role change, revoke or +// a pairing code shown. A box that now has a phone to reach it, or a code on +// its screen, links at once, so the code's home id exists before the code +// is drawn. func (l *Link) RosterChanged() { + if l.wantsLink() { + l.mu.Lock() + if err := l.linkLocked(); err != nil && !errors.Is(err, ErrNoService) { + l.log.Warn("could not link to FTW Cloud", "err", err) + } + l.mu.Unlock() + } l.mu.Lock() l.rosterDue = true l.mu.Unlock() l.signal() } +// wantsLink reports whether anything needs the link: a phone that may open a +// tunnel, or a pairing code on the screen. A grant needs it too, and makes it +// itself. +func (l *Link) wantsLink() bool { + if _, _, ok := l.opts.Roster.Pending(); ok { + return true + } + for _, d := range l.opts.Roster.Devices() { + if d.Tun { + return true + } + } + return false +} + +// linkLocked makes the link record when there is none, around the +// cloud.key on disk if there is one, so its home id stays, or a new key. A +// link waiting for its end to be confirmed stays as it is: the next one +// comes after the receipt, under a new home id. The caller holds l.mu. +func (l *Link) linkLocked() error { + if l.link != nil { + return nil + } + if len(l.opts.ServiceKeys) == 0 { + return ErrNoService + } + key, err := loadKey(l.opts.KeyPath) + switch { + case err == nil: + l.log.Info("FTW Cloud links with the cloud.key already on this box") + case errors.Is(err, os.ErrNotExist): + if key, err = createKey(l.opts.KeyPath); err != nil { + return err + } + default: + // A file that is not a key is not ours to replace. + return err + } + fp, pk, err := fingerprint(key) + if err != nil { + return err + } + rec := &linkRecord{KeyFp: fp} + if err := saveJSON(l.opts.KV, linkKey, rec); err != nil { + return err + } + // A grant on disk made with another key stays off; the grant load + // left it nil. + l.link, l.key, l.pk, l.home, l.notice = rec, key, pk, cloudwire.HomeID(pk), "" + l.table, l.tableKey, l.sent, l.rosterDue = nil, "", map[string]sentFrame{}, true + l.log.Info("audit: FTW Cloud linked", "home", l.home) + l.signal() + return nil +} + +// endIfIdle starts the end of a link nothing needs any more: no phone may +// open a tunnel, no code is shown and the readable copy is off. The box asks +// the service to delete the home and keeps the key until the receipt. +func (l *Link) endIfIdle() { + l.mu.Lock() + idle := l.link != nil && !l.link.Ending && l.grant == nil + l.mu.Unlock() + if !idle || l.wantsLink() { + return + } + l.mu.Lock() + defer l.mu.Unlock() + if l.link == nil || l.link.Ending || l.grant != nil { + return + } + rec := *l.link + rec.Ending = true + if err := saveJSON(l.opts.KV, linkKey, rec); err != nil { + l.log.Warn("could not store the end of the FTW Cloud link", "err", err) + return + } + l.link = &rec + l.log.Info("audit: FTW Cloud link ending; nothing on the box needs it", "home", l.home) +} + // GrantRequest is an owner's request to turn FTW Cloud on. type GrantRequest struct { Scopes []string `json:"scopes"` @@ -318,7 +475,8 @@ type GrantRequest struct { By string `json:"-"` } -// Grant turns FTW Cloud on, or changes the scopes of a grant already on. +// Grant turns the readable copy on, or changes the scopes of a grant already +// on. It links the box first when it is not linked. func (l *Link) Grant(req GrantRequest) error { if len(l.opts.ServiceKeys) == 0 { return ErrNoService @@ -336,6 +494,9 @@ func (l *Link) Grant(req GrantRequest) error { case l.grant != nil && l.grant.Revoked: l.mu.Unlock() return ErrPurgePending + case l.link != nil && l.link.Ending: + l.mu.Unlock() + return ErrLinkEnding case l.grant != nil: g := *l.grant g.Scopes, g.Policy, g.By = scopes, req.Policy, req.By @@ -350,39 +511,33 @@ func (l *Link) Grant(req GrantRequest) error { l.signal() return nil } - l.mu.Unlock() - - key, err := createKey(l.opts.KeyPath) - if err != nil { - return err - } - fp, pk, err := fingerprint(key) - if err != nil { + if err := l.linkLocked(); err != nil { + l.mu.Unlock() return err } id, err := randomBytes(cloudwire.GrantIDBytes) if err != nil { + l.mu.Unlock() return err } gk, err := randomBytes(cloudwire.GrantKeyBytes) if err != nil { + l.mu.Unlock() return err } g := grantRecord{ ID: id, Scopes: scopes, Purposes: []string{protocol.CloudPurposeServe}, - KeyFp: fp, GrantKey: gk, By: req.By, Policy: req.Policy, + KeyFp: l.link.KeyFp, GrantKey: gk, By: req.By, Policy: req.Policy, Controller: cloudwire.ControllerSourceful, } if l.clockSynced() { g.AtMs = time.Now().UnixMilli() } if err := saveJSON(l.opts.KV, grantKey, g); err != nil { - _ = os.Remove(l.opts.KeyPath) + l.mu.Unlock() return err } - - l.mu.Lock() - l.grant, l.key, l.pk, l.home, l.notice = &g, key, pk, cloudwire.HomeID(pk), "" + l.grant, l.notice = &g, "" l.table, l.tableKey, l.sent, l.rosterDue = nil, "", map[string]sentFrame{}, true l.mu.Unlock() @@ -392,9 +547,10 @@ func (l *Link) Grant(req GrantRequest) error { return nil } -// Revoke turns FTW Cloud off. The box stops sending readings at once, asks -// the service to delete the home, and keeps the key until the service's -// signed receipt arrives. +// Revoke turns the readable copy off. The box stops sending readings at +// once, and its next roster asks the service to delete the copy. The grant +// record stays until the service's signed receipt arrives. The link, the +// home id and the tunnel stay. func (l *Link) Revoke(by string) error { l.mu.Lock() if l.grant == nil || l.grant.Revoked { @@ -411,6 +567,7 @@ func (l *Link) Revoke(by string) error { return err } l.grant = &g + l.rosterDue = true l.mu.Unlock() l.audit("revoked", &g) @@ -498,28 +655,50 @@ func withPurpose(purposes []string, p string, on bool) []string { return out } -// finishPurge deletes the key and the grant once the service has confirmed -// the deletion. -func (l *Link) finishPurge(atMs int64) { +// finishCopy forgets a revoked grant once the service has confirmed that its +// copy is gone. It reports whether the receipt was the one the box waits for. +func (l *Link) finishCopy(atMs int64, grantID []byte) bool { l.mu.Lock() g := l.grant - if g == nil || !g.Revoked { + if g == nil || !g.Revoked || !bytes.Equal(g.ID, grantID) { l.mu.Unlock() - return + return false } - if err := saveJSON(l.opts.KV, purgedKey, receipt{AtMs: atMs, GrantID: g.ID}); err != nil { + if err := saveJSON(l.opts.KV, purgedKey, receipt{What: cloudwire.PurgedCopy, AtMs: atMs, GrantID: g.ID}); err != nil { l.log.Warn("could not store the FTW Cloud purge receipt", "err", err) } if err := saveJSON(l.opts.KV, grantKey, nil); err != nil { l.log.Warn("could not clear the FTW Cloud grant", "err", err) } + l.grant, l.purgedAt = nil, atMs + l.table, l.tableKey, l.sent, l.rosterDue = nil, "", map[string]sentFrame{}, true + l.mu.Unlock() + l.audit("purged", g) + l.signal() + return true +} + +// finishLink deletes the key and the link once the service has confirmed +// that the home is gone. +func (l *Link) finishLink() { + l.mu.Lock() + if l.link == nil || !l.link.Ending { + l.mu.Unlock() + return + } + home := l.home + for _, key := range []string{linkKey, grantKey} { + if err := saveJSON(l.opts.KV, key, nil); err != nil { + l.log.Warn("could not clear the FTW Cloud link", "err", err) + } + } if err := os.Remove(l.opts.KeyPath); err != nil && !errors.Is(err, os.ErrNotExist) { l.log.Warn("could not delete cloud.key", "err", err) } - l.grant, l.key, l.pk, l.home, l.purgedAt = nil, nil, nil, "", atMs + l.link, l.grant, l.key, l.pk, l.home = nil, nil, nil, nil, "" l.table, l.tableKey, l.sent = nil, "", map[string]sentFrame{} l.mu.Unlock() - l.audit("purged", g) + l.log.Info("audit: FTW Cloud link ended", "home", home) } // audit writes the line that records a grant change on the box. @@ -541,10 +720,13 @@ func (l *Link) clockSynced() bool { // Status is what GET /api/cloud shows. type Status struct { - // State is off, on or revoking. + // State is the readable copy's: off, on or revoking. State string `json:"state"` - // Notice says why a grant on disk reads as off: key_missing, - // key_mismatch or grant_unreadable. + // Link is off, on or ending: whether the box is linked, which the + // phones' tunnels need and the readable copy rides on. + Link string `json:"link"` + // Notice says why a link or grant on disk reads as off: key_missing, + // key_mismatch, link_unreadable or grant_unreadable. Notice string `json:"notice,omitempty"` Available bool `json:"available"` Home string `json:"home,omitempty"` @@ -566,6 +748,12 @@ type Status struct { // the queue was full, or the uplink was down for longer than a phone // would still act on them. EventsDropped int64 `json:"eventsDropped"` + // TunBytesLast24h counts the tunnel's bytes, apart from the readable + // copy's, against TunBudgetBytes. TunDropped counts tun messages that + // never left: no uplink, a full queue or the budget reached. + TunBytesLast24h int64 `json:"tunBytesLast24h"` + TunBudgetBytes int64 `json:"tunBudgetBytes"` + TunDropped int64 `json:"tunDropped"` } // Status reports the grant and the link. @@ -577,13 +765,21 @@ func (l *Link) Status() Status { Scopes: []string{}, Purposes: []string{}, Connected: l.connected, LastSendAgeMs: -1, PurgedAtMs: l.purgedAt, EventsDropped: l.eventsDropped, BytesLast24h: l.budget.total(l.uptimeMs()), BudgetBytes: budgetBytes, + Link: "off", TunBytesLast24h: l.tunBudget.total(l.uptimeMs()), TunBudgetBytes: tunBudgetBytes, + TunDropped: l.tunDropped, } s.BudgetReached = s.BytesLast24h >= budgetBytes if !l.lastSend.IsZero() { s.LastSendAgeMs = time.Since(l.lastSend).Milliseconds() } + if l.link != nil { + s.Link, s.Home = "on", l.home + if l.link.Ending { + s.Link = "ending" + } + } if g := l.grant; g != nil { - s.State, s.Home, s.Scopes, s.Purposes = "on", l.home, g.Scopes, g.Purposes + s.State, s.Scopes, s.Purposes = "on", g.Scopes, g.Purposes s.GrantedAtMs, s.GrantedBy, s.Policy = g.AtMs, g.By, g.Policy if g.Revoked { s.State = "revoking" @@ -592,27 +788,46 @@ func (l *Link) Status() Status { return s } +// Home is the home id at FTW Cloud while the box is linked, and "" when it +// is not or its link is ending. A pairing QR that names the home reads it. +func (l *Link) Home() string { + l.mu.Lock() + defer l.mu.Unlock() + if l.link == nil || l.link.Ending { + return "" + } + return l.home +} + // Available reports whether this build names a service, and so can run the // uplink at all. Release builds without service keys cannot. func (l *Link) Available() bool { return len(l.opts.ServiceKeys) > 0 } +// TakesTun reports whether the link carries the phones' own sessions as tun, +// so a phone that knows the home id can reach the box without the relay. +func (l *Link) TakesTun() bool { return l.Available() && l.opts.Tunnel != nil } + // SessionInfo is what a phone needs, over its Noise session with the box, to -// read this home from FTW Cloud: the key that signs it, the home id, the boot -// that anchors its times and names for the opaque ids. It is nil unless the -// grant is on. +// reach this home through FTW Cloud: the home id, the key that signs the +// readable copy, the boot that anchors its times and names for the opaque +// ids. It is nil while the box is not linked or its link is ending, and its +// tables are empty while the readable copy is off. // // The tables name every driver and loadpoint the box knows now. One that // appears later has no name on a phone until its next session. func (l *Link) SessionInfo() *protocol.CloudSession { snap := l.current() - if snap.g == nil || snap.g.Revoked || len(snap.pk) != cloudwire.PublicKeyBytes { + if snap.link == nil || snap.link.Ending || len(snap.pk) != cloudwire.PublicKeyBytes { return nil } - gk := snap.g.GrantKey s := &protocol.CloudSession{ PK: bytes.Clone(snap.pk), Home: snap.home, Boot: bytes.Clone(l.boot), Sources: map[string]string{}, Loadpoints: map[string]string{}, } + if !snap.copyOn() { + return s + } + gk := snap.g.GrantKey source := func(name string) { if name != "" { s.Sources[cloudwire.SourceID(gk, name)] = name @@ -655,8 +870,9 @@ func (l *Link) header() cloudwire.Header { return h } -// grantSnapshot is the grant and key as one consistent read. +// grantSnapshot is the link, the grant and the key as one consistent read. type grantSnapshot struct { + link *linkRecord g *grantRecord key *ecdsa.PrivateKey pk []byte @@ -666,17 +882,37 @@ type grantSnapshot struct { func (l *Link) current() grantSnapshot { l.mu.Lock() defer l.mu.Unlock() - return grantSnapshot{g: l.grant, key: l.key, pk: l.pk, home: l.home} + return grantSnapshot{link: l.link, g: l.grant, key: l.key, pk: l.pk, home: l.home} +} + +// copyOn reports whether the readable copy is on and not being withdrawn. +func (snap grantSnapshot) copyOn() bool { return snap.g != nil && !snap.g.Revoked } + +// grantID is the grant hello names: the readable copy's, or none. +func (snap grantSnapshot) grantID() []byte { + if !snap.copyOn() { + return nil + } + return snap.g.ID } -// Run keeps the uplink up while a grant is on, until ctx ends. +// Run keeps the uplink up while the box is linked, until ctx ends. func (l *Link) Run(ctx context.Context) error { for { if l.isClosing() { return nil } snap := l.current() - if snap.g == nil || len(l.opts.ServiceKeys) == 0 { + if snap.link == nil && l.wantsLink() { + // The first link, or the next one after an end was confirmed. + l.mu.Lock() + if err := l.linkLocked(); err != nil && !errors.Is(err, ErrNoService) { + l.log.Warn("could not link to FTW Cloud", "err", err) + } + l.mu.Unlock() + snap = l.current() + } + if snap.link == nil || len(l.opts.ServiceKeys) == 0 { select { case <-ctx.Done(): return ctx.Err() @@ -707,24 +943,30 @@ func (l *Link) isClosing() bool { return l.closing } -// allowed reports whether the grant lets a class leave the house. +// allowed reports whether the grant lets a class leave the house. Nothing +// readable leaves while the copy is off or being withdrawn. func (snap grantSnapshot) allowed(class string) bool { scope, forwarded := cloudwire.ClassScope[class] - if !forwarded { + if !forwarded || !snap.copyOn() { return false } return scope == "" || snap.g.has(scope) } -// caps is the classes this box sends under the grant. -func (snap grantSnapshot) caps() []string { +// caps is the classes this box sends under the grant, the roster, and tun +// when the box takes the tunnel. +func (l *Link) caps(snap grantSnapshot) []string { var out []string for _, c := range []string{cloudwire.MsgLive, cloudwire.MsgLP, cloudwire.MsgPrices, cloudwire.MsgHB, cloudwire.MsgEvent} { if snap.allowed(c) { out = append(out, c) } } - return append(out, cloudwire.MsgRoster) + out = append(out, cloudwire.MsgRoster) + if l.opts.Tunnel != nil { + out = append(out, cloudwire.CapTun) + } + return out } // tlsConfig is TLS 1.3 only, with the system roots unless a test or a dev diff --git a/go/internal/cloudlink/link_test.go b/go/internal/cloudlink/link_test.go index 7d64849d3..0e352f62e 100644 --- a/go/internal/cloudlink/link_test.go +++ b/go/internal/cloudlink/link_test.go @@ -105,11 +105,68 @@ func (s *fakeSite) Prices() (cloudwire.Prices, bool) { SpotMinor: []*int64{&p}, TotalMinor: []*int64{&p}}, true } -type fakeRoster struct{} +// fakeRoster is one owner phone. Its zero value is a box from before the +// tunnel, whose phone may not open one, so nothing links it but a grant. +type fakeRoster struct { + state *fakeRosterState +} + +type fakeRosterState struct { + mu sync.Mutex + tun bool + gone bool + code []byte + expires time.Time +} -func (fakeRoster) Devices() []RosterDevice { +func (r fakeRoster) Devices() []RosterDevice { + tun := false + if r.state != nil { + r.state.mu.Lock() + defer r.state.mu.Unlock() + if r.state.gone { + return nil + } + tun = r.state.tun + } return []RosterDevice{{ID: "q5Lw0Zb1", Key: bytes.Repeat([]byte{5}, 32), Role: "owner", Epoch: 1, - Scopes: []string{protocol.ScopeLiveRead, protocol.ScopeDispatchWrite, protocol.ScopeEventsRead}}} + Scopes: []string{protocol.ScopeLiveRead, protocol.ScopeDispatchWrite, protocol.ScopeEventsRead}, Tun: tun}} +} + +func (r fakeRoster) Pending() ([]byte, time.Time, bool) { + if r.state == nil { + return nil, time.Time{}, false + } + r.state.mu.Lock() + defer r.state.mu.Unlock() + // As enrolment does, a code that ran out is none. + return r.state.code, r.state.expires, r.state.code != nil && time.Now().Before(r.state.expires) +} + +// fakeTunnel is the box's session table, as the link sees it. +type fakeTunnel struct { + ch chan cloudwire.Tun + mu sync.Mutex + drops int +} + +func (f *fakeTunnel) Deliver(t cloudwire.Tun) { + select { + case f.ch <- t: + default: + } +} + +func (f *fakeTunnel) Drop() { + f.mu.Lock() + f.drops++ + f.mu.Unlock() +} + +func (f *fakeTunnel) dropped() int { + f.mu.Lock() + defer f.mu.Unlock() + return f.drops } // fakeService is ftwcloudd's half of the handshake, enough to test the box. @@ -125,6 +182,7 @@ type fakeService struct { afterHello func(conn *websocket.Conn, home string) rosterSeqOK uint64 + wmu sync.Mutex mu sync.Mutex conns int hello []byte @@ -163,6 +221,8 @@ func (s *fakeService) write(conn *websocket.Conn, key *ecdsa.PrivateKey, home, t } raw, _ := env.Encode() frame, _ := cloudwire.EncodeFrame(raw) + s.wmu.Lock() + defer s.wmu.Unlock() _ = conn.WriteMessage(websocket.BinaryMessage, frame) } @@ -233,8 +293,17 @@ func (s *fakeService) handle(w http.ResponseWriter, r *http.Request) { return } env, err := cloudwire.DecodeEnvelope(payload) - if err != nil || cloudwire.Verify(pub, home, env) != nil { - s.t.Errorf("box envelope %q does not verify: %v", env.T, err) + switch { + case err != nil: + s.t.Errorf("box frame: %v", err) + return + case env.T == cloudwire.MsgTun: + // d is Noise ciphertext; the box signs no tun. + if len(env.Sig) != 0 { + s.t.Errorf("the box signed a tun") + } + case cloudwire.Verify(pub, home, env) != nil: + s.t.Errorf("box envelope %q does not verify", env.T) return } s.mu.Lock() @@ -242,19 +311,22 @@ func (s *fakeService) handle(w http.ResponseWriter, r *http.Request) { s.raw = append(s.raw, msg) s.mu.Unlock() s.ch <- env - if env.T == cloudwire.MsgBye && s.purge { + if !s.purge { + continue + } + switch env.T { + case cloudwire.MsgBye: var bye cloudwire.Bye - _ = cloudwire.Unmarshal(env.B, &bye) - var grantID []byte - for _, e := range s.received() { - var r cloudwire.Roster - if e.T == cloudwire.MsgRoster && cloudwire.Unmarshal(e.B, &r) == nil { - grantID = r.Grant.ID - } + if cloudwire.Unmarshal(env.B, &bye) == nil && bye.Purge { + s.write(conn, s.key, home, cloudwire.MsgPurged, cloudwire.Purged{ + AtMs: 1_791_821_000_000, What: cloudwire.PurgedHome}) } - if bye.Purge { + case cloudwire.MsgRoster: + // A grant with an id and no purposes asks for its copy to go. + var r cloudwire.Roster + if cloudwire.Unmarshal(env.B, &r) == nil && len(r.Grant.ID) > 0 && len(r.Grant.Purposes) == 0 { s.write(conn, s.key, home, cloudwire.MsgPurged, cloudwire.Purged{ - AtMs: 1_791_821_000_000, What: cloudwire.PurgedHome, GrantID: grantID}) + AtMs: 1_791_821_000_000, What: cloudwire.PurgedCopy, GrantID: r.Grant.ID}) } } } @@ -287,6 +359,8 @@ type harness struct { link *Link svc *fakeService site *fakeSite + roster *fakeRosterState + tunnel *fakeTunnel kv *memKV keyPath string cancel context.CancelFunc @@ -304,11 +378,13 @@ func newHarness(t *testing.T, svc *fakeService) *harness { func newHarnessKeys(t *testing.T, svc *fakeService, keys []*ecdsa.PublicKey) *harness { t.Helper() h := &harness{svc: svc, site: &fakeSite{grid: 1200, state: protocol.SourceLive}, kv: &memKV{}, - keyPath: filepath.Join(t.TempDir(), "cloud.key"), pushed: make(chan string, 8)} + keyPath: filepath.Join(t.TempDir(), "cloud.key"), pushed: make(chan string, 8), + roster: &fakeRosterState{}, tunnel: &fakeTunnel{ch: make(chan cloudwire.Tun, 64)}} link, err := New(Options{ - Site: h.site, Roster: fakeRoster{}, KV: h.kv, KeyPath: h.keyPath, Build: "test", + Site: h.site, Roster: fakeRoster{state: h.roster}, KV: h.kv, KeyPath: h.keyPath, Build: "test", Started: time.Now().Add(-time.Minute), ClockSynced: func() bool { return true }, Notify: func(kind string) { h.pushed <- kind }, + Tunnel: h.tunnel, Endpoint: svc.endpoint(), ServiceKeys: keys, RootCAs: svc.roots(), Tick: 20 * time.Millisecond, Logger: slog.New(slog.NewTextHandler(io.Discard, nil)), @@ -327,6 +403,14 @@ func newHarnessKeys(t *testing.T, svc *fakeService, keys []*ecdsa.PublicKey) *ha return h } +// setRoster changes the phones as enrolment would, and tells the link. +func (h *harness) setRoster(change func(r *fakeRosterState)) { + h.roster.mu.Lock() + change(h.roster) + h.roster.mu.Unlock() + h.link.RosterChanged() +} + func (h *harness) grant(t *testing.T, scopes ...string) { t.Helper() if len(scopes) == 0 { @@ -561,13 +645,18 @@ func TestATableChangeGoesAtOnceWithANewRevision(t *testing.T) { } } -func TestRevokeDeletesTheKeyOnlyAfterTheReceipt(t *testing.T) { +// Turning the readable copy off stops readings at once, asks the service to +// delete the copy, and forgets the grant on the receipt. The link stays, with +// its key and home id, for the phones' tunnels. +func TestRevokeDeletesTheCopyAndKeepsTheLink(t *testing.T) { svc := newFakeService(t) svc.purge = true h := newHarness(t, svc) + h.setRoster(func(r *fakeRosterState) { r.tun = true }) h.grant(t) <-h.pushed svc.waitFor(cloudwire.MsgLive, 5*time.Second) + home := h.link.Status().Home if err := h.link.Revoke("test"); err != nil { t.Fatal(err) @@ -575,10 +664,12 @@ func TestRevokeDeletesTheKeyOnlyAfterTheReceipt(t *testing.T) { if kind := <-h.pushed; kind != protocol.PushCloudRevoked { t.Fatalf("owners were told %q", kind) } - bye := svc.waitFor(cloudwire.MsgBye, 3*time.Second) - var b cloudwire.Bye - if cloudwire.Unmarshal(bye.B, &b) != nil || !b.Purge || b.Why != cloudwire.ByeRevoked { - t.Fatalf("bye %+v", b) + r := h.nextRoster(t) + for len(r.Grant.Purposes) != 0 { + r = h.nextRoster(t) + } + if len(r.Grant.ID) != cloudwire.GrantIDBytes || len(r.Grant.Scopes) != 0 { + t.Fatalf("the revoking roster's grant is %+v", r.Grant) } deadline := time.Now().Add(3 * time.Second) for h.link.Status().State != "off" { @@ -587,22 +678,35 @@ func TestRevokeDeletesTheKeyOnlyAfterTheReceipt(t *testing.T) { } time.Sleep(20 * time.Millisecond) } - if _, err := os.Stat(h.keyPath); !os.IsNotExist(err) { - t.Fatal("cloud.key survived a confirmed purge") + st := h.link.Status() + if st.PurgedAtMs != 1_791_821_000_000 || st.Link != "on" || st.Home != home { + t.Fatalf("after the receipt: %+v", st) } - if st := h.link.Status(); st.PurgedAtMs != 1_791_821_000_000 { - t.Fatalf("receipt time %d", st.PurgedAtMs) + if _, err := os.Stat(h.keyPath); err != nil { + t.Fatal("cloud.key went with the copy") } - // After the bye, nothing but the receipt's ack: no readings. - got := svc.received() - for i, env := range got { - if env.T == cloudwire.MsgBye && i != len(got)-1 { - t.Fatalf("the box sent %s after bye", got[i+1].T) + if g, _ := loadJSON[grantRecord](h.kv, grantKey); g != nil { + t.Fatalf("the grant record survived its receipt: %+v", g) + } + // After the revoking roster, no reading left the house. + after := false + for _, env := range svc.received() { + switch { + case env.T == cloudwire.MsgRoster: + var r cloudwire.Roster + _ = cloudwire.Unmarshal(env.B, &r) + after = after || (len(r.Grant.ID) > 0 && len(r.Grant.Purposes) == 0) + case after && env.T != cloudwire.MsgTun: + t.Fatalf("the box sent %s after the revoke", env.T) } } + // The next roster has no grant at all. + if r := h.nextRoster(t); len(r.Grant.ID) != 0 || len(r.Grant.Purposes) != 0 { + t.Fatalf("the roster after the receipt still names a grant: %+v", r.Grant) + } } -func TestRevokeWithoutAReceiptKeepsTheKey(t *testing.T) { +func TestRevokeWithoutAReceiptKeepsTheGrantRevoking(t *testing.T) { svc := newFakeService(t) h := newHarness(t, svc) h.grant(t) @@ -610,10 +714,9 @@ func TestRevokeWithoutAReceiptKeepsTheKey(t *testing.T) { if err := h.link.Revoke("test"); err != nil { t.Fatal(err) } - svc.waitFor(cloudwire.MsgBye, 3*time.Second) time.Sleep(200 * time.Millisecond) - if _, err := os.Stat(h.keyPath); err != nil { - t.Fatal("cloud.key went before the service confirmed the purge") + if g, _ := loadJSON[grantRecord](h.kv, grantKey); g == nil || !g.Revoked { + t.Fatalf("grant record before the receipt: %+v", g) } if st := h.link.Status(); st.State != "revoking" { t.Fatalf("state %s", st.State) @@ -818,19 +921,21 @@ func TestSessionInfoLetsAPhoneCheckAndNameWhatTheServiceForwards(t *testing.T) { } } -// Turning FTW Cloud off withdraws the session information at once, before the -// service confirms the deletion. -func TestSessionInfoGoesWithTheGrant(t *testing.T) { +// Turning the readable copy off withdraws the names at once, before the +// service confirms the deletion. The home id stays, for the tunnel. +func TestSessionInfoKeepsTheHomeWithoutTheGrant(t *testing.T) { svc := newFakeService(t) h := newHarness(t, svc) h.grant(t) - if h.link.SessionInfo() == nil { - t.Fatal("no session information with FTW Cloud on") + if s := h.link.SessionInfo(); s == nil || len(s.Sources) == 0 { + t.Fatalf("session information with FTW Cloud on: %+v", s) } if err := h.link.Revoke("test"); err != nil { t.Fatal(err) } - if s := h.link.SessionInfo(); s != nil || h.link.Status().State != "revoking" { + s := h.link.SessionInfo() + if s == nil || s.Home != h.link.Status().Home || len(s.Sources) != 0 || len(s.Loadpoints) != 0 || + h.link.Status().State != "revoking" { t.Fatalf("while revoking: %+v", s) } } diff --git a/go/internal/cloudlink/sender.go b/go/internal/cloudlink/sender.go index 499e735ad..88503652a 100644 --- a/go/internal/cloudlink/sender.go +++ b/go/internal/cloudlink/sender.go @@ -1,7 +1,6 @@ package cloudlink import ( - "bytes" "context" "crypto/sha256" "encoding/hex" @@ -31,6 +30,10 @@ const ( rosterTTLS = 86_400 budgetBytes = 20_000_000 budgetMinutes = 24 * 60 + // tunBudgetBytes bounds the tunnel apart from the readable copy. A phone + // holding a session open all day at 1 Hz sends about 50 MB; this is five + // of them, and stops a runaway before it eats a household's uplink. + tunBudgetBytes = 250_000_000 ) // sentFrame is the last frame of a class that left the house, exactly. @@ -81,6 +84,9 @@ type cadence struct { anyAt time.Time // last live or hb rosterAt time.Time watchUntil time.Time + // pendingUntil is when the pairing code the last roster listed runs + // out, zero when it listed none. + pendingUntil time.Time // When each class was last composed, and whether a change it found is // still waiting for its period. Composing reads the box, and on a Pi @@ -94,7 +100,7 @@ type cadence struct { // change goes within a second whatever the period. const liveCheckEvery = time.Second -// run sends until the connection or the grant ends. +// run sends until the connection or the link ends. func (w *writer) run(ctx context.Context, events <-chan cloudwire.Envelope) error { l := w.link tick := time.NewTicker(l.opts.Tick) @@ -107,11 +113,11 @@ func (w *writer) run(ctx context.Context, events <-chan cloudwire.Envelope) erro for { snap := l.current() - if snap.g == nil || snap.home != connHome { - return errors.New("cloudlink: the grant changed under the connection") + if snap.link == nil || snap.home != connHome { + return errors.New("cloudlink: the link changed under the connection") } now := time.Now() - if snap.g.Revoked { + if snap.link.Ending { if purgeSince.IsZero() { if err := w.bye(snap, cloudwire.ByeRevoked, true); err != nil { return err @@ -178,34 +184,60 @@ func (w *writer) downlink(snap grantSnapshot, st *cadence, env cloudwire.Envelop return nil } -// acceptPurged finishes a revoke on a valid receipt: signed by the service, -// for this home and grant, while the grant is being revoked. +// acceptPurged takes a valid receipt: signed by the service for this home. +// A copy receipt for the grant being revoked forgets that grant; a home +// receipt while the link ends ends it, and true says so, so the connection +// closes. func (l *Link) acceptPurged(snap grantSnapshot, env cloudwire.Envelope) bool { var b cloudwire.Purged cur := l.current() - if cur.g == nil || !cur.g.Revoked || cur.home != snap.home || !l.verifiedByService(snap.home, env) || - cloudwire.Unmarshal(env.B, &b) != nil || b.What != cloudwire.PurgedHome || !bytes.Equal(b.GrantID, cur.g.ID) { + if cur.link == nil || cur.home != snap.home || !l.verifiedByService(snap.home, env) || + cloudwire.Unmarshal(env.B, &b) != nil { return false } - l.finishPurge(b.AtMs) - return true + switch b.What { + case cloudwire.PurgedCopy: + l.finishCopy(b.AtMs, b.GrantID) + case cloudwire.PurgedHome: + if cur.link.Ending { + l.finishLink() + return true + } + } + return false } // tick sends whatever is due. func (w *writer) tick(snap grantSnapshot, st *cadence, now time.Time) error { l := w.link + // A pairing code that ran out unused changes the roster too, and may + // leave nothing that needs the link. + if !st.pendingUntil.IsZero() && !now.Before(st.pendingUntil) { + st.pendingUntil = time.Time{} + l.mu.Lock() + l.rosterDue = true + l.mu.Unlock() + } l.mu.Lock() - due := l.rosterDue - l.rosterDue = false h := time.Duration(cloudwire.Heartbeat(l.table)) * time.Millisecond l.mu.Unlock() - if due || st.rosterAt.IsZero() || now.Sub(st.rosterAt) >= rosterRenew { - _, roster := l.rosterNumber(snap) + force := st.rosterAt.IsZero() || now.Sub(st.rosterAt) >= rosterRenew + if roster, ok := l.takeRoster(force); ok { + if cur := l.current(); cur.link == nil || cur.link.Ending { + // The end goes on the next pass, as bye. + return nil + } + // What follows reads the state the roster was built from, so a + // revoke that just landed sends no reading after it. + snap = l.current() if _, err := w.sendBody(snap, cloudwire.MsgRoster, &roster, true); err != nil { return err } st.rosterAt = now + if _, expires, ok := l.opts.Roster.Pending(); ok { + st.pendingUntil = expires + } } period := periodIdle @@ -281,8 +313,9 @@ func (w *writer) tick(snap grantSnapshot, st *cadence, now time.Time) error { } // hb never vouches for a live or lp that waits for its period, and its - // ages were read on this tick. - if now.Sub(st.anyAt) >= h && !st.liveWaiting && !st.lpWaiting { + // ages were read on this tick. It names readings, so it goes only with + // the readable copy. + if snap.allowed(cloudwire.MsgHB) && now.Sub(st.anyAt) >= h && !st.liveWaiting && !st.lpWaiting { hb := l.heartbeat(ages) sent, err := w.sendBody(snap, cloudwire.MsgHB, &hb, false) if err != nil { @@ -295,6 +328,24 @@ func (w *writer) tick(snap grantSnapshot, st *cadence, now time.Time) error { return nil } +// takeRoster says whether a roster is due, and builds it from the link as it +// is now, after reading that it is due: never from a snapshot taken before, +// or a revoke that lands in between would wait six hours for the next +// renewal while the service kept the copy. A roster about to go may leave +// nothing that needs the link, which then starts to end. +func (l *Link) takeRoster(force bool) (cloudwire.Roster, bool) { + l.mu.Lock() + due := l.rosterDue + l.rosterDue = false + l.mu.Unlock() + if !due && !force { + return cloudwire.Roster{}, false + } + l.endIfIdle() + _, roster := l.rosterNumber(l.current()) + return roster, true +} + // sendBody stamps the header, sends, and records the seq of a live or lp so // hb can name it. body is a pointer to a cloudwire body with a Header. func (w *writer) sendBody(snap grantSnapshot, t string, body any, force bool) (bool, error) { @@ -497,11 +548,21 @@ func pricesKey(p cloudwire.Prices) string { } // rosterNumber builds the roster and its number. The number rises when the -// devices or the grant differ from what it was last computed from. +// devices, the grant or the pending code differ from what it was last +// computed from. +// +// The grant has three shapes: the readable copy's while it is on; its id +// alone while a revoke waits for the receipt, which asks the service to +// delete the copy; and none while the copy is off. func (l *Link) rosterNumber(snap grantSnapshot) (uint64, cloudwire.Roster) { - r := cloudwire.Roster{TTLS: rosterTTLS, Grant: cloudwire.Grant{ - ID: snap.g.ID, Scopes: snap.g.Scopes, Purposes: snap.g.Purposes, Controller: snap.g.Controller, - }} + r := cloudwire.Roster{TTLS: rosterTTLS, Grant: cloudwire.Grant{Controller: cloudwire.ControllerSourceful}} + switch g := snap.g; { + case g == nil: + case g.Revoked: + r.Grant.ID = g.ID + default: + r.Grant = cloudwire.Grant{ID: g.ID, Scopes: g.Scopes, Purposes: g.Purposes, Controller: g.Controller} + } for _, d := range l.opts.Roster.Devices() { var scopes []string for _, s := range protocol.CloudScopes { @@ -509,10 +570,25 @@ func (l *Link) rosterNumber(snap grantSnapshot) (uint64, cloudwire.Roster) { scopes = append(scopes, s) } } - r.Devices = append(r.Devices, cloudwire.Device{ID: d.ID, Key: d.Key, Role: d.Role, Scopes: scopes, Epoch: d.Epoch}) + r.Devices = append(r.Devices, cloudwire.Device{ID: d.ID, Key: d.Key, Role: d.Role, Scopes: scopes, + Epoch: d.Epoch, Tun: d.Tun}) } sort.Slice(r.Devices, func(a, b int) bool { return r.Devices[a].ID < r.Devices[b].ID }) - sum := digest([]any{r.Devices, r.Grant}) + if code, expires, ok := l.opts.Roster.Pending(); ok && snap.home != "" { + if left := time.Until(expires); left > 0 { + r.Pending = append(r.Pending, cloudwire.Pending{ + Hash: cloudwire.PairingHash(snap.home, code), + TTLS: int64((left + time.Second - 1) / time.Second), + }) + } + } + // The number follows what the roster says, not how long a code has + // left, so it does not rise every second a code is shown. + var pending [][]byte + for _, p := range r.Pending { + pending = append(pending, p.Hash) + } + sum := digest([]any{r.Devices, r.Grant, pending}) l.mu.Lock() defer l.mu.Unlock() diff --git a/go/internal/cloudlink/tunnel_test.go b/go/internal/cloudlink/tunnel_test.go new file mode 100644 index 000000000..ccae96889 --- /dev/null +++ b/go/internal/cloudlink/tunnel_test.go @@ -0,0 +1,509 @@ +package cloudlink + +import ( + "bytes" + "crypto/ecdsa" + "errors" + "os" + "path/filepath" + "slices" + "testing" + "time" + + "github.com/gorilla/websocket" + "github.com/srcfl/ftw/go/protocol" + "github.com/srcfl/ftw/go/protocol/cloudwire" +) + +// A box links when its first phone may open a tunnel, with the readable copy +// off: it sends its roster and nothing to read. +func TestTheBoxLinksForItsPhonesAndSendsNoReadings(t *testing.T) { + svc := newFakeService(t) + h := newHarness(t, svc) + if _, err := os.Stat(h.keyPath); !os.IsNotExist(err) { + t.Fatal("cloud.key exists before anything needs the link") + } + h.setRoster(func(r *fakeRosterState) { r.tun = true }) + + r := h.nextRoster(t) + if len(r.Devices) != 1 || !r.Devices[0].Tun { + t.Fatalf("roster devices %+v", r.Devices) + } + if len(r.Grant.ID) != 0 || len(r.Grant.Scopes) != 0 || len(r.Grant.Purposes) != 0 || + r.Grant.Controller != cloudwire.ControllerSourceful { + t.Fatalf("a roster without a grant names %+v", r.Grant) + } + var hello cloudwire.Hello + svc.mu.Lock() + payload, _ := cloudwire.DecodeFrame(svc.hello) + svc.mu.Unlock() + env, _ := cloudwire.DecodeEnvelope(payload) + if err := cloudwire.Unmarshal(env.B, &hello); err != nil { + t.Fatal(err) + } + if len(hello.GrantID) != 0 || !slices.Equal(hello.Caps, []string{cloudwire.MsgRoster, cloudwire.CapTun}) { + t.Fatalf("hello names grant %x and caps %v", hello.GrantID, hello.Caps) + } + + time.Sleep(2500 * time.Millisecond) + for _, env := range svc.received() { + if env.T != cloudwire.MsgRoster { + t.Fatalf("%s left a box whose readable copy is off", env.T) + } + } + st := h.link.Status() + if st.Link != "on" || st.State != "off" || st.Home != svc.home { + t.Fatalf("status %+v", st) + } + if s := h.link.SessionInfo(); s == nil || s.Home != svc.home || len(s.Sources) != 0 { + t.Fatalf("session information %+v", s) + } +} + +// A QR code on the screen links the box and is listed by its hash, never by +// the code. +func TestAPendingCodeIsListedByItsHash(t *testing.T) { + svc := newFakeService(t) + h := newHarness(t, svc) + code := bytes.Repeat([]byte{0x42}, 16) + h.setRoster(func(r *fakeRosterState) { + r.gone, r.code, r.expires = true, code, time.Now().Add(10*time.Minute) + }) + home := h.link.Status().Home + if home == "" { + t.Fatal("a code on the screen did not link the box at once") + } + r := h.nextRoster(t) + if len(r.Pending) != 1 || !bytes.Equal(r.Pending[0].Hash, cloudwire.PairingHash(home, code)) || + r.Pending[0].TTLS < 590 || r.Pending[0].TTLS > 600 { + t.Fatalf("pending %+v", r.Pending) + } + svc.mu.Lock() + defer svc.mu.Unlock() + for _, raw := range svc.raw { + if bytes.Contains(raw, code) { + t.Fatal("the pairing code itself left the house") + } + } +} + +// tun goes both ways unsigned, reaches the Tunnel unread, and counts against +// its own budget: a readable copy past its budget does not stop it, and the +// tunnel past its own does not stop the copy. +func TestTunGoesBothWaysApartFromTheBudget(t *testing.T) { + svc := newFakeService(t) + svc.afterHello = func(conn *websocket.Conn, home string) { + svc.write(conn, nil, home, cloudwire.MsgTun, cloudwire.Tun{Dev: "q5Lw0Zb1", Sid: 9, D: []byte{1, 2, 3}}) + } + h := newHarness(t, svc) + h.link.mu.Lock() + h.link.budget.add(h.link.uptimeMs(), budgetBytes) + h.link.mu.Unlock() + h.setRoster(func(r *fakeRosterState) { r.tun = true }) + + select { + case got := <-h.tunnel.ch: + if got.Dev != "q5Lw0Zb1" || got.Sid != 9 || !bytes.Equal(got.D, []byte{1, 2, 3}) { + t.Fatalf("the Tunnel got %+v", got) + } + case <-time.After(5 * time.Second): + t.Fatal("no tun reached the Tunnel") + } + if h.link.SendTun(cloudwire.Tun{Dev: "q5Lw0Zb1", Sid: 9, D: []byte{4, 5}}, false) != nil { + t.Fatal("SendTun refused with the uplink up") + } + up := svc.waitFor(cloudwire.MsgTun, 3*time.Second) + var tun cloudwire.Tun + if err := cloudwire.Unmarshal(up.B, &tun); err != nil || tun.Sid != 9 || !bytes.Equal(tun.D, []byte{4, 5}) { + t.Fatalf("the service got %+v: %v", tun, err) + } + // The tun went although the copy's budget is spent, and it did not + // count against that budget: only the roster, sent past the budget on + // purpose, added to it. + // hello and the roster go past the budget on purpose; nothing else that + // left counts but them. + copyBytes := func() int64 { + svc.mu.Lock() + defer svc.mu.Unlock() + n := int64(len(svc.hello)) + for i, env := range svc.got { + if env.T != cloudwire.MsgTun { + n += int64(len(svc.raw[i])) + } + } + return n + } + deadline := time.Now().Add(2 * time.Second) + for { + st := h.link.Status() + if st.TunBytesLast24h > 0 && st.BudgetReached && st.BytesLast24h == budgetBytes+copyBytes() { + break + } + if time.Now().After(deadline) { + t.Fatalf("status %+v; the copy's own frames are %d bytes", st, copyBytes()) + } + time.Sleep(20 * time.Millisecond) + } + + // Past the tunnel's own budget, tun stops. + h.link.mu.Lock() + h.link.tunBudget.add(h.link.uptimeMs(), tunBudgetBytes) + h.link.mu.Unlock() + if h.link.SendTun(cloudwire.Tun{Dev: "q5Lw0Zb1", Sid: 9, D: []byte{6}}, false) == nil { + t.Fatal("SendTun took a message past the tunnel's budget") + } + if h.link.Status().TunDropped == 0 { + t.Fatal("the drop was not counted") + } +} + +// The readable copy's budget does not count tun, so a phone at work never +// stops hb. +func TestTunDoesNotSpendTheCopysBudget(t *testing.T) { + svc := newFakeService(t) + h := newHarness(t, svc) + h.setRoster(func(r *fakeRosterState) { r.tun = true }) + h.nextRoster(t) + before := h.link.Status().BytesLast24h + for range 10 { + if h.link.SendTun(cloudwire.Tun{Dev: "q5Lw0Zb1", Sid: 1, D: make([]byte, 4096)}, false) != nil { + t.Fatal("SendTun refused") + } + svc.waitFor(cloudwire.MsgTun, 3*time.Second) + } + st := h.link.Status() + if st.BytesLast24h != before || st.TunBytesLast24h < 40_960 { + t.Fatalf("copy %d → %d bytes, tunnel %d", before, st.BytesLast24h, st.TunBytesLast24h) + } +} + +// Without an uplink nothing queues: a session the phone cannot reach is one +// it starts again. +func TestSendTunWithoutAnUplinkIsRefused(t *testing.T) { + svc := newFakeService(t) + h := newHarness(t, svc) + if h.link.SendTun(cloudwire.Tun{Dev: "x", Sid: 1, D: []byte{1}}, false) == nil { + t.Fatal("SendTun took a message with no uplink") + } + if h.link.SendTun(cloudwire.Tun{Dev: "x", Sid: 1, D: make([]byte, cloudwire.TunDataMax+1)}, false) == nil { + t.Fatal("SendTun took a d over the limit") + } +} + +// Every session the uplink carried ends with it. +func TestTheTunnelDropsWithTheUplink(t *testing.T) { + svc := newFakeService(t) + svc.afterHello = func(conn *websocket.Conn, _ string) { + time.Sleep(300 * time.Millisecond) + _ = conn.Close() + } + h := newHarness(t, svc) + h.setRoster(func(r *fakeRosterState) { r.tun = true }) + deadline := time.Now().Add(5 * time.Second) + for h.tunnel.dropped() == 0 { + if time.Now().After(deadline) { + t.Fatal("the Tunnel was not told the uplink ended") + } + time.Sleep(20 * time.Millisecond) + } +} + +// When nothing needs the link any more, the box asks the service to delete +// the home and deletes cloud.key on the receipt. The next link has a new home. +func TestTheLinkEndsWhenNothingNeedsIt(t *testing.T) { + svc := newFakeService(t) + svc.purge = true + h := newHarness(t, svc) + h.setRoster(func(r *fakeRosterState) { r.tun = true }) + h.nextRoster(t) + home := h.link.Status().Home + + h.setRoster(func(r *fakeRosterState) { r.gone = true }) + bye := svc.waitFor(cloudwire.MsgBye, 3*time.Second) + var b cloudwire.Bye + if cloudwire.Unmarshal(bye.B, &b) != nil || !b.Purge || b.Why != cloudwire.ByeRevoked { + t.Fatalf("bye %+v", b) + } + deadline := time.Now().Add(3 * time.Second) + for h.link.Status().Link != "off" { + if time.Now().After(deadline) { + t.Fatalf("still %+v after the receipt", h.link.Status()) + } + time.Sleep(20 * time.Millisecond) + } + if _, err := os.Stat(h.keyPath); !os.IsNotExist(err) { + t.Fatal("cloud.key survived the end of the link") + } + + h.setRoster(func(r *fakeRosterState) { r.gone, r.tun = false, true }) + if next := h.link.Status().Home; next == "" || next == home { + t.Fatalf("the next link's home is %q after %q", next, home) + } +} + +// A grant made before the link existed keeps its key and home id: the link +// takes them over. +func TestAGrantFromBeforeTheLinkKeepsItsHome(t *testing.T) { + kv := &memKV{} + keyPath := filepath.Join(t.TempDir(), "cloud.key") + key, err := createKey(keyPath) + if err != nil { + t.Fatal(err) + } + fp, pk, _ := fingerprint(key) + if err := saveJSON(kv, grantKey, grantRecord{ID: bytes.Repeat([]byte{4}, 16), KeyFp: fp, + Scopes: []string{protocol.ScopeLiveRead}, Purposes: []string{protocol.CloudPurposeServe}, + GrantKey: bytes.Repeat([]byte{7}, 32), Controller: cloudwire.ControllerSourceful}); err != nil { + t.Fatal(err) + } + svc := newFakeService(t) + link, err := New(Options{Site: &fakeSite{}, Roster: fakeRoster{}, KV: kv, KeyPath: keyPath, + ServiceKeys: []*ecdsa.PublicKey{&svc.key.PublicKey}}) + if err != nil { + t.Fatal(err) + } + st := link.Status() + if st.State != "on" || st.Link != "on" || st.Home != cloudwire.HomeID(pk) { + t.Fatalf("status %+v", st) + } + if rec, _ := loadJSON[linkRecord](kv, linkKey); rec == nil || rec.KeyFp != fp { + t.Fatalf("link record %+v", rec) + } +} + +// A grant on a box whose link is ending waits for the receipt. +func TestAGrantWaitsForAnEndingLink(t *testing.T) { + svc := newFakeService(t) + h := newHarness(t, svc) + h.setRoster(func(r *fakeRosterState) { r.tun = true }) + h.nextRoster(t) + h.setRoster(func(r *fakeRosterState) { r.gone = true }) + svc.waitFor(cloudwire.MsgBye, 3*time.Second) + if err := h.link.Grant(GrantRequest{Scopes: []string{protocol.ScopeLiveRead}, Policy: "p"}); !errors.Is(err, ErrLinkEnding) { + t.Fatalf("grant while the link ends: %v", err) + } +} + +// A QR that names the home reads it here: empty until the box links, and +// empty again while its link ends. +func TestHomeIsSetOnlyWhileLinked(t *testing.T) { + svc := newFakeService(t) + h := newHarness(t, svc) + if h.link.Home() != "" { + t.Fatal("a home before the box linked") + } + h.setRoster(func(r *fakeRosterState) { r.tun = true }) + if home := h.link.Home(); home == "" || home != h.link.Status().Home { + t.Fatalf("home %q, status %q", home, h.link.Status().Home) + } + h.nextRoster(t) + h.setRoster(func(r *fakeRosterState) { r.gone = true }) + svc.waitFor(cloudwire.MsgBye, 3*time.Second) + if h.link.Home() != "" { + t.Fatal("a home while the link ends") + } +} + +// Bulk is paced under the service's limits, and lane 0 never waits behind +// it: a phone's command and its answer go first. +func TestLaneZeroGoesAheadOfPacedBulk(t *testing.T) { + svc := newFakeService(t) + h := newHarness(t, svc) + h.setRoster(func(r *fakeRosterState) { r.tun = true }) + h.nextRoster(t) + // More bulk than the pacer's burst lets through at once. + big := make([]byte, 16_384) + n := 0 + for range 240 { + if h.link.SendTun(cloudwire.Tun{Dev: "q5Lw0Zb1", Sid: 1, D: big}, true) == nil { + n++ + } + } + if n < 230 { + t.Fatalf("only %d of 240 bulk messages queued", n) + } + time.Sleep(50 * time.Millisecond) + if h.link.SendTun(cloudwire.Tun{Dev: "q5Lw0Zb1", Sid: 2, D: []byte{1}}, false) != nil { + t.Fatal("lane 0 was refused") + } + start := time.Now() + for { + env := svc.waitFor(cloudwire.MsgTun, 5*time.Second) + var m cloudwire.Tun + _ = cloudwire.Unmarshal(env.B, &m) + if m.Sid == 2 { + break + } + } + if waited := time.Since(start); waited > 500*time.Millisecond { + t.Fatalf("lane 0 waited %s behind bulk", waited) + } +} + +// The pacer lets a burst through and then holds bulk to nine tenths of the +// service's rate. +func TestThePacerKeepsBulkUnderTheServicesLimits(t *testing.T) { + now := time.Unix(1_791_820_800, 0) + q := newPacer(now) + sent, at := 0, now + for at.Sub(now) < 10*time.Second { + at = at.Add(q.take(16_384, at)) + sent++ + } + limit := cloudwire.TunMsgBurst + 10*cloudwire.TunMsgsPerSecond + byteLimit := (cloudwire.TunByteBurst + 10*cloudwire.TunBytesPerSecond) / 16_384 + if sent > limit || sent > byteLimit { + t.Fatalf("%d bulk messages of 16 KiB in 10 s, over the service's %d messages or %d by bytes", sent, limit, byteLimit) + } +} + +// A roster built after a revoke landed carries the revoke, whatever snapshot +// the caller held from before. +func TestTheRosterReadsTheLinkAsItIsWhenDue(t *testing.T) { + svc := newFakeService(t) + link, err := New(Options{Site: &fakeSite{}, Roster: fakeRoster{state: &fakeRosterState{tun: true}}, KV: &memKV{}, + KeyPath: filepath.Join(t.TempDir(), "cloud.key"), ServiceKeys: []*ecdsa.PublicKey{&svc.key.PublicKey}}) + if err != nil { + t.Fatal(err) + } + if err := link.Grant(GrantRequest{Scopes: []string{protocol.ScopeLiveRead}, Policy: "p"}); err != nil { + t.Fatal(err) + } + stale := link.current() + if _, ok := link.takeRoster(false); !ok { + t.Fatal("no roster due after a grant") + } + if err := link.Revoke("test"); err != nil { + t.Fatal(err) + } + r, ok := link.takeRoster(false) + if !ok || len(r.Grant.ID) == 0 || len(r.Grant.Purposes) != 0 { + t.Fatalf("the roster after a revoke is %+v", r.Grant) + } + if !stale.copyOn() || stale.allowed(cloudwire.MsgLive) == link.current().allowed(cloudwire.MsgLive) { + t.Fatal("the stale snapshot is not stale") + } +} + +// A QR code shown and never used: when it runs out, nothing needs the link, +// and the link ends. +func TestAnUnusedCodeEndsTheLinkWhenItRunsOut(t *testing.T) { + svc := newFakeService(t) + svc.purge = true + h := newHarness(t, svc) + h.setRoster(func(r *fakeRosterState) { + r.gone, r.code, r.expires = true, bytes.Repeat([]byte{1}, 16), time.Now().Add(400*time.Millisecond) + }) + if h.link.Status().Link != "on" { + t.Fatal("a code on the screen did not link the box") + } + h.nextRoster(t) + svc.waitFor(cloudwire.MsgBye, 3*time.Second) + deadline := time.Now().Add(3 * time.Second) + for h.link.Status().Link != "off" { + if time.Now().After(deadline) { + t.Fatalf("the link outlived its unused code: %+v", h.link.Status()) + } + time.Sleep(20 * time.Millisecond) + } +} + +// A cloud.key already on disk is adopted, never replaced: its home id is +// one phones may dial. +func TestAKeyOnDiskIsAdoptedNeverReplaced(t *testing.T) { + svc := newFakeService(t) + keyPath := filepath.Join(t.TempDir(), "cloud.key") + key, err := createKey(keyPath) + if err != nil { + t.Fatal(err) + } + _, pk, _ := fingerprint(key) + if _, err := createKey(keyPath); err == nil { + t.Fatal("createKey replaced an existing key") + } + link, err := New(Options{Site: &fakeSite{}, Roster: fakeRoster{state: &fakeRosterState{tun: true}}, KV: &memKV{}, + KeyPath: keyPath, ServiceKeys: []*ecdsa.PublicKey{&svc.key.PublicKey}}) + if err != nil { + t.Fatal(err) + } + link.RosterChanged() + if home := link.Home(); home != cloudwire.HomeID(pk) { + t.Fatalf("linked as %q, the key on disk is %q", home, cloudwire.HomeID(pk)) + } +} + +// The readable copy turns on and off while the link stays up: readings start +// with the grant on the same connection and stop with the revoke. +func TestTheCopyTurnsOnAndOffOnALiveConnection(t *testing.T) { + svc := newFakeService(t) + h := newHarness(t, svc) + h.setRoster(func(r *fakeRosterState) { r.tun = true }) + h.nextRoster(t) + time.Sleep(300 * time.Millisecond) + h.grant(t) + svc.waitFor(cloudwire.MsgLive, 5*time.Second) + if err := h.link.Revoke("test"); err != nil { + t.Fatal(err) + } + r := h.nextRoster(t) + for len(r.Grant.Purposes) != 0 { + r = h.nextRoster(t) + } + time.Sleep(2500 * time.Millisecond) + svc.mu.Lock() + conns := svc.conns + var order []string + for _, env := range svc.got { + order = append(order, env.T) + } + svc.mu.Unlock() + if conns != 1 { + t.Fatalf("the copy turning on and off took %d connections", conns) + } + // off: roster only; on: readings; off again: nothing readable after the + // withdrawing roster. + firstLive, withdrawn := -1, -1 + rosters := 0 + for i, typ := range order { + switch typ { + case cloudwire.MsgLive: + if firstLive < 0 { + firstLive = i + } + if withdrawn >= 0 { + t.Fatalf("live after the withdrawing roster: %v", order) + } + case cloudwire.MsgRoster: + rosters++ + var ro cloudwire.Roster + _ = cloudwire.Unmarshal(svc.received()[i].B, &ro) + if len(ro.Grant.ID) > 0 && len(ro.Grant.Purposes) == 0 { + withdrawn = i + } + case cloudwire.MsgHB, cloudwire.MsgLP, cloudwire.MsgPrices: + if firstLive < 0 || withdrawn >= 0 { + t.Fatalf("%s outside the copy: %v", typ, order) + } + } + } + if firstLive <= 0 || withdrawn <= firstLive || rosters < 3 { + t.Fatalf("order %v", order) + } +} + +// The copy keeps flowing when the tunnel's own budget is spent. +func TestTheCopyFlowsPastTheTunnelBudget(t *testing.T) { + svc := newFakeService(t) + h := newHarness(t, svc) + h.link.mu.Lock() + h.link.tunBudget.add(h.link.uptimeMs(), tunBudgetBytes) + h.link.mu.Unlock() + h.setRoster(func(r *fakeRosterState) { r.tun = true }) + h.grant(t) + svc.waitFor(cloudwire.MsgLive, 5*time.Second) + h.site.setGrid(4321) + svc.waitFor(cloudwire.MsgLive, 7*time.Second) + if err := h.link.SendTun(cloudwire.Tun{Dev: "q5Lw0Zb1", Sid: 1, D: []byte{1}}, false); err == nil { + t.Fatal("tun went past its own budget") + } +} diff --git a/go/protocol/cloudwire/allowlist.go b/go/protocol/cloudwire/allowlist.go index 22fb4f80f..a0e79a0ba 100644 --- a/go/protocol/cloudwire/allowlist.go +++ b/go/protocol/cloudwire/allowlist.go @@ -33,16 +33,22 @@ var Fields = map[string][]string{ MsgEvent: {"eventId", "ts", "kind", "severity", "args", "src", "lp"}, - MsgRoster: {"rosterSeq", "ttlS", "devices", "grant"}, - "Device": {"id", "key", "role", "scopes", "epoch"}, + MsgRoster: {"rosterSeq", "ttlS", "devices", "grant", "pending"}, + "Device": {"id", "key", "role", "scopes", "epoch", "tun"}, "Grant": {"id", "scopes", "purposes", "controller"}, + "Pending": {"hash", "ttlS"}, MsgBye: {"why", "purge"}, + + // tun carries no header and no signature: d is Noise ciphertext its two + // ends authenticate, and the service reads only dev and sid. + MsgTun: {"dev", "sid", "d"}, } // Bodies maps each class the box sends to its body type. Classes the box does // not send yet (plan, health, tiles) have no entry; the service treats their -// bytes as opaque either way. +// bytes as opaque either way. tun is not here: it has no header and is not a +// class, and Unheaded holds it. var Bodies = map[string]any{ MsgHello: Hello{}, MsgLive: Live{}, @@ -53,3 +59,8 @@ var Bodies = map[string]any{ MsgRoster: Roster{}, MsgBye: Bye{}, } + +// Unheaded are the box's messages without the common header. +var Unheaded = map[string]any{ + MsgTun: Tun{}, +} diff --git a/go/protocol/cloudwire/allowlist_test.go b/go/protocol/cloudwire/allowlist_test.go index 9e07d4fe8..e78dace54 100644 --- a/go/protocol/cloudwire/allowlist_test.go +++ b/go/protocol/cloudwire/allowlist_test.go @@ -74,6 +74,9 @@ func TestEveryFieldHasARow(t *testing.T) { for class, body := range Bodies { walk(class, reflect.TypeOf(body), true) } + for t, body := range Unheaded { + walk(t, reflect.TypeOf(body), false) + } header, _, _ := cborFields(t, reflect.TypeOf(Header{})) if !slices.Equal(sorted(header), sorted(Fields["header"])) { t.Errorf("header: fields %v, rows %v", header, Fields["header"]) diff --git a/go/protocol/cloudwire/app.go b/go/protocol/cloudwire/app.go index d7005687c..701e6e44b 100644 --- a/go/protocol/cloudwire/app.go +++ b/go/protocol/cloudwire/app.go @@ -15,7 +15,8 @@ import ( // because the prologue binds it. Message 1 carries a now envelope, message 2 // the snapshot, and after that each box envelope the device may read rides, // unchanged, in an unpadded lane 1 frame inside a Noise transport message, -// exactly as appwire.Transport writes one. +// exactly as appwire.Transport writes one. The phone's session with its box +// rides the same transport as tun (tunnel.go). // // Every envelope between phone and service has an empty sig. Only box // envelopes carry signatures, and the phone checks them with the box key it @@ -36,6 +37,8 @@ const ( MsgUnsuspend = "unsuspend" // MsgWatch (shared with the downlink) changes whether this phone is // looking: hz 1 while it can see the home, 0 when it cannot. + // MsgTun (shared with the uplink, tunnel.go) carries the phone's + // session with its box, as AppTun, both ways. ) // Service to phone. @@ -49,8 +52,8 @@ const ( // CloseProtocol: the phone sent something this protocol does not allow. CloseProtocol = 4400 // CloseDenied is every refusal: no such home, a device not in the roster, - // a suspended device, an expired roster or a grant that is off. It does - // not say which. + // a suspended device, an expired roster, or a device that may neither + // read nor open a tunnel. It does not say which. CloseDenied = 4403 // CloseReplaced: a newer session from the same device took over. CloseReplaced = 4408 @@ -71,6 +74,14 @@ const ( // Now is message 1: whether the phone is looking. type Now struct { Hz int `cbor:"hz"` + // Pair is PairingHash of the code a new phone scanned. It admits a key + // the roster does not list yet, once, while the roster lists the hash. + Pair []byte `cbor:"pair,omitempty"` + // Tun is the first message of the phone's session with its box, + // handshake message 1. The service forwards it as soon as it has + // admitted the phone, so the box session is up one round trip after + // TLS rather than two. + Tun *AppTun `cbor:"tun,omitempty"` } // Suspend names the roster device an owner suspends or lifts a suspension diff --git a/go/protocol/cloudwire/app_vectors_test.go b/go/protocol/cloudwire/app_vectors_test.go index d5749cfc8..1806d4d97 100644 --- a/go/protocol/cloudwire/app_vectors_test.go +++ b/go/protocol/cloudwire/app_vectors_test.go @@ -35,9 +35,22 @@ type appVectorFile struct { HandshakeHash string `json:"handshakeHash"` Snapshot []string `json:"snapshot"` Transport []appVectorFrame `json:"transport"` + Pairing appVectorPairing `json:"pairing"` Notes map[string]string `json:"notes"` } +// appVectorPairing is a new phone's proof: the code it scanned, the hash the +// roster lists and now.pair carries, and the device id the service names it +// by until the roster lists it. Msg1NowTun is the same now with the box +// session's message 1 riding along, as a phone sends it to save a round trip. +type appVectorPairing struct { + Code string `json:"code"` + Hash string `json:"hash"` + Msg1Now string `json:"msg1Now"` + Msg1NowTun string `json:"msg1NowTun"` + DeviceID string `json:"deviceId"` +} + type appVectorFrame struct { From string `json:"from"` Name string `json:"name"` @@ -203,6 +216,23 @@ func TestAppSessionVectors(t *testing.T) { addFrame("phone", "watch away", frameOf(away)) suspend, _ := Unsigned(MsgSuspend, Suspend{DeviceID: "Zf09xQ2a"}) addFrame("phone", "suspend", frameOf(suspend)) + // The phone's own session with its box rides as tun: d is opaque here. + tunUp, _ := Unsigned(MsgTun, AppTun{Sid: 7, D: bytes.Repeat([]byte{0x9a}, 48)}) + addFrame("phone", "tun", frameOf(tunUp)) + tunDown, _ := Unsigned(MsgTun, AppTun{Sid: 7, D: bytes.Repeat([]byte{0x99}, 112)}) + addFrame("service", "tun", frameOf(tunDown)) + tunEnd, _ := Unsigned(MsgTun, AppTun{Sid: 7, D: []byte{}}) + addFrame("service", "tun end", frameOf(tunEnd)) + + code := bytes.Repeat([]byte{0x88}, 16) + pairNow, _ := Unsigned(MsgNow, Now{Hz: HzLooking, Pair: PairingHash(home, code)}) + pairNowTun, _ := Unsigned(MsgNow, Now{Hz: HzLooking, Pair: PairingHash(home, code), + Tun: &AppTun{Sid: 1, D: bytes.Repeat([]byte{0x9b}, 112)}}) + out.Pairing = appVectorPairing{ + Code: hex.EncodeToString(code), Hash: hex.EncodeToString(PairingHash(home, code)), + Msg1Now: hex.EncodeToString(pairNow), Msg1NowTun: hex.EncodeToString(pairNowTun), + DeviceID: DeviceID(phoneStatic.Public()), + } got, err := json.MarshalIndent(out, "", " ") if err != nil { diff --git a/go/protocol/cloudwire/messages.go b/go/protocol/cloudwire/messages.go index 8b89033dd..ba4b7512b 100644 --- a/go/protocol/cloudwire/messages.go +++ b/go/protocol/cloudwire/messages.go @@ -2,8 +2,8 @@ package cloudwire import "github.com/srcfl/ftw/go/protocol" -// Message types. The service sends exactly the first five; anything else from -// it closes the uplink. +// Message types. The service sends exactly the first five and tun; anything +// else from it closes the uplink. const ( MsgChallenge = "challenge" MsgHelloOK = "hello_ok" @@ -27,8 +27,10 @@ const ( MsgBye = "bye" ) -// Downlink is the closed set of types the service may send the box. -var Downlink = []string{MsgChallenge, MsgHelloOK, MsgWatch, MsgResync, MsgPurged} +// Downlink is the closed set of types the service may send the box. None of +// the first five asks the box to act. tun carries ciphertext the box decrypts +// itself and judges as it judges the same session over the relay. +var Downlink = []string{MsgChallenge, MsgHelloOK, MsgWatch, MsgResync, MsgPurged, MsgTun} // ClassScope is the scope a device needs to read each class the service // forwards. hb maps to "" because every device in the roster gets it: it @@ -60,9 +62,11 @@ const ( ByeUpdate = "update" ) -// What a purge receipt or a resync names. +// What a purge receipt or a resync names. PurgedCopy confirms that the +// readable copy of a grant is gone while the home stays, for its tunnel. const ( PurgedHome = "home" + PurgedCopy = "copy" PurgedStats = "stats" ResyncFull = "full" @@ -266,6 +270,10 @@ type Roster struct { TTLS int64 `cbor:"ttlS"` Devices []Device `cbor:"devices"` Grant Grant `cbor:"grant"` + // Pending lists the pairing codes the box shows now. Each admits one + // first session through the service, for a phone the roster does not + // list yet. + Pending []Pending `cbor:"pending"` } // Device is one paired phone. @@ -278,9 +286,22 @@ type Device struct { // role table. Scopes []string `cbor:"scopes"` Epoch uint64 `cbor:"epoch"` + // Tun says the device may open a session to the box through the + // service. It needs no grant: the readable copy is a purpose on top. + Tun bool `cbor:"tun"` } -// Grant is what the owner turned on. +// Pending is one pairing code on the box's screen, by its PairingHash. +type Pending struct { + Hash []byte `cbor:"hash"` + // TTLS is how long the code has left, counted from when the service + // receives the roster. + TTLS int64 `cbor:"ttlS"` +} + +// Grant is what the owner turned on. With the readable copy off it has no +// id, scopes or purposes. While a revoke waits for its receipt it keeps the +// id and has neither scopes nor purposes. type Grant struct { ID []byte `cbor:"id"` Scopes []string `cbor:"scopes"` diff --git a/go/protocol/cloudwire/testdata/cloud-app-vectors.json b/go/protocol/cloudwire/testdata/cloud-app-vectors.json index b49c1b364..feb481ec0 100644 --- a/go/protocol/cloudwire/testdata/cloud-app-vectors.json +++ b/go/protocol/cloudwire/testdata/cloud-app-vectors.json @@ -43,8 +43,33 @@ "name": "suspend", "frame": "010100000026a3616253a1686465766963654964685a6630397851326161746773757370656e646373696740", "message": "00000000000000018ef98c7df1fa903675ab1bf0e8aff5cd342be0cdc21b8ead3d04a153cf0fe6ce0d28f5fdf88e409e68eadc1827bfc61039294263085fc27f2b9b980b" + }, + { + "from": "phone", + "name": "tun", + "frame": "01010000004aa36162583aa2616458309a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a637369640761746374756e6373696740", + "message": "00000000000000022731255fc9340e3fc8fe0933b3cea6ee0a7bade6b3019c5e9b09696018bb2c57b1cc7cf32d3d556cc16ab7187ed6cba0c662469026d4c99a61f40e131cbd35f3b3d25ed22d445a9664f429271c8539fa0401884a10d87cdca3c10fd0e3273a2c" + }, + { + "from": "service", + "name": "tun", + "frame": "01010000008aa36162587aa26164587099999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999637369640761746374756e6373696740", + "message": "00000000000000026cf288f8e3930b62ce5f00bda38eb90f3dc6fe833042f9cd777b8369dbb658abdf10ea42bdac4e40bae9482ae4e95a43e64a0d34d9c6aa7afc59c2455e7c3f8d1b4b081b2a8353f22e6cf6be6340e10a9dd0b9af405ad53d46feb2d6985d8426b76a8c887ff6f9b25c16be90dd9ba24455db43b6f869a1d7a51866bd4e104e9faacce088903a41f9a3c4d95c6d4dd2db54cdbf607a569cf1956cce138dc461c9" + }, + { + "from": "service", + "name": "tun end", + "frame": "010100000018a3616249a2616440637369640761746374756e6373696740", + "message": "0000000000000003237e42f082e81124d36ef77e2467933a52c8c41f3ecf41b2247ee239a9dd9dfd72d74d129a742d70765cbaf6e61c" } ], + "pairing": { + "code": "88888888888888888888888888888888", + "hash": "b4e707c0ac2241df0e0325b42b951b3c71bbd5099bc3c6f2894a48ee23fa3563", + "msg1Now": "a36162582ca262687a0164706169725820b4e707c0ac2241df0e0325b42b951b3c71bbd5099bc3c6f2894a48ee23fa35636174636e6f776373696740", + "msg1NowTun": "a3616258aaa362687a016374756ea2616458709b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b9b637369640164706169725820b4e707c0ac2241df0e0325b42b951b3c71bbd5099bc3c6f2894a48ee23fa35636174636e6f776373696740", + "deviceId": "99CBpeiI" + }, "notes": { "keys": "Secrets are SHA-256 of the labels in app_vectors_test.go. X25519 clamps them.", "snapshot": "Message 2 holds the cloud-vectors.json envelopes named here, in this order.", diff --git a/go/protocol/cloudwire/testdata/cloud-vectors.json b/go/protocol/cloudwire/testdata/cloud-vectors.json index 9ad224461..7a9dd7804 100644 --- a/go/protocol/cloudwire/testdata/cloud-vectors.json +++ b/go/protocol/cloudwire/testdata/cloud-vectors.json @@ -66,11 +66,11 @@ "name": "roster", "signer": "box", "t": "roster", - "body": "a8637365710264626f6f7450b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b06474746c531a00015180656772616e74a462696450444444444444444444444444444444446673636f706573826d6674772e6c6976652e726561646f6674772e6576656e74732e7265616468707572706f736573816573657276656a636f6e74726f6c6c657269736f7572636566756c6677616c6c4d731b000001a130d7a00a676465766963657382a56269646871354c77305a6231636b65795820555555555555555555555555555555555555555555555555555555555555555564726f6c65656f776e65726565706f6368016673636f706573856d6674772e6c6976652e72656164706674772e686973746f72792e726561646d6674772e706c616e2e726561646f6674772e6865616c74682e726561646f6674772e6576656e74732e72656164a5626964685a66303978513261636b65795820666666666666666666666666666666666666666666666666666666666666666664726f6c65667669657765726565706f6368026673636f706573816d6674772e6c6976652e7265616468757074696d654d7319ea6a69726f7374657253657104", - "signedInputSha256": "f0926ee704ac9558f742f604315ef8ea260a9edef63a210172d9c299d5b171cf", - "sig": "cbb6ead0b031278d2252885c2a6f7770c98335676b1cf29f57115c9e76609c81e53eb6260d79df419aebc93ed29c80935190368e1730de2d703867abe31bf124", - "envelope": "a361625901b2a8637365710264626f6f7450b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b06474746c531a00015180656772616e74a462696450444444444444444444444444444444446673636f706573826d6674772e6c6976652e726561646f6674772e6576656e74732e7265616468707572706f736573816573657276656a636f6e74726f6c6c657269736f7572636566756c6677616c6c4d731b000001a130d7a00a676465766963657382a56269646871354c77305a6231636b65795820555555555555555555555555555555555555555555555555555555555555555564726f6c65656f776e65726565706f6368016673636f706573856d6674772e6c6976652e72656164706674772e686973746f72792e726561646d6674772e706c616e2e726561646f6674772e6865616c74682e726561646f6674772e6576656e74732e72656164a5626964685a66303978513261636b65795820666666666666666666666666666666666666666666666666666666666666666664726f6c65667669657765726565706f6368026673636f706573816d6674772e6c6976652e7265616468757074696d654d7319ea6a69726f7374657253657104617466726f73746572637369675840cbb6ead0b031278d2252885c2a6f7770c98335676b1cf29f57115c9e76609c81e53eb6260d79df419aebc93ed29c80935190368e1730de2d703867abe31bf124", - "frame": "010100000207a361625901b2a8637365710264626f6f7450b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b06474746c531a00015180656772616e74a462696450444444444444444444444444444444446673636f706573826d6674772e6c6976652e726561646f6674772e6576656e74732e7265616468707572706f736573816573657276656a636f6e74726f6c6c657269736f7572636566756c6677616c6c4d731b000001a130d7a00a676465766963657382a56269646871354c77305a6231636b65795820555555555555555555555555555555555555555555555555555555555555555564726f6c65656f776e65726565706f6368016673636f706573856d6674772e6c6976652e72656164706674772e686973746f72792e726561646d6674772e706c616e2e726561646f6674772e6865616c74682e726561646f6674772e6576656e74732e72656164a5626964685a66303978513261636b65795820666666666666666666666666666666666666666666666666666666666666666664726f6c65667669657765726565706f6368026673636f706573816d6674772e6c6976652e7265616468757074696d654d7319ea6a69726f7374657253657104617466726f73746572637369675840cbb6ead0b031278d2252885c2a6f7770c98335676b1cf29f57115c9e76609c81e53eb6260d79df419aebc93ed29c80935190368e1730de2d703867abe31bf124" + "body": "a9637365710264626f6f7450b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b06474746c531a00015180656772616e74a462696450444444444444444444444444444444446673636f706573826d6674772e6c6976652e726561646f6674772e6576656e74732e7265616468707572706f736573816573657276656a636f6e74726f6c6c657269736f7572636566756c6677616c6c4d731b000001a130d7a00a676465766963657382a66269646871354c77305a6231636b6579582055555555555555555555555555555555555555555555555555555555555555556374756ef464726f6c65656f776e65726565706f6368016673636f706573856d6674772e6c6976652e72656164706674772e686973746f72792e726561646d6674772e706c616e2e726561646f6674772e6865616c74682e726561646f6674772e6576656e74732e72656164a6626964685a66303978513261636b6579582066666666666666666666666666666666666666666666666666666666666666666374756ef464726f6c65667669657765726565706f6368026673636f706573816d6674772e6c6976652e726561646770656e64696e678068757074696d654d7319ea6a69726f7374657253657104", + "signedInputSha256": "54a45d8ab064d1d9fda8dfd3d43708dca9294daefe67078b15ab17ae1c3c086a", + "sig": "26de4f6b491496569361fce489167aa3a445c91235689a79f5499f5a65d4ed5e35ecd601251013d345f8171904bd5b508e0a05d543dc25f2335b2d7a743ad916", + "envelope": "a361625901c5a9637365710264626f6f7450b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b06474746c531a00015180656772616e74a462696450444444444444444444444444444444446673636f706573826d6674772e6c6976652e726561646f6674772e6576656e74732e7265616468707572706f736573816573657276656a636f6e74726f6c6c657269736f7572636566756c6677616c6c4d731b000001a130d7a00a676465766963657382a66269646871354c77305a6231636b6579582055555555555555555555555555555555555555555555555555555555555555556374756ef464726f6c65656f776e65726565706f6368016673636f706573856d6674772e6c6976652e72656164706674772e686973746f72792e726561646d6674772e706c616e2e726561646f6674772e6865616c74682e726561646f6674772e6576656e74732e72656164a6626964685a66303978513261636b6579582066666666666666666666666666666666666666666666666666666666666666666374756ef464726f6c65667669657765726565706f6368026673636f706573816d6674772e6c6976652e726561646770656e64696e678068757074696d654d7319ea6a69726f7374657253657104617466726f7374657263736967584026de4f6b491496569361fce489167aa3a445c91235689a79f5499f5a65d4ed5e35ecd601251013d345f8171904bd5b508e0a05d543dc25f2335b2d7a743ad916", + "frame": "01010000021aa361625901c5a9637365710264626f6f7450b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b06474746c531a00015180656772616e74a462696450444444444444444444444444444444446673636f706573826d6674772e6c6976652e726561646f6674772e6576656e74732e7265616468707572706f736573816573657276656a636f6e74726f6c6c657269736f7572636566756c6677616c6c4d731b000001a130d7a00a676465766963657382a66269646871354c77305a6231636b6579582055555555555555555555555555555555555555555555555555555555555555556374756ef464726f6c65656f776e65726565706f6368016673636f706573856d6674772e6c6976652e72656164706674772e686973746f72792e726561646d6674772e706c616e2e726561646f6674772e6865616c74682e726561646f6674772e6576656e74732e72656164a6626964685a66303978513261636b6579582066666666666666666666666666666666666666666666666666666666666666666374756ef464726f6c65667669657765726565706f6368026673636f706573816d6674772e6c6976652e726561646770656e64696e678068757074696d654d7319ea6a69726f7374657253657104617466726f7374657263736967584026de4f6b491496569361fce489167aa3a445c91235689a79f5499f5a65d4ed5e35ecd601251013d345f8171904bd5b508e0a05d543dc25f2335b2d7a743ad916" }, { "name": "live full", @@ -191,6 +191,66 @@ "sig": "e75a581948135cbd2b54eec888fa0396f57b9159de07881ed78cc51b271cad39f1e81431d1125b51040c7794b109ff106702004dccb0285a61a879d29afb04bb", "envelope": "a3616258b7ab626c706b6a33485a4f524e435748736274731b000001a130d7d200637365710b637372636b4455595757516d6e2d58676461726773a1636b77686431382e3464626f6f7450b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0646b696e6478196368617267696e672e73657373696f6e5f636f6d706c6574656677616c6c4d731b000001a130d7d2c8676576656e7449646c5a4739755a5441774d44417868736576657269747964696e666f68757074696d654d731a00011d286174656576656e74637369675840e75a581948135cbd2b54eec888fa0396f57b9159de07881ed78cc51b271cad39f1e81431d1125b51040c7794b109ff106702004dccb0285a61a879d29afb04bb", "frame": "01010000010aa3616258b7ab626c706b6a33485a4f524e435748736274731b000001a130d7d200637365710b637372636b4455595757516d6e2d58676461726773a1636b77686431382e3464626f6f7450b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0646b696e6478196368617267696e672e73657373696f6e5f636f6d706c6574656677616c6c4d731b000001a130d7d2c8676576656e7449646c5a4739755a5441774d44417868736576657269747964696e666f68757074696d654d731a00011d286174656576656e74637369675840e75a581948135cbd2b54eec888fa0396f57b9159de07881ed78cc51b271cad39f1e81431d1125b51040c7794b109ff106702004dccb0285a61a879d29afb04bb" + }, + { + "name": "roster tunnel only", + "signer": "box", + "t": "roster", + "body": "a9637365710c64626f6f7450b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b06474746c531a00015180656772616e74a4626964406673636f7065738068707572706f736573806a636f6e74726f6c6c657269736f7572636566756c6677616c6c4d731b000001a130d7d6b0676465766963657382a66269646871354c77305a6231636b6579582055555555555555555555555555555555555555555555555555555555555555556374756ef564726f6c65656f776e65726565706f6368016673636f70657380a6626964685a66303978513261636b6579582066666666666666666666666666666666666666666666666666666666666666666374756ef564726f6c65667669657765726565706f6368026673636f706573806770656e64696e6781a264686173685820b4e707c0ac2241df0e0325b42b951b3c71bbd5099bc3c6f2894a48ee23fa35636474746c5319025868757074696d654d731a0001211069726f7374657253657105", + "signedInputSha256": "30ae9e4de78ae0f21421f8a7b172759f3b244da825c0072f2781cb49a73a907c", + "sig": "f9b4b3e144492cf784a07fce0272fe0252d3d71164645b092ebc086a4eab2f0f4e8c11e20aec3567e454a5641a06a97166eaf20ccdd8a022f1eaaa2a18a52e8b", + "envelope": "a36162590168a9637365710c64626f6f7450b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b06474746c531a00015180656772616e74a4626964406673636f7065738068707572706f736573806a636f6e74726f6c6c657269736f7572636566756c6677616c6c4d731b000001a130d7d6b0676465766963657382a66269646871354c77305a6231636b6579582055555555555555555555555555555555555555555555555555555555555555556374756ef564726f6c65656f776e65726565706f6368016673636f70657380a6626964685a66303978513261636b6579582066666666666666666666666666666666666666666666666666666666666666666374756ef564726f6c65667669657765726565706f6368026673636f706573806770656e64696e6781a264686173685820b4e707c0ac2241df0e0325b42b951b3c71bbd5099bc3c6f2894a48ee23fa35636474746c5319025868757074696d654d731a0001211069726f7374657253657105617466726f73746572637369675840f9b4b3e144492cf784a07fce0272fe0252d3d71164645b092ebc086a4eab2f0f4e8c11e20aec3567e454a5641a06a97166eaf20ccdd8a022f1eaaa2a18a52e8b", + "frame": "0101000001bda36162590168a9637365710c64626f6f7450b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b06474746c531a00015180656772616e74a4626964406673636f7065738068707572706f736573806a636f6e74726f6c6c657269736f7572636566756c6677616c6c4d731b000001a130d7d6b0676465766963657382a66269646871354c77305a6231636b6579582055555555555555555555555555555555555555555555555555555555555555556374756ef564726f6c65656f776e65726565706f6368016673636f70657380a6626964685a66303978513261636b6579582066666666666666666666666666666666666666666666666666666666666666666374756ef564726f6c65667669657765726565706f6368026673636f706573806770656e64696e6781a264686173685820b4e707c0ac2241df0e0325b42b951b3c71bbd5099bc3c6f2894a48ee23fa35636474746c5319025868757074696d654d731a0001211069726f7374657253657105617466726f73746572637369675840f9b4b3e144492cf784a07fce0272fe0252d3d71164645b092ebc086a4eab2f0f4e8c11e20aec3567e454a5641a06a97166eaf20ccdd8a022f1eaaa2a18a52e8b" + }, + { + "name": "roster revoking", + "signer": "box", + "t": "roster", + "body": "a9637365710d64626f6f7450b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b06474746c531a00015180656772616e74a462696450444444444444444444444444444444446673636f7065738068707572706f736573806a636f6e74726f6c6c657269736f7572636566756c6677616c6c4d731b000001a130d7da98676465766963657381a66269646871354c77305a6231636b6579582055555555555555555555555555555555555555555555555555555555555555556374756ef564726f6c65656f776e65726565706f6368016673636f706573806770656e64696e678068757074696d654d731a000124f869726f7374657253657106", + "signedInputSha256": "d8775b10218f6c69d9b7c9064d1cea8713058eaa9d91aeeba2e2f29419b7f9cc", + "sig": "4332ec4bf6f166175f50eecea08ecbc981d762c10b37f9e676d7e0a763b5d73e58533683fd6b566cf0a33feff903af40d4bf70bedc5245b2696fa85a3191186f", + "envelope": "a3616258f5a9637365710d64626f6f7450b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b06474746c531a00015180656772616e74a462696450444444444444444444444444444444446673636f7065738068707572706f736573806a636f6e74726f6c6c657269736f7572636566756c6677616c6c4d731b000001a130d7da98676465766963657381a66269646871354c77305a6231636b6579582055555555555555555555555555555555555555555555555555555555555555556374756ef564726f6c65656f776e65726565706f6368016673636f706573806770656e64696e678068757074696d654d731a000124f869726f7374657253657106617466726f737465726373696758404332ec4bf6f166175f50eecea08ecbc981d762c10b37f9e676d7e0a763b5d73e58533683fd6b566cf0a33feff903af40d4bf70bedc5245b2696fa85a3191186f", + "frame": "010100000149a3616258f5a9637365710d64626f6f7450b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b06474746c531a00015180656772616e74a462696450444444444444444444444444444444446673636f7065738068707572706f736573806a636f6e74726f6c6c657269736f7572636566756c6677616c6c4d731b000001a130d7da98676465766963657381a66269646871354c77305a6231636b6579582055555555555555555555555555555555555555555555555555555555555555556374756ef564726f6c65656f776e65726565706f6368016673636f706573806770656e64696e678068757074696d654d731a000124f869726f7374657253657106617466726f737465726373696758404332ec4bf6f166175f50eecea08ecbc981d762c10b37f9e676d7e0a763b5d73e58533683fd6b566cf0a33feff903af40d4bf70bedc5245b2696fa85a3191186f" + }, + { + "name": "purged copy", + "signer": "service", + "t": "purged", + "body": "a36461744d731b000001a130dc33e0647768617464636f7079676772616e7449645044444444444444444444444444444444", + "signedInputSha256": "72a96fe8a54dcebc90f78451152b7223088bcfcc7d48877006e1b3b6fc1aeed9", + "sig": "25e59e26aa33f7aaa1f4e463b721ea04c125f4cf1cec9f547dc49d07143e786dc35c05cd4320753d703759116c312e77f386579e61f1649763435fcbac5079c1", + "envelope": "a361625832a36461744d731b000001a130dc33e0647768617464636f7079676772616e744964504444444444444444444444444444444461746670757267656463736967584025e59e26aa33f7aaa1f4e463b721ea04c125f4cf1cec9f547dc49d07143e786dc35c05cd4320753d703759116c312e77f386579e61f1649763435fcbac5079c1", + "frame": "010100000086a361625832a36461744d731b000001a130dc33e0647768617464636f7079676772616e744964504444444444444444444444444444444461746670757267656463736967584025e59e26aa33f7aaa1f4e463b721ea04c125f4cf1cec9f547dc49d07143e786dc35c05cd4320753d703759116c312e77f386579e61f1649763435fcbac5079c1" + }, + { + "name": "tun down", + "signer": "", + "t": "tun", + "body": "a36164587099999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999636465766871354c77305a62316373696407", + "signedInputSha256": "09de8e31df87d982960f45e5bf4315b72dd8c315ba56a506d367ef7400596c7a", + "sig": "", + "envelope": "a361625887a36164587099999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999636465766871354c77305a6231637369640761746374756e6373696740", + "frame": "010100000097a361625887a36164587099999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999636465766871354c77305a6231637369640761746374756e6373696740" + }, + { + "name": "tun up", + "signer": "", + "t": "tun", + "body": "a3616458309a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a636465766871354c77305a62316373696407", + "signedInputSha256": "302a45e7a661c5e8ad29bf2e1c56ad4c544736b90630a2553dc78495da275c9d", + "sig": "", + "envelope": "a361625847a3616458309a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a636465766871354c77305a6231637369640761746374756e6373696740", + "frame": "010100000057a361625847a3616458309a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a636465766871354c77305a6231637369640761746374756e6373696740" + }, + { + "name": "tun end", + "signer": "", + "t": "tun", + "body": "a3616440636465766871354c77305a62316373696407", + "signedInputSha256": "e3e082a098ccdc6cb7c8cd2b863db4fd5060b40358ac3941d7ae983d83e35f27", + "sig": "", + "envelope": "a3616256a3616440636465766871354c77305a6231637369640761746374756e6373696740", + "frame": "010100000025a3616256a3616440636465766871354c77305a6231637369640761746374756e6373696740" } ] } diff --git a/go/protocol/cloudwire/tunnel.go b/go/protocol/cloudwire/tunnel.go new file mode 100644 index 000000000..cb5537d21 --- /dev/null +++ b/go/protocol/cloudwire/tunnel.go @@ -0,0 +1,108 @@ +package cloudwire + +import ( + "crypto/sha256" + "encoding/base64" + + "github.com/srcfl/ftw/go/protocol/appwire" +) + +// The tunnel: a phone's own session with its box, carried by FTW Cloud. +// docs/protocol-cloud.md, "Tunnel", is the normative text. +// +// The phone sends tun {sid, d} on its /v1/app session. d is one message of +// the phone's session with the box, ftw.session.v1 (Noise IK to the box's +// static key), byte for byte as it would cross the relay. The service adds +// the device id of the roster entry its own handshake matched and sends +// tun {dev, sid, d} down /v1/box. The box answers tun {dev, sid, d} up, and +// the service hands {sid, d} to that device's session. The service never +// takes dev from a phone. It can drop or delay d; it cannot read or forge it, +// because only the phone and the box hold the session's keys, and Noise +// transport refuses a replay. +// +// An empty d ends the session it names, in either direction: the phone +// closing it, the box dropping it, or the service saying that the other end +// is gone. + +// MsgTun is the tunnel message on both sockets. +const MsgTun = "tun" + +// CapTun in hello.caps says the box takes tun. The service sends tun to no +// other box: an older one closes the uplink on a type it does not know. +const CapTun = MsgTun + +// Sizes. The largest d is one transport message around the app protocol's +// largest bulk frame. A handshake message is far smaller. +const ( + TunDataMax = 16384 + appwire.TransportOverhead + // MaxTunFrame bounds one tun frame on either socket: d, the device id, + // the sid and the envelope around them. + MaxTunFrame = TunDataMax + 256 +) + +// The service's limits on tun, per home and per direction: a burst, then a +// rate. Past them it drops what is over; at twice them it closes. The box +// paces its bulk under them and never meets them. +const ( + TunMsgBurst = 240 + TunMsgsPerSecond = 60 + TunByteBurst = 4 << 20 + TunBytesPerSecond = 512 << 10 +) + +// Tun is the tunnel message on /v1/box, both ways. +type Tun struct { + // Dev is the roster device id whose session this is. The service sets + // it from the key its own handshake with the phone proved. + Dev string `cbor:"dev"` + // Sid is the phone's number for one session with the box, drawn at + // random for each new session, so that the late end of an older one + // can never end it. A new sid from the same device starts a new + // session and ends the older one. + Sid uint32 `cbor:"sid"` + // D is one message of that session, or empty to end it. + D []byte `cbor:"d"` +} + +// AppTun is the tunnel message on /v1/app, both ways. It has no device id: +// the service knows the device from the handshake, and ignores a dev a phone +// adds. +type AppTun struct { + Sid uint32 `cbor:"sid"` + D []byte `cbor:"d"` +} + +// Ended reports whether the message ends its session. +func (t Tun) Ended() bool { return len(t.D) == 0 } + +// DeviceID is the box's name for a paired device: the first 8 characters of +// its Noise static key in base64url without padding. The roster's ids are cut +// this way, and the service names a device it admitted for pairing the same +// way before the roster lists it. +func DeviceID(key []byte) string { + s := base64.RawURLEncoding.EncodeToString(key) + if len(s) <= 8 { + return s + } + return s[:8] +} + +// PairingTag starts the hash a pending pairing is listed by. +const PairingTag = "ftw.cloud.pair.v1\x00" + +// PairingHashBytes is the length of a pairing hash. +const PairingHashBytes = sha256.Size + +// PairingHash is what the roster lists for a pairing code the box shows, and +// what a new phone proves in now.pair: SHA-256(PairingTag ‖ home ‖ code). +// The service never sees the code itself, so it cannot pair a key of its own +// with the box; the box still checks the code inside the phone's Noise +// handshake. Only QR codes are listed. A spoken code has 40 bits, and its +// hash would let the service find it offline. +func PairingHash(home string, code []byte) []byte { + h := sha256.New() + h.Write([]byte(PairingTag)) + h.Write([]byte(home)) + h.Write(code) + return h.Sum(nil) +} diff --git a/go/protocol/cloudwire/tunnel_test.go b/go/protocol/cloudwire/tunnel_test.go new file mode 100644 index 000000000..d817c817d --- /dev/null +++ b/go/protocol/cloudwire/tunnel_test.go @@ -0,0 +1,102 @@ +package cloudwire + +import ( + "bytes" + "encoding/base64" + "encoding/hex" + "slices" + "strings" + "testing" + + "github.com/srcfl/ftw/go/protocol/appwire" +) + +// The largest d fits the frame limit on both sockets, with the longest +// device id the box makes. +func TestTheLargestTunFitsItsFrame(t *testing.T) { + if TunDataMax != appwire.BulkBuckets[len(appwire.BulkBuckets)-1]+appwire.TransportOverhead { + t.Fatalf("TunDataMax %d is not the largest bulk bucket in a transport message", TunDataMax) + } + body, err := Marshal(Tun{Dev: strings.Repeat("x", 8), Sid: ^uint32(0), D: make([]byte, TunDataMax)}) + if err != nil { + t.Fatal(err) + } + raw, err := Envelope{T: MsgTun, B: body}.Encode() + if err != nil { + t.Fatal(err) + } + frame, err := EncodeFrame(raw) + if err != nil { + t.Fatal(err) + } + if len(frame) > MaxTunFrame { + t.Fatalf("a full tun frame is %d bytes, over MaxTunFrame %d", len(frame), MaxTunFrame) + } + if MaxTunFrame+appwire.TransportOverhead > MaxPayload { + t.Fatal("a tun on the app session would not fit one Noise message") + } +} + +// The downlink grows by tun alone. +func TestTheDownlinkIsSixTypes(t *testing.T) { + want := []string{MsgChallenge, MsgHelloOK, MsgWatch, MsgResync, MsgPurged, MsgTun} + if !slices.Equal(Downlink, want) { + t.Fatalf("Downlink is %v, want %v", Downlink, want) + } +} + +// A phone's tun carries no device id, and one it adds is not read. +func TestAnAppTunHasNoDevice(t *testing.T) { + forged, err := Marshal(map[string]any{"dev": "someone", "sid": 3, "d": []byte{1}}) + if err != nil { + t.Fatal(err) + } + var got AppTun + if err := Unmarshal(forged, &got); err != nil { + t.Fatal(err) + } + if got.Sid != 3 || !bytes.Equal(got.D, []byte{1}) { + t.Fatalf("decoded %+v", got) + } + var box Tun + if err := Unmarshal(forged, &box); err != nil || box.Dev != "someone" { + t.Fatalf("the uplink shape lost its dev: %+v %v", box, err) + } +} + +func TestEndedIsAnEmptyD(t *testing.T) { + if !(Tun{D: nil}).Ended() || !(Tun{D: []byte{}}).Ended() || (Tun{D: []byte{0}}).Ended() { + t.Fatal("only an empty d ends a session") + } +} + +// The device id is the box's: the first eight characters of the key in +// base64url. +func TestDeviceIDIsTheBoxs(t *testing.T) { + key := bytes.Repeat([]byte{0x55}, 32) + if got, want := DeviceID(key), base64.RawURLEncoding.EncodeToString(key)[:8]; got != want { + t.Fatalf("DeviceID = %q, want %q", got, want) + } +} + +// The pairing hash binds the home and the code, and nothing else. +func TestPairingHash(t *testing.T) { + home := "fjzyr67buzpyncqmjliwgezvde" + code := bytes.Repeat([]byte{0x88}, 16) + h := PairingHash(home, code) + if len(h) != PairingHashBytes { + t.Fatalf("%d bytes", len(h)) + } + // The known answer is in cloud-app-vectors.json, under pairing. + if bytes.Equal(h, PairingHash("ajzyr67buzpyncqmjliwgezvde", code)) { + t.Fatal("two homes share a pairing hash") + } + other := bytes.Clone(code) + other[0] ^= 1 + if bytes.Equal(h, PairingHash(home, other)) { + t.Fatal("two codes share a pairing hash") + } + if hex.EncodeToString(h) == hex.EncodeToString(code) { + t.Fatal("the hash is the code") + } +} diff --git a/go/protocol/cloudwire/vectors_test.go b/go/protocol/cloudwire/vectors_test.go index 172253841..448378196 100644 --- a/go/protocol/cloudwire/vectors_test.go +++ b/go/protocol/cloudwire/vectors_test.go @@ -37,8 +37,8 @@ type vectorFile struct { } type vectorKey struct { - D string `json:"d"` - PK string `json:"pk"` + D string `json:"d"` + PK string `json:"pk"` // Home is the box key's home id. The service signs over the box's home // too, so its own key has none. Home string `json:"home,omitempty"` @@ -106,8 +106,8 @@ func vectorBodies(grantKey []byte, boxPK []byte) []struct { Scopes: []string{protocol.ScopeLiveRead}, Epoch: 2}, }, Grant: Grant{ID: bytes.Repeat([]byte{0x44}, GrantIDBytes), - Scopes: []string{protocol.ScopeLiveRead, protocol.ScopeEventsRead}, - Purposes: []string{protocol.CloudPurposeServe}, Controller: ControllerSourceful}, + Scopes: []string{protocol.ScopeLiveRead, protocol.ScopeEventsRead}, + Purposes: []string{protocol.CloudPurposeServe}, Controller: ControllerSourceful}, }}, {"live full", "box", MsgLive, Live{ Header: h(3, 60_020), @@ -174,6 +174,32 @@ func vectorBodies(grantKey []byte, boxPK []byte) []struct { Kind: protocol.PushChargingSessionComplete, Severity: SeverityInfo, Args: map[string]string{"kwh": "18.4"}, Src: charger, LP: garage, }}, + // The tunnel. A box linked at pairing, with the readable copy off: + // two phones that may open a session, one QR code on the screen. + {"roster tunnel only", "box", MsgRoster, Roster{ + Header: h(12, 74_000), RosterSeq: 5, TTLS: 86_400, + Devices: []Device{ + {ID: "q5Lw0Zb1", Key: bytes.Repeat([]byte{0x55}, 32), Role: "owner", Scopes: []string{}, Epoch: 1, Tun: true}, + {ID: "Zf09xQ2a", Key: bytes.Repeat([]byte{0x66}, 32), Role: "viewer", Scopes: []string{}, Epoch: 2, Tun: true}, + }, + Grant: Grant{Scopes: []string{}, Purposes: []string{}, Controller: ControllerSourceful}, + Pending: []Pending{{Hash: PairingHash(HomeID(boxPK), bytes.Repeat([]byte{0x88}, 16)), TTLS: 600}}, + }}, + // A revoke that waits for its receipt: the grant keeps its id and + // loses its scopes and purposes. + {"roster revoking", "box", MsgRoster, Roster{ + Header: h(13, 75_000), RosterSeq: 6, TTLS: 86_400, + Devices: []Device{ + {ID: "q5Lw0Zb1", Key: bytes.Repeat([]byte{0x55}, 32), Role: "owner", Scopes: []string{}, Epoch: 1, Tun: true}, + }, + Grant: Grant{ID: bytes.Repeat([]byte{0x44}, GrantIDBytes), Scopes: []string{}, Purposes: []string{}, + Controller: ControllerSourceful}, + }}, + {"purged copy", "service", MsgPurged, Purged{AtMs: 1_791_821_100_000, What: PurgedCopy, + GrantID: bytes.Repeat([]byte{0x44}, GrantIDBytes)}}, + {"tun down", "", MsgTun, Tun{Dev: "q5Lw0Zb1", Sid: 7, D: bytes.Repeat([]byte{0x99}, 112)}}, + {"tun up", "", MsgTun, Tun{Dev: "q5Lw0Zb1", Sid: 7, D: bytes.Repeat([]byte{0x9a}, 48)}}, + {"tun end", "", MsgTun, Tun{Dev: "q5Lw0Zb1", Sid: 7, D: []byte{}}}, } } @@ -185,14 +211,14 @@ func TestGoldenVectors(t *testing.T) { grantKey := bytes.Repeat([]byte{0x77}, GrantKeyBytes) var stored vectorFile - if !*update { - raw, err := os.ReadFile(vectorsPath) - if err != nil { - t.Fatalf("%v (run with -update to create it)", err) - } + raw, err := os.ReadFile(vectorsPath) + switch { + case err == nil: if err := json.Unmarshal(raw, &stored); err != nil { t.Fatal(err) } + case !*update: + t.Fatalf("%v (run with -update to create it)", err) } out := vectorFile{ @@ -223,7 +249,14 @@ func TestGoldenVectors(t *testing.T) { SignedInputSHA256: hex.EncodeToString(digest[:])} var sig []byte + // A message whose signed input did not change keeps its stored + // signature under -update, so a new vector does not re-sign the + // rest of the file. + unchanged := i < len(stored.Messages) && stored.Messages[i].Name == v.name && + stored.Messages[i].SignedInputSHA256 == vec.SignedInputSHA256 switch { + case v.signer != "" && *update && unchanged: + sig, _ = hex.DecodeString(stored.Messages[i].Sig) case v.signer == "" || *update: if v.signer != "" { key := box diff --git a/go/protocol/cloudwire/wire.go b/go/protocol/cloudwire/wire.go index 806233be7..af2152f7e 100644 --- a/go/protocol/cloudwire/wire.go +++ b/go/protocol/cloudwire/wire.go @@ -3,9 +3,11 @@ // scope each class belongs to. docs/protocol-cloud.md is the normative text; // this package and its golden vectors hold the bytes. // -// FTW Cloud reads what the box sends. It never controls. Nothing here can -// express a command: the service may send the box five message types, none of -// which asks for anything beyond a faster cadence, a resend or a receipt. +// FTW Cloud reads what the box sends and never chooses an action. Of the six +// message types the service may send the box, five ask for nothing beyond a +// faster cadence, a resend or a receipt. The sixth, tun, carries a phone's own +// Noise session with the box, which the service can neither read nor forge; +// the box judges what is inside as it judges the same session over the relay. // // Like the rest of go/protocol, this package imports nothing from the box, so // the service that imports it links no control, planner or storage code. @@ -41,8 +43,9 @@ const ( HeaderBytes = 6 // MaxPayload is what the u16 length field can describe. MaxPayload = 0xffff - // MaxServiceFrame is the largest message the service may send the box. - // Anything larger closes the uplink. + // MaxServiceFrame is the largest message but tun the service may send + // the box, and MaxTunFrame bounds a tun. Anything larger closes the + // uplink. MaxServiceFrame = 4096 // SignTag starts every signed input: "ftw.cloud.v1" and a zero byte. diff --git a/go/protocol/contract_gen.go b/go/protocol/contract_gen.go index f808a4778..0dc9e9ef9 100644 --- a/go/protocol/contract_gen.go +++ b/go/protocol/contract_gen.go @@ -125,6 +125,7 @@ const ( CapPriceSpot = "price.spot" CapApiPassthrough = "api.passthrough" CapCloudLink = "cloud.link" + CapCloudTun = "cloud.tun" CapCmdExchange = "cmd.exchange" ) @@ -147,6 +148,7 @@ var AllCapabilities = []string{ CapPriceSpot, CapApiPassthrough, CapCloudLink, + CapCloudTun, CapCmdExchange, }