Skip to content

feat(cloud): FTW Cloud C2: signed uplink, grant routes and wire format - #1595

Closed
frahlg wants to merge 8 commits into
masterfrom
claude/cloud-c2-a453be
Closed

frahlg wants to merge 8 commits into
masterfrom
claude/cloud-c2-a453be

Conversation

@frahlg

@frahlg frahlg commented Oct 12, 2026 •

Copy link
Copy Markdown
Member

FTW Cloud step C2 on the box: the wire format in go/protocol, the signed uplink, and the grant routes. The box streams a signed, readable copy of its state to ftwcloudd, which keeps it in memory. FTW Cloud reads and never controls. Decision and normative wire format: ADR 0008 and docs/protocol-cloud.md (draft). Builds on #1588.

Contract-pair: srcfl/ftw-webapp@claude/cloud-c2-registry

Paired pull requests

  • srcfl/ftw-webapp: srcfl/ftw-webapp#81 (the registry names, source of the contract copy)
  • srcfl/ftw-app: srcfl/ftw-app#15 (the registry copy and Swift names ContractTests needs)
  • srcfl/ftw-cloud: not opened yet: creating the public repository srcfl/ftw-cloud needs the owner's go-ahead in this session; the code is ready on a local branch (ftwcloudd, first version)

What changes

go/protocol/cloudwire (Apache-2.0, imports nothing from the box):

  • The unpadded lane 1 frame, the {t, b, sig} envelope and its ES256 rule over ftw.cloud.v1\0 ‖ CBOR([home, t, b]), home ids, opaque source and loadpoint ids, the message bodies and the scope of each class.
  • Fields is the allowlist: one table per class and per nested shape. TestEveryFieldHasARow fails when a body grows a field without a row, and TestFieldsMatchTheProtocolDocument compares the tables with docs/protocol-cloud.md once docs: propose FTW Cloud (ADR 0008) and its protocol #1589 lands (it skips until then; it passes against docs: propose FTW Cloud (ADR 0008) and its protocol #1589's text today).
  • Golden vectors in testdata/cloud-vectors.json, which srcfl/ftw-cloud and the phone replay.
  • View: the phone's freshness rules as a reference, with 13 of the protocol's 16 cases as tests (10, 11 and 15 are the same rules seen from lp, the budget and the notification extension).

go/internal/cloudlink (the uplink):

  • Off until an owner grants it. A grant whose key fingerprint does not match cloud.key reads as off, with a notice.
  • One WebSocket, TLS 1.3, no proxy from the environment. The box sends only hello until the service has signed a hello_ok bound to this TLS session's exporter value and the box's nonce.
  • Every message signed with cloud.key, with seq, boot, uptimeMs, and wallMs while NTP has set the clock (adjtimex on Linux).
  • Change-driven live, lp and prices at most every 5 s (1 s while watched, lp at least 2 s); a source-table change goes at once with a new srcRev; hb after H and never while a changed live or lp waits; full state every 15 minutes and after resync.
  • One writer per socket: a slow socket gets the newest values, nothing queues, a write blocked for 10 s closes. Past 20 MB in 24 hours of uptime only roster and bye go.
  • Driver and loadpoint names become opaque ids; capability fields have no place in the shapes. The downlink is a closed set of five types, under a fuzz test.
  • Revoke sends bye{purge} and keeps the key until the service's signed receipt arrives.
  • Import ban: .github/scripts/check-cloudlink-imports.sh (in CI and make verify) fails if the package's build graph, tests included, reaches anything of the box but go/protocol.

Core wiring:

  • GET /api/cloud, GET /api/cloud/preview (owner), POST /api/cloud/grant (owner over a session; on the LAN only from the box itself or with the house password), DELETE /api/cloud/grant (any door).
  • Enrolment lists whole keys for the signed roster and says when pairing, roles or revokes change.
  • Owners hear cloud.granted and cloud.revoked. These two rules are on by default, unlike the rest; two existing tests now allow that.
  • Backups skip cloud.key.
  • Release builds name no service (cloudlink.ServiceKeys is empty), so a box refuses a grant and never dials. A local build sets the endpoint, the service key and a dev root with -ldflags -X.

End-to-end, on this laptop

Core built from this branch (a87386a8, and 06989083 for the last revoke run; the master merge in between touches no cloud file) with the endpoint, a local service key and a dev root set by -ldflags -X, against a local ftwcloudd from srcfl/ftw-cloud built with -tags ftwclouddev. The house is simulated: sim-ferroamp as site meter with battery and PV (1 Hz), sim-sungrow with battery and PV, and sim-ocpp as an Easee charger with a car plugged in, drawing about 11 kW, adopted by a loadpoint. Prices from elprisetjustnu.se, SE3. App link and fleet ping off.

Off until granted. Before the grant, /api/cloud said "state":"off" and nothing dialled. /api/cloud/preview showed an unsigned sample (live 732 B, lp 972 B, prices 1 312 B) with opaque ids and no driver names. The grant came from the box itself (curl on loopback): cloud.key was made, the home id is xgsch4rq….

The box refuses a bad certificate. The first local certificate was signed with SHA-1. The box refused it (insecure algorithm ECDSA-SHA1) and kept backing off until the certificate was replaced.

What the cloud holds is what the preview shows. With the uplink up, the last frame of each class in /api/cloud/preview, less its 6-byte header, was compared with the envelope the service holds:

class   seq   frame   held by the service     identical
hello     1   429 B   (service only)          -
roster    2   324 B   registry, 318 B          yes (SQLite row = frame less header)
live     35   260 B   seq 35                  yes
lp       15   811 B   seq 15                  yes
prices    5  1064 B   seq 5                   yes

Every envelope the service holds verifies with the box's key, and the home id is the hash of the key in hello.

A replayed snapshot goes stale. The held live (with its table), lp, prices and hb were fed to cloudwire.View, the reference for the phone's rules:

read now:                sources live, fields live, loadpoint live
same bytes, an hour on:  sources down, fields 2–6 and 10 down, field 1 (mode) stale, loadpoint down

Restarts. A restart of ftwcloudd (empty memory, same registry): the box reconnected after its backoff and the service was full again within a second of it. A restart of Core kept the grant and the home id.

A frozen service leaves the box alone. kill -STOP on ftwcloudd kept its sockets open and answered nothing. /api/status answered in 1–3 ms every 5 s throughout, and the control log has no slow-tick line. The box dropped the uplink after 36 s (i/o timeout, inside the 45 s read deadline) and was connected again 1.6 s after kill -CONT.

A revoked grant closes the stream and erases the copy. DELETE /api/cloud/grant from the box: the box sent bye{purge}, the service deleted the home and returned a signed purged, and /api/cloud read "state":"off" with purgedAtMs 0.07 s after the request. cloud.key is gone, /dev/held answers 404 for the home, the registry has no row and one tombstone, and the box makes no further connection. A second grant made a new key and a new home id, and its revoke finished in 1 ms.

Bytes per day. 30 minutes on the final sender (a87386a8), uplink connected throughout:

Measured Per day
Frames sent (the box's own count, WebSocket payload) 95 404 B in 1 804 s 4.57 MB
Messages 331 15 853
TLS records out, from nettop for this socket 89 546 B in 1 494 s 5.18 MB
TLS records in (pings, service messages) 2 400 B 0.14 MB

Sizes: live 263 B, lp 815 B, prices 1 064 B, roster 324 B, hello 429 B. The design estimated about 5.0 MB a day on the wire for the home box (19 372 messages), so this is about 1.2× once TCP and IP headers are added, inside the 2× the milestone allows. The simulated house is the harder case: a 1 Hz meter and a car charging at 11 kW the whole time.

Before the lp deadband the same house sent 12.7 MB a day: lp (814 B) went at every period because measured power moves every reading. With the deadband lp goes about every 35 s while the car charges, as session energy passes each 100 Wh. Composer cost on a Pi 4 is not measured here.

Where this differs from docs/protocol-cloud.md (#1589)

  1. fieldSources travels with sources (every full live and every table change), not in every live. It changes only with the table, and in every message it would add about 1.5 MB a day.
  2. A roster goes at every connect, not only when the numbers differ. An equal number renews ttlS; without it a box back after 20 hours offline would have 4 hours left.
  3. lp compares measured values with a deadband: power and actual watts past max(100 W, 5 %), session energy past 100 Wh, charge level past 1 %. The document ignores only the ages. Without the deadband the simulated house sent 12.7 MB a day.
  4. Nil lists are [] and unknown numbers null on the wire.
  5. session is a count of the plug-ins this boot has seen; Core has no session id yet (K6). socAgeS is set only when the level came from the car.
  6. manual.reason is the hold's limit reason; manual.commandedW is null at zero.
  7. hello.caps lists the classes this box sends.
  8. prices.t0Ms is the start of today on the box's clock, so the price hash moves only when new prices land.
  9. A grant names a policy, and purposes is serve only until the statistics route exists.
  10. Restore in place loses the key. Restore moves the whole data directory aside, so cloud.key does not come back and the grant reads as off (key_missing). The document says a box restored in place keeps its key; that needs a change to the restore flow.
  11. The service signs hello_ok and purged over the box's home id.
  12. POST /api/cloud/grant asks no step-up. The box cannot verify one, and the document has the consent screen claim no more than an owner's session.

Not in this pull request

  • C3: the phone's half, including K11 (the box key, home id, boot and the opaque id table in the Noise session info). The cloud.link capability is in the registry but not advertised until the service exists.
  • C4 and later: event, plan, health, tiles and stats, the statistics consent route, APNs.
  • Composer cost on a Pi 4: not benched here.

Checks

  • make verify: green (Go suites, go/protocol, vet, build, protocol and cloudlink boundaries).
  • After merging master: make verify green again; targeted go test for api, cloudlink, appenroll, notifications, backup, appproto and the cloud tests in cmd/ftw.
  • go test -race -count=3 ./internal/cloudlink: green.
  • Fuzz: FuzzDecodeFrameAndEnvelope 20 s (4.9 M runs), FuzzDownlink 60 s (3.1 M runs).
  • TestFieldsMatchTheProtocolDocument against docs: propose FTW Cloud (ADR 0008) and its protocol #1589's docs/protocol-cloud.md: passes.

A changeset (patch) is included. Release files, #1584 and the version are untouched.

🤖 Generated with Claude Code

frahlg and others added 5 commits October 12, 2026 20:19
Copied byte for byte from srcfl/ftw-webapp, which holds the source:
scope ftw.events.read, the cloud_scopes and cloud_purposes lists,
capability cloud.link, carrier state cloud, and push kinds
cloud.granted and cloud.revoked. The generator now writes CloudScopes
and CloudPurposes into go/protocol, and the registry header names the
file it actually generates.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
go/protocol/cloudwire holds what docs/protocol-cloud.md specifies:
the unpadded lane 1 frame, the {t, b, sig} envelope and its ES256
signing rule, home and opaque ids, the message bodies, the scope of
each class and the field allowlist. A test fails when a body grows a
field the allowlist lacks, and compares the allowlist with the
protocol document's tables when the document is present.

Golden vectors in testdata/cloud-vectors.json freeze the bytes for
srcfl/ftw-cloud and the phone. View is the phone's freshness rules as
a reference, with the document's cases as tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
go/internal/cloudlink is the box's uplink to FTW Cloud. It is off until
an owner grants it, and a grant whose key fingerprint does not match
cloud.key reads as off.

- One WebSocket over TLS 1.3 with no proxy. The box sends nothing but
  hello until the service has signed a hello_ok bound to this TLS
  session's exporter value and the box's nonce.
- Every message is signed with cloud.key and carries seq, boot,
  uptimeMs and wallMs while NTP has set the clock.
- live, lp and prices go on change, at most every 5 s (1 s while
  watched, lp at least 2 s); a table change goes at once with a new
  srcRev; hb goes after H and never while a changed live or lp waits.
- One writer per socket: a slow socket gets the newest values, nothing
  queues, and a write blocked for 10 s closes it. Past 20 MB in 24 h
  of uptime only roster and bye go.
- Driver and loadpoint names become opaque ids; capability fields are
  not in the shapes. The downlink is a closed set of five types, under
  a fuzz test.
- A revoke sends bye with purge and keeps the key until the service's
  signed receipt arrives.

CI runs go list -deps to keep control, API and loadpoint code out of
the package's build graph.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- GET /api/cloud and /api/cloud/preview for an owner: the grant, the
  bytes sent in 24 hours, and the exact frames that left the house.
- POST /api/cloud/grant from an owner's session, or on the LAN only
  from the box itself or with the house password. DELETE from any door.
- Enrolment lists whole keys for the signed roster and says when who is
  paired, or as what, has changed.
- The adapters read the live snapshot the app session sends, the
  loadpoint view GET /api/loadpoints composes and the price store.
- Owners hear cloud.granted and cloud.revoked; these two rules are on
  by default, unlike the rest.
- Backups skip cloud.key, so a restored box has FTW Cloud off.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A charging car moves its measured power every reading, so lp went at
every period: about 12 MB a day on a simulated site, against an
estimate of 5 MB. lp now counts power and actual watts as changed only
past max(100 W, 5 %), session energy past 100 Wh and charge level past
1 %. A change of state still goes at once, the full lp every 15 minutes
carries exact values, and live field 10 carries the charging power at
every period.

Also say which way a dev root CA file failed to load.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
frahlg and others added 3 commits October 12, 2026 21:10
The sender looked at live, loadpoints and the roster number on every
250 ms tick. On a Pi the loadpoint view composes control feedback, and
the roster number is a SQLite read. Now live is read once a second (so
a table change still goes within a second), loadpoints once a period,
and the roster only when it changed, at connect and every six hours.
An hb still reads fresh ages on the tick it goes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The service closes the uplink right after it sends purged. The receipt
went to the sender through a channel, and the reader could see the
close first and end the connection with the receipt unread; the box
then had to ask again. The reader now checks the receipt itself and
ends the connection only after the key and the grant are gone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
frahlg added a commit that referenced this pull request Oct 13, 2026
* docs: propose FTW Cloud (ADR 0008) and its protocol

ADR 0008, status proposed, records Fredrik's decisions of 12 October
2026: a readable, read-only copy of what a home's owner turns on, served
to the iPhone app, its widgets and APNs from ftwcloudd in srcfl/ftw-cloud
on AWS eu-north-1. The box signs every message; the service never
controls and never chooses actions; the relay, escrow and lane 0 do not
change. It states the statistics rules now, because consent comes at
first use, orders the work core first (uplink, cache, first screen),
and records the turn away from #1323 and ADR 0006's single remote path.

docs/protocol-cloud.md is the normative wire format: unpadded lane 1
frames, one signing rule, a handshake bound to the TLS session, a field
table per class and scope, cadence, the closed set the service may send
the box, roster, extension token, push, freshness with eight vectors
described in text, and the new registry names.

VISION.md and AGENTS.md gain the matching paragraph and invariants;
architecture.md and ADR 0006 point to the ADR.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(cloud): close freshness gaps found in review

An independent review found ways stale data could still show as live.

- hb now names the newest live and lp it covers (liveSeq, lpSeq) and
  carries each source's reading age, so the phone grades a reading by
  the box's own ladder instead of keeping it live for another S.
- The box sends no hb while a changed live or lp waits, nor past the
  byte budget, so an hb never vouches for values the box has not sent.
- The phone keys its table by boot, tracks seq per class, takes the
  earlier of wallMs and arrival under rule N, adds a drift margin and a
  24 h lifetime under rule U, and shows old alerts without buttons.
- hello now carries the box's TLS exporter value too, so a proxy that
  ends TLS cannot hold a socket for the home; the label starts with
  EXPERIMENTAL as RFC 5705 asks.
- Restores, roster expiry, purge receipts, tiles for phones, Live
  Activity tokens and what Apple sees are now stated; timers agree
  between the ADR and the protocol. Eight new vectors cover the cases.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(cloud): take in what C2 and C3 built

The protocol was written before the code. #1595 (C2) and
#1603 (C3) each list where the code had to differ. This moves
those differences into the text so it stays normative.

From C2: fieldSources travels only with sources; a roster at every
connect; a deadband on lp's measured values; [] and null on the wire;
session as a plug-in count; manual.reason as the limit reason;
hello.caps as the classes sent; prices from the start of today; the
service signs over the box's home id; no step-up on the grant; a
restore never brings cloud.key back.

From C3: /v1/app?home=<id>; message 2 as an unsigned snapshot
envelope with spill-over frames; a phone's watch holds until it
changes, and the service renews the box's every 50 s; close codes
4400, 4403, 4408, 4409, 4429 and 1013, with the phone's retries;
session limits; hello_ok.cloud (K11); the statistics consent routes,
which push nothing; case 12 shows no sources rather than stale ones;
role names are not generated in go/protocol.

ADR 0008: turning FTW Cloud on or off notifies owners, not every
change; the protocol has sixteen cases, not eight.

TestFieldsMatchTheProtocolDocument passes against this text on the
C3 branch (c6301a0).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
@frahlg

frahlg commented Oct 13, 2026

Copy link
Copy Markdown
Member Author

Included in #1607 (FTW Cloud C2, C3 and C4 joined), squash-merged to master.

@frahlg frahlg closed this Oct 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant