Skip to content

refactor(protocol): move the wire format into its own Apache-2.0 module - #1588

Merged
frahlg merged 2 commits into
masterfrom
claude/cloud-c1a-protocol-3444a6
Oct 12, 2026
Merged

frahlg merged 2 commits into
masterfrom
claude/cloud-c1a-protocol-3444a6

Conversation

@frahlg

@frahlg frahlg commented Oct 12, 2026

Copy link
Copy Markdown
Member

Why

FTW Cloud (C1 onward, own repo srcfl/ftw-cloud, AGPL) must speak the box protocol. Today the wire types share package appproto with the handler, and the handler imports control, loadpoint, mpc and config. A service that imports the wire types therefore links the box: control code, the planner and sqlite. This PR moves the wire format into its own module so that "the cloud never controls" holds in the build graph, not only in a test. It is step C1a; ADR 0008 is written in a parallel session.

What moved

New nested module github.com/srcfl/ftw/go/protocol in go/protocol, Apache-2.0, requiring only fxamacker/cbor and x/crypto:

New place From Change
go/protocol/appwire/ (all 10 files, testdata included) go/internal/appwire/ path in comments only
go/protocol/envelope.go, envelope_test.go go/internal/appproto/ newEnvelope and bulkBucketFor exported as NewEnvelope and BulkBucketFor
go/protocol/messages.go go/internal/appproto/ wireModes stays in the box (appproto/modes.go)
go/protocol/contract_gen.go go/internal/appproto/ package clause only
go/protocol/tiles.go, tiles_test.go go/internal/appproto/history.go, history_test.go tile geometry, HistorySample, and the packing half of buildTile as PackTile
go/protocol/push.go, push_gen.go, push_test.go go/internal/notifications/catalogue*.go package clause only
go/protocol/contract_test.go TestFrozenFieldIdsAreFrozen from appproto/contract_test.go unchanged

The box keeps every old name:

  • appproto/protocol.go re-exports the moved types, constants and values as aliases, plus one-line wrappers for the four functions. notifications/catalogue.go does the same for the push catalogue.
  • No call site changes. The five files that import appwire change only their import path.
  • buildTile reads the store and calls protocol.PackTile. The planner (planHistoryQuery) stays in the box. plannedTile.points was read only by buildTile, so it is gone.
  • Error text from the moved functions now starts with protocol: instead of appproto: or notifications:. No test or client reads it.

The generator stays in the box at go/internal/appproto/gencontract, and its output moves:

  • Its go:generate lines write go/protocol/contract_gen.go and go/protocol/push_gen.go. It also writes apiauth/contract_gen.go, which stays in the box. go generate ./internal/... from go/ regenerates all three, as before.
  • Moving it into the protocol module would add yaml.v3 to that module and make the generator its public API, only so that apiauth could reach it.
  • The drift tests for the generated files stay beside the generator. Regenerating gives the committed bytes.

The box requires the module through replace github.com/srcfl/ftw/go/protocol => ./protocol in go/go.mod. There are no tags yet; they come after C3.

Licence

go/protocol/LICENSE is the Apache-2.0 text, byte-identical to srcfl/ftw-webapp/LICENSE (sha256 cfc7749b…). NOTICE and LICENSING.md each gain a short paragraph that names the exception. The rest of the repository stays AGPL-3.0-only.

Fredrik approved on 12 Oct 2026 that the parts he wrote are relicensed to Apache-2.0 ("de är mina").

Which files change licence, and why:

Code State at the Apache snapshot d281df8c (2026-09-12) Since then Author by git blame
appwire/* (10 files) present unchanged Fredrik
envelope.go, envelope_test.go present unchanged Fredrik
tile helpers and buildTile packing (now tiles.go), the two moved geometry tests present unchanged Fredrik
push catalogue and its tests (now push*.go) present unchanged Fredrik
TestFrozenFieldIdsAreFrozen present unchanged Fredrik
messages.go present +4 lines, OpPlannerPrefsSet (1853d674, 2026-09-29) Fredrik; those 4 lines Cursor Agent
contract_gen.go (generated) present +2 lines (1853d674, Cursor Agent), −2 lines (a71f777c, Fredrik) generated from contract/registry.yaml
new in this PR: PackTile signature, NewEnvelope, BulkBucketFor exports, TestPackTileFoldsMeansAndMarksHoles, test helpers — — Claude Opus 5.5, at Fredrik's direction
  • Everything except the last three rows already stood under Apache-2.0 in the snapshot, which NOTICE keeps available. For those files the change only extends Apache-2.0 to the current text.
  • The AGPL-era changes inside the module are 6 added lines by Cursor Agent and 2 deleted lines by Fredrik.
  • The generated files restate names from contract/registry.yaml and contract/push-catalogue.yaml. Those are edited in srcfl/ftw-webapp, which is Apache-2.0.
  • No line by Erik Arenhill, or by any author other than Fredrik and Cursor Agent, moves into the module.

A lawyer should confirm that the copyright in these files belongs to Fredrik or to Sourceful Labs AB, and so can be relicensed. They should also confirm how to treat the 6 lines committed by Cursor Agent (with a Signed-off-by: Cursor Agent), and the code written by Claude in this PR. LICENSING.md already says that a DCO sign-off is not a copyright assignment.

Proof the module builds without the box

A test module outside the repository, with an unrelated module path (example.com/ftwcloudprobe, as srcfl/ftw-cloud will have), imports protocol and protocol/appwire. It starts a Noise IK responder, encodes a snap envelope, frames it, packs a tile and renders a push sentence. The "before" probe makes the same calls against 851eaee0. It needs a module path under github.com/srcfl/ftw/go/ to be allowed to import internal/.

Built with the release flags (CGO_ENABLED=0 -trimpath -tags netgo,osusergo -ldflags '-s -w'), Go 1.27.1, linux:

Probe arch packages (go list -deps) srcfl packages sqlite binary
before: internal/appproto + internal/appwire + internal/notifications arm64 340 17 yes 13,959,328 B
after: go/protocol + go/protocol/appwire arm64 124 2 no 2,818,208 B
before amd64 342 17 yes 14,778,528 B
after amd64 125 2 no 2,945,184 B

The "after" binary's module list (go version -m) contains only fxamacker/cbor/v2 v2.9.4, x448/float16 v0.8.4, golang.org/x/crypto v0.57.0 and srcfl/ftw/go/protocol.

Probe source (after)
// Probe: what a service that speaks the FTW protocol links.
package main

import (
	"fmt"
	"os"

	"github.com/srcfl/ftw/go/protocol"
	"github.com/srcfl/ftw/go/protocol/appwire"
)

func main() {
	kp, err := appwire.GenerateKeyPair()
	check(err)
	hs, err := appwire.NewResponder(appwire.HandshakeOptions{StaticKey: kp, Prologue: []byte("ftw.cloud.v1:probe")})
	check(err)
	env, err := protocol.NewEnvelope(protocol.MsgSnap, nil, protocol.Snap{UptimeMs: 1, Fields: map[string]int64{"2": 1200}})
	check(err)
	raw, err := protocol.EncodeEnvelope(env)
	check(err)
	back, err := protocol.DecodeEnvelope(raw)
	check(err)
	bucket, _ := appwire.BulkBucketFor(len(back.B))
	frame, err := appwire.EncodeFrame(appwire.Frame{Lane: protocol.LaneBulk, Bucket: bucket, Envelope: appwire.Envelope{T: back.T, B: back.B}})
	check(err)
	tile := protocol.PackTile(protocol.ResN5m, 1, 0, []string{"grid_w"}, nil, 0)
	title, _, err := protocol.RenderPush(protocol.PushBoxUnreachable, nil)
	check(err)
	fmt.Println(len(frame), tile.Etag, title, hs.Complete())
}

func check(err error) {
	if err != nil {
		fmt.Fprintln(os.Stderr, err)
		os.Exit(1)
	}
}

go.mod: module example.com/ftwcloudprobe, require github.com/srcfl/ftw/go/protocol v0.0.0-00010101000000-000000000000, replace to the checkout.

The import ban, and its test

.github/scripts/check-protocol-boundary.sh fails when go list -deps -test ./... in go/protocol reaches any github.com/srcfl/ftw/go/... package outside the module, or when go list -m all lists the box module. test.yml (core job) runs it after the module's own go vet and go test. make verify runs it as make protocol-boundary.

The check is needed: Go's internal rule does not block this import, because github.com/srcfl/ftw/go/protocol sits under the box's path.

  • Negative control: a copy of the module plus a test that imports internal/control and a require/replace of the box. It compiles and vets, and the check exits 1. It lists 11 box packages under It reaches: (internal/control, events, loadpoint, mpc, state and others) and the box module under It requires:.
  • On this branch the check passes: go/protocol: 121 packages, none from the box.

The box builds as before

Release flags as above, ./cmd/ftw:

arch packages srcfl packages binary
851eaee0 arm64 458 53 29,884,576 B
this branch arm64 459 54 29,884,576 B (same)
851eaee0 amd64 460 53 32,075,936 B
this branch amd64 461 54 32,080,032 B (+4,096 B)

The one extra package is the new protocol root package.

Other build paths:

  • Dockerfile: go mod download reads go/protocol/go.mod through the replace, so the dependency layer copies it as well. Without that line the layer fails with reading protocol/go.mod: no such file or directory; I reproduced that in a temporary directory, and it passes with the line.
  • go/tools/history-migrate: needs no change. go mod tidy -diff is clean and go vet ./... passes, because graph pruning never loads the protocol requirement there.
  • Dependabot: watches /go and /go/protocol.

Tests

  • cd go/protocol && go vet ./... && go test ./...: green.
  • Targeted box suites (appproto, notifications, apiauth, appuplink, appenroll): green.
  • make verify on this branch: green, exit 0. 69 Go packages report ok, including both protocol packages, and the boundary check passes. Master's known red test, TestDispatchReachesEveryInstalledPublisher, passed in this run. It fails by chance on a race in the test itself (fixed in test(notifications): wait for the dispatch event, not the fake transport #1587), and this PR does not touch that path.

Generated files: go generate ./internal/appproto/ ./internal/notifications/ ./internal/apiauth/ writes the committed bytes. Against master, the only difference is the package clause.

Not in this PR

  • appuplink/codec.go stays in the box. It is the 49-line FrameCodec that joins protocol.Frame to appwire, and FTW Cloud will need one too. Moving it means either appwire imports protocol, or a third package. That is a design choice for C2.
  • The telemetry builders (sourceState, wireSources, fieldDict, sourcesEqual) and planHistoryQuery stay in the box. The task list did not include them; core report C8 covers exporting them.
  • Paths I do not own still name the old location: the link in contract/README.md, a comment in contract/registry.yaml (byte-paired with ftw-webapp), and docs/architecture.md:368. The text has gone to the session that owns those files.

No paired PR is needed: no name in registry.yaml changes. Label no-changeset, because users see no change.

🤖 Generated with Claude Code

FTW Cloud must speak the box protocol without linking the box. Today the
wire types share package appproto with the handler, which imports control,
loadpoint, mpc and config, so a service that imports them links 340
packages and sqlite.

go/protocol is a nested module, github.com/srcfl/ftw/go/protocol, under
Apache-2.0. It holds appwire whole, the envelope, the message shapes
(without wireModes), the generated registry names, the history tile
geometry with the packing from buildTile, and the push catalogue. It
requires only cbor and x/crypto.

The box requires it through a replace directive; there are no tags yet.
Aliases and one-line wrappers keep every old name in appproto and
notifications, so no call site changes. Five files import appwire from its
new path. The generator stays in the box and writes into go/protocol.

CI and make verify run the module's own vet and tests, and a go list -deps
check fails if it reaches any box package. Go's internal rule does not
stop that import, because the module's path sits under the box's.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@frahlg frahlg added the no-changeset PR intentionally exempt from the changeset requirement (dev tooling / non-shipping) label Oct 12, 2026
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 12, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-12T16:27:07.546961Z 4ce0371 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@frahlg
frahlg merged commit 3a73c9a into master Oct 12, 2026
18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

no-changeset PR intentionally exempt from the changeset requirement (dev tooling / non-shipping)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant