Skip to content

test(e2e): prove offline messages redeliver on session start, durably (RIG-3532) - #1225

Open
rigel-mintaka wants to merge 1 commit into
mainfrom
compass-comms/rig3532-offline-redelivery
Open

test(e2e): prove offline messages redeliver on session start, durably (RIG-3532)#1225
rigel-mintaka wants to merge 1 commit into
mainfrom
compass-comms/rig3532-offline-redelivery

Conversation

@rigel-mintaka

@rigel-mintaka rigel-mintaka commented Sep 14, 2026

Copy link
Copy Markdown
Contributor

This PR is part of a stack containing 2 PRs:

  1. main
  2. "test(e2e): prove offline messages redeliver on session start, durably (RIG-3532)" (this PR)
  3. docs(designs): correct T5 — the owed surface and the offline premise (RIG-3532) #1227

RIG-3473 T5. The offline-redelivery design records had no positive e2e
coverage: the start-sweep appeared only as a hazard other legs acked around
(legfive_test.go). TestCommsOfflineRedeliveryOnSessionStart makes the
redelivery the asserted behavior.

The frozen record's prescribed mechanism does not match the code, so this
test takes a different shape on two axes. Both were verified against the
tree; a follow-up amends the record's T5 section.

The durable surface is UndeliveredMessages, not owed_mentions. The
record names CountOwedMentions/ClearOwedMention, but owed_mentions rows
are written only by RecordOwedMention, whose callsites require the message
to @-mention the agent AND the agent to be outside the channel's sweep set.
InSweepSet is satisfied by cm.channel_id = aa.home_channel_id, so a
home-channel post never records a row — CountOwedMentions would read 0 both
before and after the sweep, and the test would be green while asserting
nothing. UndeliveredMessages is what the sweep itself consumes
(drainStarts -> sweepSession), so it is the surface that actually moves.

"Offline" has to be a despawn, not a never-started agent. The record's
"post to a provisioned-but-not-started agent, then StartSession" is
unreachable: the post wakes the agent and fresh-starts it, because fanOut
finds no live session, calls wake, and freshStart resolves the placement
Provision already recorded — and the waker is wired in the real assembly
the fixture runs (sinks.go SetAgentWaker). StartSession would then be a
second, racing start. Posting after RemoveWorkspace deletes the placement
makes the wake a benign no-placement no-op, so the message stays owed.

The assertion is durable rather than on the live tail, which is the point of
the task: the sweep is enqueued inside Resume and drained asynchronously,
so a tail opened after Resume returns can miss the redelivery. That is the
RIG-3044 flake shape. Event-gated throughout, no sleeps and no retries.

A pre-condition check asserts the message IS owed before the resume, so the
"no longer owed" post-condition cannot pass vacuously.

Verified: PASS in 60.45s. Mutation-tested against production — disabling
sweepSession in drainStarts reddens it at the redelivery assertion
("delivery cursor did not advance past message ... within 15s"); production
restored byte-identical.

Co-authored-by: Matt Wilkinson matt@rigel.build

@linear-code

linear-code Bot commented Sep 14, 2026

Copy link
Copy Markdown

RIG-3532

@trunk-io

trunk-io Bot commented Sep 14, 2026

Copy link
Copy Markdown

Merging to main in this repository is managed by Trunk.

  • To merge this pull request, check the box to the left or comment /trunk merge below.

After your PR is submitted to the merge queue, this comment will be automatically updated with its status. If the PR fails, failure details will also be posted here

@github-actions

github-actions Bot commented Sep 14, 2026

Copy link
Copy Markdown

Compass engineering docs preview: https://compass-comms-rig3532-offlin.compass-eng-docs.pages.dev

Deployed from compass-comms/rig3532-offline-redelivery at 272e1ef.

… (RIG-3532)

RIG-3473 T5. The offline-redelivery design records had no positive e2e
coverage: the start-sweep appeared only as a hazard other legs acked around
(legfive_test.go). `TestCommsOfflineRedeliveryOnSessionStart` makes the
redelivery the asserted behavior.

The frozen record's prescribed mechanism does not match the code, so this
test takes a different shape on two axes. Both were verified against the
tree; a follow-up amends the record's T5 section.

**The durable surface is `UndeliveredMessages`, not `owed_mentions`.** The
record names `CountOwedMentions`/`ClearOwedMention`, but `owed_mentions` rows
are written only by `RecordOwedMention`, whose callsites require the message
to `@`-mention the agent AND the agent to be outside the channel's sweep set.
`InSweepSet` is satisfied by `cm.channel_id = aa.home_channel_id`, so a
home-channel post never records a row — `CountOwedMentions` would read 0 both
before and after the sweep, and the test would be green while asserting
nothing. `UndeliveredMessages` is what the sweep itself consumes
(`drainStarts` -> `sweepSession`), so it is the surface that actually moves.

**"Offline" has to be a despawn, not a never-started agent.** The record's
"post to a provisioned-but-not-started agent, then StartSession" is
unreachable: the post wakes the agent and fresh-starts it, because `fanOut`
finds no live session, calls `wake`, and `freshStart` resolves the placement
`Provision` already recorded — and the waker is wired in the real assembly
the fixture runs (`sinks.go` `SetAgentWaker`). `StartSession` would then be a
second, racing start. Posting after `RemoveWorkspace` deletes the placement
makes the wake a benign no-placement no-op, so the message stays owed.

The assertion is durable rather than on the live tail, which is the point of
the task: the sweep is enqueued inside `Resume` and drained asynchronously,
so a tail opened after `Resume` returns can miss the redelivery. That is the
RIG-3044 flake shape. Event-gated throughout, no sleeps and no retries.

A pre-condition check asserts the message IS owed before the resume, so the
"no longer owed" post-condition cannot pass vacuously.

Verified: PASS in 60.45s. Mutation-tested against production — disabling
`sweepSession` in `drainStarts` reddens it at the redelivery assertion
("delivery cursor did not advance past message ... within 15s"); production
restored byte-identical.

Co-authored-by: Matt Wilkinson <matt@rigel.build>
rigel-mintaka added a commit that referenced this pull request Sep 14, 2026
…(RIG-3532)

The frozen coverage record (RIG-3473) prescribes a T5 mechanism the code
falsifies on two independent axes. Both were verified against the tree
before T5 was implemented (#1225); this amends the record so the next
reader is not misled, and records WHY the shipped test differs rather than
rewriting the task as if it had always said this.

1. `CountOwedMentions`/`ClearOwedMention` are the wrong durable surface: a
   home-channel post never writes an `owed_mentions` row, because
   `RecordOwedMention` requires an @-mention AND an out-of-sweep-set agent,
   and `InSweepSet` is satisfied by the home-channel disjunct. A test
   asserting on that count is green while proving nothing. The surface the
   sweep actually consumes is `UndeliveredMessages`.
2. Posting to a provisioned-but-not-started agent cannot leave a message
   owed: the post wakes the agent and fresh-starts it against the placement
   `Provision` recorded, and the waker is wired in the assembly the e2e
   fixture runs. Offline has to be a despawn boundary.

The durable-not-live-tail requirement the record states is unaffected and
is why the shipped test reads the store.

Co-authored-by: Matt Wilkinson <matt@rigel.build>
@rigel-mintaka
rigel-mintaka force-pushed the compass-comms/rig3532-offline-redelivery branch from 59f770e to 272e1ef Compare September 14, 2026 02:45
@rigel-mintaka
rigel-mintaka marked this pull request as ready for review September 14, 2026 02:46
@rigel-mintaka

Copy link
Copy Markdown
Contributor Author

Verification

  • PASS --- PASS: TestCommsOfflineRedeliveryOnSessionStart (72.64s) / ok github.com/RigelBuild/compass/go/e2e 78.190s, run with -count=1 -tags podman on the rebased tree (also passed pre-rebase at 60.45s).
  • It bites. Mutation-tested against production, not the test: replacing c.sweepSession(ctx, ev.account, ev.sessionID) with _ = ev in drainStarts (go/internal/delivery/settle.go) reddens it at the redelivery assertion — waitDeliveryCursorPast (offline redelivery): delivery cursor did not advance past message f28dc6eb... within 15s. settle.go restored byte-identical (md5 de38ba7cc937db3a9d03d1d290bc59f8).
  • gofmt -l clean · go vet -tags podman ./e2e/ clean · moon run compass-go:lint --force → 0 issues · design-ledger-gate OK.
  • Review loop: 0 high, 0 medium, 1 low (a comment one line over the ceiling), applied.

Why this differs from the frozen record

The record's T5 section prescribes a mechanism the code falsifies on two independent axes. Both were verified before implementation; #1227 amends the record. Matt approved proceeding on the correction.

The record's durable surface is the wrong one. It names CountOwedMentions/ClearOwedMention. But owed_mentions rows are only written by RecordOwedMention, whose callsites (routeMentionsFor, routeAskAnswerFor in delivery/dispatch.go) require the message to @-mention the agent and the agent to be outside the channel's sweep set. InSweepSet is satisfied by cm.channel_id = aa.home_channel_id, so a post to the agent's own home channel never writes a row — the count reads 0 both before and after the sweep. A test asserting on it would be green while proving nothing. This test uses UndeliveredMessages, which is what drainStartssweepSession actually consumes.

The record's scenario is unreachable. "Post to a provisioned-but-not-started agent, then StartSession" cannot leave a message owed, because the post starts the agent: fanOut finds no live session, calls wake, and freshStart resolves the placement Provision already recorded and calls hub.Start — the behaviour TestWakeAgentNeverStartedFreshStarts pins. The waker is wired in the real assembly the fixture runs (server/sinks.go SetAgentWaker). So this test establishes "offline" with a despawn boundary instead: posting after RemoveWorkspace deletes the placement makes the wake a benign no_placement no-op (wakeOnce's errWakeNoPlacement arm). Cost: two containers rather than the one the record estimated, plus one warm turn in lifetime 1 so the resume has a transcript to reconstruct.

The point of the task

The assertion is durable rather than on the live session tail — which is what the record gets right and why it flagged the hazard. The sweep is enqueued inside Resume and drained asynchronously, so a tail opened after Resume returns can miss the redelivery; that is the RIG-3044 flake shape. A pre-condition check asserts the message is owed before the resume, so the "no longer owed" post-condition cannot pass vacuously. Every wait is event-gated with a bounded fail-loud deadline — no sleeps, no retries.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant