Repository navigation
fix(runnerhub): ERRORED cleanup releases only the binding it saw (RIG-4742) - #1810
Open
rigel-mintaka wants to merge 7 commits into
Open
rigel-mintaka wants to merge 7 commits into
rigel-mintaka wants to merge 7 commits into
Conversation
|
Compass engineering docs preview: https://compass-runner-4742-binding.compass-eng-docs.pages.dev Deployed from |
mattwilkinsonn
added this pull request to stack #1824
October 7, 2026 00:25
rigel-mintaka
force-pushed
the
compass-runner/4742-binding-lifetime-token
branch
from
October 7, 2026 03:00
9078fe7 to
6427ae0
Compare
…-4742) The detached ERRORED cleanup unbound whatever row held the session id when it ran, so a resume that re-bound the same id in the same enrollment could be unbound and archived by the old lifetime's cleanup. A binding now carries the durable row version (xmin) and a cache lifetime. The cleanup passes the binding it saw; dropLostSession skips a different one, and releaseSession re-checks the cache around a delete conditioned on that version. DeleteSessionBinding takes a version and reports a removal, so a peer's re-bind is also left alone. Co-authored-by: Matt Wilkinson <matt@rigel.build>
…G-4742) xmin is a 32-bit transaction id and repeats after wraparound, so an old cached version could match a later re-bind. binding_version is a fresh UUID per upsert. The conditional release is a separate DeleteSessionBindingVersion, and DeleteSessionBinding is back to its unconditional form for Stop. A cached binding whose durable write failed has an empty version, which matches no row, so its cleanup cannot delete a peer's row. Co-authored-by: Matt Wilkinson <matt@rigel.build>
A cached binding whose durable write failed has an empty version. A legacy row also has the column default '', so a versioned delete with '' could remove it. The limited release now skips the store for an empty version. The removed check no longer reads only, which nilaway could not prove non-nil. Co-authored-by: Matt Wilkinson <matt@rigel.build>
…ll versions (RIG-4742) A cached binding with no durable version now checks for a competing row before releasing. If one exists, the session was re-bound elsewhere: the stale cache entry is dropped and no loss is reported. The migration also gives pre-existing rows a real version, so no row keeps the '' default. Co-authored-by: Matt Wilkinson <matt@rigel.build>
…sion (RIG-4742) A cleanup that found its row re-bound evicted only the session's cache entry. The reverse account entry still named the session, so delivery for the old account could reach the new owner's session. Evict both. Co-authored-by: Matt Wilkinson <matt@rigel.build>
A stale-version delete records nothing; the current-version delete ends its open interval. Co-authored-by: Matt Wilkinson <matt@rigel.build>
…h enroll (RIG-4742) A binding learned through SessionForAccount had no version or lifetime, so its ERRORED cleanup took the version-less path, found its own row, and never released or reported it. The reverse read now returns the version. A Stop's release now takes bindingWriteMu, as promotion does since the rebase onto the enroll serialization, so its delete cannot land between an enroll's map-clear and its reap and hide the session from the sweep. Co-authored-by: Matt Wilkinson <matt@rigel.build>
rigel-mintaka
force-pushed
the
compass-runner/4742-binding-lifetime-token
branch
from
October 7, 2026 03:56
6427ae0 to
260e55e
Compare
rigel-mintaka
marked this pull request as ready for review
October 7, 2026 05:13
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR is part of a stack containing 2 PRs:
mainSummary
The ERRORED lost-session cleanup runs detached. Before this change, it unbound whatever row held the session id when it ran. A resume that re-bound the same session id in the same enrollment could therefore be unbound and archived by the old lifetime's cleanup. Implements RIG-4742 option A (binding lifetime token).
sessionBindingnow carries:version: a newsession_bindings.binding_versioncolumn, set to a fresh UUID on everyRecordSessionBindingupsert and returned by Record and Resolve. A UUID, notxmin, because transaction ids wrap around.lifetime: a per-insert counter for the cache, which stores without a durable row rely on.dropLostSessionskips if the current binding differs.releaseSessionre-checks the cache before and after the durable delete.DeleteSessionBindingVersion(ctx, sessionID, version)deletes only that write and reports whether it removed a row. If a peer Server re-bound the row, it removes nothing and the cleanup stops. A cached binding whose durable write failed has version"". It never deletes a row; if a row exists, the session was re-bound elsewhere, so the stale cache entry is dropped and no loss is reported.DeleteSessionBindingis unchanged and still serves Stop and the deliver-refusal loss.SessionForAccountnow returns the row's version, so a binding cached from a delivery's reverse read carries it. Its ERRORED cleanup then deletes that row and reports the loss.releaseSessiontakesbindingWriteMulikepromoteSession, so a Stop's delete cannot land between an enroll's map-clear and its reap.0003_session_binding_version.sql: adds the column with a constant default (no table rewrite), then backfills existing rows with a UUID.0003after main took0002_forge_scopes.sql. Any open PR that lands first with a0003forces another renumber.Internal token, no API change. This was decided by the compass lane on RIG-4742; Matt can overrule it there.
Verification
TestErroredCleanupKeepsSameEnrollmentRebind: pauses the old cleanup, re-promotes without re-enroll, releases. The cache and durable bindings survive and no loss is reported.TestErroredCleanupKeepsRebindDuringDurableDelete: re-bind lands while the durable delete is paused. Fails if the delete is unconditional.TestErroredCleanupKeepsPeerRebind: a peer re-binds the row; the cache is stale. Fails if a not-removed delete is ignored.TestErroredCleanupWithoutDurableWriteKeepsPeerRow: the hub's durable write failed and a peer wrote the row. The row is kept, no loss is reported, and the peer's binding resolves. Fails if a version-less release ignores the competing row.TestErroredCleanupWithoutDurableWriteKeepsLegacyRow: a version-less release meets a row with the default''version. Fails if it calls the versioned delete.TestErroredCleanupDropsStaleAccountForPeerRebind: a peer re-binds the session id to another account mid-cleanup. Fails if the old account still resolves to it.TestErroredCleanupReleasesReverseReadThroughBinding: a binding learned by reverse read goes ERRORED. Fails if the read-through caches no version.TestStopUnbindWaitsForEnrollReap: a Stop races an in-flight reap. Fails if the unbind does not wait for the enroll.TestDeleteSessionBindingByVersionSkipsRebind(pgtest, real Postgres): a stale version deletes nothing and records no usage event; the current version deletes and ends its interval. Fails if the SQL condition or the end event is dropped.go test -racepasses for./server/,./internal/runner/...,./internal/runnerhub/and./internal/store/../internal/store/,./internal/runnerhub/and./server/pass against local Postgres 17.moon cicaught one nil-flow, fixed). Withpgtest, none of its findings are in touched lines.sqlc generateoutput is committed.Spec-impact: none (server-internal binding release; no API or documented state change).
Ledger-impact: none