Repository navigation
fix: fence timer completion against replacement and lease takeover - #126
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A handler completing an earlier timer could delete a same-ID replacement or a timer reclaimed by another worker after lease expiry. Give every successful claim a unique ownership token and make ack, reject, and corrupt-payload cleanup conditional on that captured token. Scheduling invalidates earlier ownership; explicit cancellation remains unconditional.
Ownership metadata is removed on valid completion, cancellation, bulk cancellation, and orphan cleanup. Existing payloads and deadlines need no rewrite: persisted timers acquire metadata when claimed. Application migration changes are outside this PR.
Redis client settings and ownership storage
Redis client encoding, encoding error handling, and
decode_responsessettings remain under caller control. Timer ID inspection uses the client's encoder, and claims preserve binary payload bytes. Regression coverage includes Latin-1, UTF-8 with surrogateescape, ASCII with replacement errors, and both response-decoding modes. Polling fetches raw timer IDs independently of response decoding and checks decode/encode byte identity. Unrepresentable IDs are conditionally cleaned by their original bytes, preventing lossy decoding from redirecting claims or blocking later valid timers.Default ownership keys use a SHA-256 digest of the encoded timeline and payload key identities, each prefixed with its eight-byte length. This separates independently configured stores without reserving a binary key namespace. The optional
claims_keyparameter selects an explicit{claims_key}:{topic}namespace.Mutating operations verify that timeline, payload, and ownership key bytes are distinct before issuing Redis commands. Independently configured brokers must use non-overlapping namespaces; workers sharing timers must use the same ownership configuration.
Upgrade compatibility
Stop all old writers, including consumers and in-flight handlers, publishing-only processes, and cancellation clients, before fenced workers use the same namespace. Upgrade every writer before resuming it, or isolate old and new deployments in separate namespaces. Old producers do not invalidate ownership, and old consumers delete unconditionally. Do not change ownership configuration while workers are active.
Delivery remains at-least-once. Fencing protects timer state, not external side effects; handlers must remain idempotent.
Validation
just test-ciagainst Redis 8). Four existing mock coroutine warnings remain.redis==5.0.1andfaststream==0.7.6, in an isolated environment.just lint-ci: formatting, lint, and type checks passed.just docs-buildandjust adr-checkpassed.just test-dr: both Sentinel primary-kill and primary-pause failover drills passed.git diff --checkpassed.