Require finite single-flight lifetimes and expose observability

[lan17]

Priority

P1 — High — resolve before broad production adoption or large-scale use.

Problem

The process-scoped single-flight registry retains an entry until the complete leader cache chain settles. A Redis read/write, fallback, custom serializer, or other injected async hook that never settles can therefore retain the key, promise, closures, and followers indefinitely.

Many concurrently pending unique keys can grow registry state during their lifetime. A same-key follower inherits the leader's lifetime and cannot make the shared work settle.

Current evidence:

• Per-instance process registry
• Cleanup only after complete promise settlement
• Fallback, Redis publication, and local population share the leader lifetime

Decision

Use finite, resource-owned operation deadlines plus explicit coalescing observability. Do not add age-based leader replacement, a leader lease, or a mandatory registry cap in the first implementation.

1. Redis deadlines remain client-owned. Every borrowed Redis read, write, and invalidate promise must settle within a finite aggregate budget covering connection, retry, reconnect, offline queueing, network, and command execution. This contract is coordinated with Prevent Redis outages from amplifying latency, source load, and logs #38.
2. Add a default shared fallback deadline. CachedOptions gains fallbackTimeoutMs?: number | null, configured once per cached wrapper/use case rather than per invocation. Omission uses a 60,000 ms safety ceiling, a positive number overrides it, and explicit null disables it.
3. Require finite settlement for injected async work. Caller-supplied Awaitable hooks used by DialCache—including fallbacks, custom serializer load/dump, Redis operations, and custom samplers/providers—must resolve or reject within an application-defined finite deadline.
4. Expose process-flight state. Report active leaders, joined followers, and oldest leader age without a sweeper or background timer.
5. Do not hedge or evict by age. A stale-looking flight remains authoritative until it settles. Starting a replacement while the old operation continues can duplicate source work and allow late cache writes.
6. Defer a global cardinality cap. Finite deadlines bound residence time, while application ingress/backpressure owns total concurrency. A map-entry cap that runs overflow work uncoalesced bounds registry metadata but not actual promises or backend work. Revisit a configurable maxTrackedFlights only if production evidence requires a deterministic registry cardinality bound.

Timeout semantics

Shared fallback deadline

Proposed API:

const getUser = dialcache.cached(fetchUser, {
  keyType: "user_id",
  useCase: "GetUser",
  cacheKey: (id) => id,
  fallbackTimeoutMs: 2_000,
});

• The option is static per cached wrapper so whichever request happens to become leader cannot choose policy for every follower.
• One timer starts only when fallback execution begins. Cache hits and followers allocate no timeout timer.
• The leader and every joined follower reject with an exported FallbackTimeoutError when the deadline expires.
• Timeout unwinds the cache chain before Redis or local publication. An eventual late fallback result is ignored and cannot populate DialCache.
• This is a wait/publication deadline, not cancellation. The underlying database or HTTP operation may continue; source-native timeout or AbortSignal support remains preferred.
• Omission uses the 60,000 ms safety default. A positive integer overrides it for that wrapper; explicit null disables the deadline. The default is an emergency retention ceiling, not the recommended application latency budget.
• Apply the explicit option consistently whenever the cached wrapper invokes its fallback, including enabled pass-through paths; caching state must not silently change the configured execution policy.
• A follower joining an existing flight inherits the leader's remaining deadline rather than receiving a fresh 60-second budget.
• The new finite default is a deliberate behavioral compatibility change; document fallbackTimeoutMs: null as the explicit migration escape hatch in release notes.
• Preserve the existing bounded error taxonomy: report the failure site as error="fallback"; the typed error distinguishes timeout programmatically.

Redis deadlines

DialCache must not implement Redis timeouts with a superficial Promise.race. A timed-out queued write may otherwise execute later. The caller-owned client/adapter must supply a finite operational budget and define whether timed-out commands can still be dispatched.

Followers versus leader timeout

• fallbackTimeoutMs is a leader deadline shared by all followers.
• A caller that stops awaiting or applies its own outer timeout does not cancel the leader or other followers.
• No DialCache-specific per-follower AbortSignal API is part of this first implementation.
• No leader cancellation is implied; the flight clears because the promise visible to DialCache rejects.

Observability contract

Add an exact synchronous per-instance snapshot, tentatively:

dialcache.getCoalescingState();
// {
//   process: {
//     activeLeaders,
//     activeFollowers,
//     oldestLeaderAgeMs,
//   },
// }

• A leader is one process-registry entry.
• A follower is a call joined to a pending leader and remains counted until that leader settles; JavaScript promise abandonment is not observable without an explicit cancellation API.
• Store one monotonic start timestamp per leader and maintain counts incrementally.
• Calculate age when the snapshot is requested. Do not add polling, a sweeper, or hot-path scans.
• Keep request-local lifecycle/capacity behavior unchanged under the short-lived bounded-cardinality assumption from Add a runtime-controlled request-local cache layer #51.

Implementation plan

1. Public contract and validation

• Add fallbackTimeoutMs?: number | null to CachedOptions, with undefined resolving to the 60,000 ms default, a positive number overriding it, and null explicitly disabling it.
• Export FallbackTimeoutError with stable timeoutMs and useCase fields and no raw cache key.
• Validate numeric deadlines as positive integers within the runtime timer's supported range.
• Document the finite-settlement requirement on DialCacheRedisClient, Serializer, samplers/providers, and cached fallbacks.

2. Fallback deadline runtime

• Centralize fallback invocation so all intended fallback paths use one timeout helper without changing unrelated cache metrics.
• Keep cache-hit and follower paths timer-free; create only one timer when a fallback leader actually runs. Explicit null keeps fallback invocation timer-free.
• Create one timer per actual fallback invocation, clear it in finally, and unref() it where supported.
• Attach fulfillment and rejection handling to the underlying fallback so a late rejection cannot become unhandled.
• Ensure timeout rejection exits before serializer dump, Redis write, and local population.

3. Flight observability

• Replace raw process-map values with minimal flight records containing the promise, monotonic start time, and follower count.
• Add identity-checked settlement cleanup and aggregate counters.
• Export a read-only getCoalescingState() snapshot type.
• Benchmark the leader record/clock overhead and preserve the existing coalescing boundary.

4. Redis integration guidance

• Expand README and interface JSDoc with the required aggregate Redis budget.
• Add caller-owned node-redis and Valkey GLIDE configuration examples based on their supported timeout/retry/queue controls.
• Explicitly warn that an outer Promise.race is not command cancellation and may permit late writes.
• Keep all client lifecycle and timeout ownership in Prevent Redis outages from amplifying latency, source load, and logs #38.

5. Validation

• Fallback resolves and rejects before the deadline.
• Fallback never settles: leader and followers receive FallbackTimeoutError, the process flight clears, and a later call retries.
• Late fallback resolution does not write Redis or local cache.
• Late fallback rejection produces no unhandled rejection.
• Timeout timers are cleared and do not keep the process alive.
• Omitted fallbackTimeoutMs uses exactly the 60,000 ms default; a numeric override uses its configured budget; explicit null allocates no timer.
• Redis read/write rejection within the client budget clears the flight and preserves current fail-open semantics.
• Unique-key bursts may grow during the configured deadline but all timed-out entries drain afterward.
• An externally timed-out follower does not cancel the shared leader or affect another follower.
• Coalescing snapshots accurately reflect leader creation, joins, age, rejection/timeout cleanup, and retry.
• Existing request-local, invalidation, serializer, metrics, packed-consumer, and integration suites remain green.

Out of scope

• Cross-process leases or stampede control: Design optional cross-process cache stampede protection #7.
• Redis connection/retry/offline-queue policy ownership: Prevent Redis outages from amplifying latency, source load, and logs #38.
• Age-based hedging, leader eviction, or overlapping same-key generations.
• A mandatory global registry cap or application load shedding.
• Generic cancellation of arbitrary caller promises.

Acceptance criteria

• The Redis and caller-supplied async liveness requirements are explicit and tested at DialCache's boundaries.
• fallbackTimeoutMs defaults to 60,000 ms, supports positive per-wrapper overrides and explicit null opt-out, and has the shared-operation behavior and cleanup guarantees above.
• Active process leaders, followers, and oldest age are observable.
• Hung/rejected/timed-out leaders, unique-key bursts, follower abandonment, retry, and cleanup are covered.
• Timeout paths produce no late DialCache publication and no unhandled rejection.
• No leader lease, age-based replacement, or implicit duplicate-write behavior is introduced.
• Hot cache-hit and follower paths do not acquire per-call timers.
• Process-local lifecycle work remains separate from cross-process coordination.