Establish a local-hit performance budget and remove avoidable hot-path work

[lan17]

Priority

P2 — Medium — benchmark-led hot-path optimization after production safety guardrails.

Current state

Each enabled invocation still constructs a key, resolves configuration, allocates its fallback closure, and takes clocks on measured cache/fallback paths. With metrics omitted, optional chaining skips adapter calls and most label construction, but the clocks and built-in provider call/await remain.

#69 expanded the committed benchmark to five semantic scenarios and reported an informational base-versus-head overhead of about 0.10 microseconds per process-local hit. There is still no stable local-hit budget, environment-normalized baseline, or regression threshold.

Current evidence:

• Per-call path:

  DialCache/src/dialcache.ts

  Lines 252 to 309 in 34a45fa

  	const run = async (...args: Parameters<Fn>): Promise<CachedValue<Fn>> => {
  	// `fn`'s awaited result is the cached value by construction; the generic `Fn` erases it to `unknown`.
  	const rawFallback = async (): Promise<CachedValue<Fn>> => (await fn(...args)) as CachedValue<Fn>;
  	const noLayerLabels = {
  	cacheNamespace: this.namespace,
  	useCase: options.useCase,
  	keyType: options.keyType,
  	layer: NO_CACHE_LAYER,
  	} as const;
  	if (!this.isEnabled()) {
  	this.metrics?.disabled({ ...noLayerLabels, reason: "context" });
  	return await rawFallback();
  	}
  	const fallback = (): Promise<CachedValue<Fn>> =>
  	withFallbackTimeout(rawFallback, options.useCase, fallbackTimeoutMs);
  	let key: DialCacheKey;
  	try {
  	key = this.buildKey(options, options.cacheKey(...args));
  	} catch (error) {
  	this.logger.error("Could not construct DialCache key", error);
  	this.metrics?.error({
  	...noLayerLabels,
  	error: "key_construction",
  	inFallback: false,
  	});
  	return await this.callFallback(noLayerLabels, fallback);
  	}
  	let keyConfig: DialCacheKeyConfig | null;
  	try {
  	keyConfig = await fetchKeyConfig(this.configProvider, key);
  	} catch (error) {
  	// Provider failure: fail open and run uncached, mirroring the per-layer config_error path.
  	this.logger.warn("Could not resolve DialCache key config", error);
  	this.recordError(key, NO_CACHE_LAYER, "config_resolution");
  	this.metrics?.disabled({ ...noLayerLabels, reason: "config_error" });
  	return await this.callFallback(noLayerLabels, fallback);
  	}
  	// An unawaited child can inherit the async store after its outer enable()
  	// callback settles. The closed holder turns that detached work back into
  	// pass-through instead of allowing it to repopulate request state.
  	if (!this.isEnabled()) {
  	this.metrics?.disabled({ ...noLayerLabels, reason: "context" });
  	return await this.callFallback(noLayerLabels, fallback);
  	}
  	if (keyConfig?.requestLocal === true) {
  	const requestLocalCache = getOrCreateRequestLocalCache(this.context);
  	if (requestLocalCache !== null) {
  	return await this.getThroughRequestLocal(requestLocalCache, key, keyConfig, fallback);
  	}
  	}
  	return await this.getThroughSharedLayers(key, keyConfig, fallback, CacheLayer.LOCAL);

• Local and fallback clocks:

  DialCache/src/dialcache.ts

  Lines 508 to 583 in 34a45fa

  	private readLocalWithResolvedConfig<T>(key: DialCacheKey, layerConfig: ResolvedLayerConfig): CacheGetResult<T> {
  	const start = performance.now();
  	try {
  	const result = this.localCache.getWithResolvedConfig<T>(key, layerConfig);
  	this.metrics?.request(labelsFor(key, CacheLayer.LOCAL));
  	this.metrics?.observeGet(labelsFor(key, CacheLayer.LOCAL), elapsedSeconds(start));
  	if (result.status === "miss") {
  	this.metrics?.miss(labelsFor(key, CacheLayer.LOCAL));
  	}
  	return result;
  	} catch (error) {
  	this.logger.error("Error getting value from local cache", error);
  	this.recordError(key, CacheLayer.LOCAL, "cache_read");
  	this.metrics?.disabled({ ...labelsFor(key, CacheLayer.LOCAL), reason: "config_error" });
  	return { status: "disabled", reason: "config_error" } as const;
  	}
  	}
  	private async resolveRemoteLayerConfig(key: DialCacheKey, keyConfig: DialCacheKeyConfig | null) {
  	try {
  	const result = await resolveLayerConfigResult({
  	config: keyConfig,
  	key,
  	layer: CacheLayer.REMOTE,
  	rampSampler: this.rampSampler,
  	});
  	if (result.status === "disabled") {
  	this.metrics?.disabled({ ...labelsFor(key, CacheLayer.REMOTE), reason: result.reason });
  	}
  	return result;
  	} catch (error) {
  	this.logger.warn("Error resolving Redis cache config", error);
  	this.recordError(key, CacheLayer.REMOTE, "config_resolution");
  	return { status: "disabled", reason: "config_error", ...(key.trackForInvalidation ? { skipCacheWrite: true } : {}) } as const;
  	}
  	}
  	private async readRemoteWithResolvedConfig<T>(redisCache: RedisCache, key: DialCacheKey, layerConfig: ResolvedLayerConfig) {
  	const start = performance.now();
  	try {
  	const result = await redisCache.getWithResolvedConfig<T>(key, layerConfig);
  	this.metrics?.request(labelsFor(key, CacheLayer.REMOTE));
  	this.metrics?.observeGet(labelsFor(key, CacheLayer.REMOTE), elapsedSeconds(start));
  	if (result.status === "miss") {
  	this.metrics?.miss(labelsFor(key, CacheLayer.REMOTE));
  	}
  	return result;
  	} catch (error) {
  	this.logger.warn("Error getting value from Redis cache", error);
  	return {
  	status: "disabled",
  	reason: "config_error",
  	...(key.trackForInvalidation ? { skipCacheWrite: true } : {}),
  	} as const;
  	}
  	}
  	private async putLocalFailOpen<T>(key: DialCacheKey, value: T, config?: { readonly ttlSec: number }): Promise<void> {
  	try {
  	await this.localCache.put(key, value, config);
  	} catch (error) {
  	this.logger.warn("Error putting value in local cache", error);
  	this.recordError(key, CacheLayer.LOCAL, "cache_write");
  	}
  	}
  	private async callFallback<T>(labels: CacheMetricLabels, fallback: () => Promise<T>): Promise<T> {
  	const start = performance.now();
  	try {
  	return await fallback();
  	} catch (error) {
  	this.metrics?.error({ ...labels, error: "fallback", inFallback: true });
  	throw error;
  	} finally {
  	this.metrics?.observeFallback(labels, elapsedSeconds(start));
  	}

• Five-scenario benchmark: https://github.com/lan17/DialCache/blob/34a45facfc4cb11ca7c08d5506e74b3dbcef4dd2/scripts/benchmark-request-local.mjs
• Shared harness work: Add a repeatable performance and scale benchmark suite #35
• Built-in provider fast path: Keep runtime configuration lookup cheap and bounded #39

Scope

Profile first, then remove only measured hot-path work:

• Skip clocks entirely when metrics are absent.
• Use Keep runtime configuration lookup cheap and bounded #39's internal no-custom-provider fast path if approved.
• Precompute immutable definition-level metric metadata where measurement supports it.
• Reduce safe metrics-wrapper closures without changing fail-open behavior or event semantics.
• Add 0-, 1-, and 20-argument local-hit scenarios through Add a repeatable performance and scale benchmark suite #35.

Out of scope

• Cache-key identity changes.
• Counter sampling or metric semantic changes.
• Built-in memoization/timeouts for caller-supplied dynamic providers.
• Vendor-client encoding optimizations inside core.
• Performance claims without warmup, repeated samples, and captured environment.

Acceptance criteria

• Reproducible before/after results through Add a repeatable performance and scale benchmark suite #35.
• CPU/allocation profiles identify each changed cost.
• Metrics-disabled paths avoid unnecessary clocks.
• No-custom-provider calls avoid the synthetic async provider path if Keep runtime configuration lookup cheap and bounded #39 is approved.
• Metrics-enabled counters/timers and throwing-adapter isolation remain exact.
• No public API or Redis wire change.