Skip to content

Expose optional cache outcome events on the high-level Client #3777

Open

Description

@Yut0Miura

Description

The high-level Client has a built-in response cache for methods such as tools/list, prompts/list, resources/list, resources/templates/list, and resources/read.

When one of these calls returns quickly, transport-level instrumentation alone can't tell whether the result came from the cache or from a network request. This makes it harder to diagnose MCP tool-discovery latency, measure cache effectiveness, and investigate stale resource data in applications built around LLM agents.

It would be useful to expose an optional hook that reports cache decisions directly from the high-level Client, without requiring applications to wrap client methods or instrument the cache themselves.

For example, the API could expose a small event like this:

CacheEvent(
    method="tools/list",
    outcome="hit",
)

The initial set of outcomes should cover:

  • hit - a fresh cached response is returned without a transport request.
  • miss - a normal cache lookup finds no fresh entry, so the request is sent and the response is cached according to the existing policy.
  • bypass - the call skips cache reads and writes.
  • refresh - the call fetches a new response regardless of whether a fresh entry exists, then updates the cache according to the existing policy.

The reported outcome should reflect what actually happened. For example, a request carrying metadata may require a network round trip even when the default cache mode is use.

Keep the event schema small and stable so applications can map events to their own metrics, logs, or traces. Events should identify the logical MCP method and cache outcome without exposing tool arguments, resource contents, full resource URIs, credentials, or private cache keys.

TTL expiry should be reflected in the resulting lookup outcome rather than counted as a second event for the same operation. Notification-driven invalidation can have a separate event if needed, with semantics distinct from a cache miss.

The hook should be opt-in and leave existing caching behavior unchanged, including TTL handling, private/public cache partitioning, pagination, derived tool metadata, and notification-driven invalidation. Observer failures must not affect request results or prevent invalidation from completing. Callback delivery semantics should be documented, and user callbacks should never run while internal cache locks are held.

This would give MCP applications a way to understand cache behavior and diagnose latency without introducing a mandatory telemetry dependency or changing the caching policy.

Tests should cover hits without transport calls, misses, bypass, refresh, TTL expiry, notification-driven invalidation, observer failures, and the absence of cached payloads or private cache keys in emitted events.

References

Activity

  1. added
    enhancementRequest for a new feature that's not currently supported
    on Oct 27, 2026
  2. raramos9 commented on Oct 29, 2026

    @raramos9

    I've been building MCP integrations for my own projects, and having visibility into cache decisions would be useful for understanding tool-discovery latency and debugging stale responses.

    I took a look at the client caching implementation, particularly _cached_fetch() and read_resource(). It seems feasible to add an opt-in observer without changing the underlying caching behavior.

    Would maintainers be open to a small public CacheEvent type and optional callback on the high-level Client?

    I'd aim to keep the event payload minimal, isolate callback failures, and add tests for the existing cache modes, TTL expiry, and invalidation behavior.

  3. Yut0Miura commented on Oct 29, 2026

    @Yut0Miura
    Author

    This seems worth adding. An opt-in callback with a small event type should be enough, as long as it reports the actual cache outcome and doesn't change existing behavior.

    I'd make sure the tests cover metadata-triggered bypasses, TTL expiry, invalidation, and callback failures. Callbacks also shouldn't run while cache locks are held.

  4. raramos9 commented on Oct 29, 2026

    @raramos9

    Awesome sounds good. The proposed event model and test coverage seem reasonable. Before I start, would a maintainer be able to confirm that this is an approach they'd be comfortable accepting? If so, I'm happy to take this on.

  5. Yut0Miura commented on Oct 29, 2026

    @Yut0Miura
    Author

    Yeah, this sounds good to me. Go ahead with it. I'd keep the API small and make sure the callback can't interfere with normal cache behavior. The tests you outlined should cover the main cases.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementRequest for a new feature that's not currently supported

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions