[horizontal scaling] How to actually build session persistence in streamable http MCP server?

I have a question here regarding how to build session persistence into an MCP server.

I am running my mcp server (FastMCP) in an ecs service with multiple tasks. I have built a redis event store to allow for resumability. The service sits behind an ALB that directs HTTPS traffic to it. Inspite of all of it, Im still observing intermittent Bad Request: No valid session ID provided 400 errors when I connect the mcpinspector.

My suspicion is StreamableHTTPSessionManager doesn't seem to be able to store session-id data in an external data source. Therefore traffic from a client has to always hit the same task since the session id is tied to a server.

In an AWS architecture, how do I configure my server and client to accomplish this. Is using sticky sessions the only route here? Are streamable http sessions not horizontally scalable?

@ahayehaye

It works now, thanks @ahayehaye

Add stateless_http=True in mcp run() parameter
(I also added nginx.ingress.kubernetes.io/affinity: "cookie" on my app ingress)

not-mt/nmt-fastapi-reference#96

feat: Add Redis for MCP session persistence

+1 to support stateful sessions in multiple pods

fix(docling): switch MCP transport from Streamable HTTP to SSE

Hit this on ECS too. Sticky sessions via ALB cookie work but it's a tax on every load-balancer config and breaks when a task cycles mid-session.

The real fix is a pluggable SessionStore protocol so Redis or DynamoDB can replace the in-memory dict in StreamableHTTPSessionManager.

stateless_http=True isn't a substitute when tools need auth context or resumability across calls. Anyone working on a PR for the store-protocol path?

Looking a RC Spec session-id will be removed soon - https://blog.modelcontextprotocol.io/posts/2026-07-28-release-candidate/
It mean that we're going to lose at all the persistent session, I guess

@alessandro308

If this is still open for contribution, I'd like to pick up the docs task. We operate a multi-replica streamable-HTTP MCP server in production (GKE behind a load balancer), so I can write this from operational experience rather than theory.

Proposed scope for a documentation page/section:

Why naive horizontal scaling breaks: _request_streams inside StreamableHTTPServerTransport is in-process state (stream id → active request queues), so persisting only the session id in Redis/a DB is insufficient — a follow-up request landing on a different replica has the session id but not the live streams.
Working deployment patterns, with trade-offs:
- session affinity / sticky routing at the LB (simplest; what we run),
- stateless mode for request/response-style servers that don't need server-initiated streams,
- event store + resumability (EventStore) for replay across replicas.
Forward-compat caveat: the 2026-07-28 RC removes session ids, so the doc should frame these as patterns for current-spec deployments and note what changes (per @alessandro308's point).

Happy to adjust scope to whatever fits the docs structure — and can have a PR up shortly once assigned.

The v2 rework on main targets the 2026-07-28 protocol revision, where Streamable HTTP is sessionless by construction — no initialize handshake, no Mcp-Session-Id, each request a self-contained POST — so horizontal scaling needs nothing from the SDK: any replica behind a plain round-robin load balancer can answer (Deploy & scale). Clients on 2025-11-25 or earlier still land on the session-based leg of the same app, and that session record is still an in-process dict with no pluggable store, so those connections keep the trade-off this thread already found: sticky routing, or stateless_http=True at the cost of the server-to-client channel (Serving legacy clients, Protocol versions). The two things statelessness doesn't hand you across workers are on the deploy page too: a shared RequestStateSecurity(keys=[...]) plus the same server name for multi-round-trip tools, and your own SubscriptionBus if change notifications have to cross replicas. For the sampling case raised above, 2026-07-28 retires the server-to-client back-channel in favour of the multi-round-trip return path (Resolve on MCPServer), so there's no stream left to pin to a worker. So v2 doesn't add the Redis/DB-backed session store proposed here — the modern path has no session to persist — and the EventStore and session-manager workarounds people posted remain the practical answer for pre-2026 clients today.

Closing this as it's been fixed in v2, which is now released. If you still run into this on v2, feel free to reopen.

_{AI Disclaimer}

Navigation Menu

Repository navigation

[horizontal scaling] How to actually build session persistence in streamable http MCP server? #880

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

[horizontal scaling] How to actually build session persistence in streamable http MCP server? #880

Description

Activity

itdar commented on Oct 3, 2025

aniketkalamkar commented on Jan 15, 2026

MukundaKatta commented on May 10, 2026

Ar-maan05 commented on Jun 5, 2026

IgnazioDS commented on Jun 8, 2026

alessandro308 commented on Jun 9, 2026

ennsharma commented on Jun 11, 2026

maxisbey commented on Jul 29, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions