Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Cap concurrent Streamable HTTP sessions and expose the session limits…
… on the server factories

Add `max_sessions` (DEFAULT_MAX_SESSIONS = 10_000, `None` for no
limit) to StreamableHTTPSessionManager: while that many stateful
sessions are open, a request that would open another is answered 503
with a JSON-RPC error body and nothing is allocated; existing sessions
are untouched and room frees up as they end or expire. This matches the
Ruby SDK's defaults (the C# SDK uses the same 10 000 figure).

`session_idle_timeout` and `max_sessions` are accepted by
`Server.streamable_http_app()`, `MCPServer.streamable_http_app()`,
`run_streamable_http_async()` and `run(transport="streamable-http")`,
the same way `max_request_body_size` is, so applications can tune or
disable them without reaching into `session_manager` after the fact.
Docs: run/index.md options list, run/legacy-clients.md session cost,
troubleshooting.md.
  • Loading branch information
maxisbey committed Aug 26, 2026
commit ae2daca69bba7b27c0eafbfdd972498185bbf6cf
6 changes: 6 additions & 0 deletions docs/run/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,12 @@ Each transport has its own keyword arguments, all on `run()`:
* `max_request_body_size`: largest accepted request body in bytes. Defaults to 4 MiB; larger requests
receive HTTP 413 before parsing or session creation. Raise it only when legitimate MCP messages
exceed that size.
* `session_idle_timeout`: how long, in seconds, a [legacy](legacy-clients.md) (session-based)
client's session may sit with no request in flight before the server closes it. Defaults to 1800
(30 minutes); `None` keeps sessions until the client deletes them. A client with an open `GET`
stream or a request still being answered is never idle.
* `max_sessions`: how many such sessions one app holds at once. Defaults to 10 000; while that many
are open, a request that would open another gets HTTP 503. `None` removes the limit.
* `event_store`, `retry_interval`, `transport_security`: resumability and DNS-rebinding protection. They can wait, until you deploy somewhere other than localhost; **[Deploy & scale](deploy.md)** covers `transport_security`.

!!! warning
Expand Down
7 changes: 7 additions & 0 deletions docs/run/legacy-clients.md
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,13 @@ On one worker that is invisible. On two, it is the whole problem: a request that
events to a client reconnecting to the *same* session), not a session store. It never makes a
session reachable from another process.

The record is not kept forever. A client that ends its session (`DELETE`) frees it at once;
a session that has had no request in flight for `session_idle_timeout` seconds (default 1800; an
open `GET` stream or a request being answered counts as in flight) is closed, and its next request
gets the same `404` a stray ID gets, so the client has to `initialize` again. Each worker process
holds at most `max_sessions` of them (default 10 000) and answers `503` to a request that would
open one more. Both are `run()` / `streamable_http_app()` options.

## The one knob: `stateless_http`

If stickiness is a cost you refuse to pay, there is exactly one thing you can change.
Expand Down
8 changes: 4 additions & 4 deletions docs/troubleshooting.md
Original file line number Diff line number Diff line change
Expand Up @@ -246,7 +246,7 @@ app = Starlette(routes=[Mount("/", app=mcp.streamable_http_app())], lifespan=lif

## `MCPError: Session not found`

The server does not recognise the `Mcp-Session-Id` your client sent, almost always because the server **restarted** (or you were routed to a different instance). Sessions live in that one process's memory.
The server does not recognise the `Mcp-Session-Id` your client sent, because the server **restarted** (or you were routed to a different instance), or because the session **expired**: a legacy session with no request in flight for `session_idle_timeout` (30 minutes by default; an open `GET` stream or a request being answered counts as in flight) is closed, as is one the client ended with `DELETE`. Sessions live in that one process's memory.

There is no server bug to find. The HTTP response is a `404` whose body *is* JSON-RPC, so, unlike the `421` above, the python `Client` shows you this one verbatim:

Expand All @@ -256,9 +256,9 @@ There is no server bug to find. The HTTP response is a `404` whose body *is* JSO

The fix is to reconnect: leave the `async with Client(...)` block and enter a new one, which negotiates a fresh session. For a long-lived client, that means catching `MCPError` around your calls and reconnecting on this message rather than retrying inside a dead session.

If it happens *without* a restart, you are running more than one worker without sticky sessions: each worker holds its own session table, so a request routed to the wrong one lands here. **[Deploy & scale](run/deploy.md)** and **[Serving legacy clients](run/legacy-clients.md)** own that story and its two fixes (sticky routing, or `stateless_http=True`).
If it happens *without* a restart and without the client having gone quiet that long, you are running more than one worker without sticky sessions: each worker holds its own session table, so a request routed to the wrong one lands here. **[Deploy & scale](run/deploy.md)** and **[Serving legacy clients](run/legacy-clients.md)** own that story and its two fixes (sticky routing, or `stateless_http=True`).

For the server operator, the matching log line is `Rejected request with unknown or expired session ID: <id>`. It is logged at `INFO`, so it is invisible at the usual `WARNING` threshold. Seeing it in bursts right after a deploy is normal; every connected client is reconnecting.
For the server operator, the matching log line is `Rejected request with unknown or expired session ID: <id>`. It is logged at `INFO`, so it is invisible at the usual `WARNING` threshold. Seeing it in bursts right after a deploy is normal; every connected client is reconnecting. When the session expired instead, that line is preceded by `Session <id> idle timeout`, also at `INFO`.

## `MCPError: Method not found`

Expand Down Expand Up @@ -411,7 +411,7 @@ mcp = MCPServer("Weather", request_state_security=RequestStateSecurity(keys=[key
* `Tool already exists:` in the server log is the only sign that two same-named tools collapsed into one.
* One 421, three spellings: `Server returned an error response` (the python `Client`), `421 Misdirected Request` / `Invalid Host header` (everything else), `Invalid Host header: <host>` (the server log). Fix: `transport_security=TransportSecuritySettings(allowed_hosts=[...])`.
* `Task group is not initialized` -> a mounted app whose host lifespan never entered `mcp.session_manager.run()`.
* `Session not found` -> the server restarted; reconnect.
* `Session not found` -> the server restarted or the session expired (`session_idle_timeout`); reconnect.
* `Cannot send 'elicitation/create': ... no back-channel ...` -> `ctx.elicit()` needs a server-to-client channel: a `2026-07-28` connection never has one, `stateless_http=True` takes away the legacy one, and `json_response=True` takes away the request-scoped one. Use a resolver (a legacy client also needs a server that keeps the channel). Its neighbour `Method not found` is a request for a method the other side's protocol revision doesn't have.
* `Client did not declare the form elicitation capability ...` and `Elicitation not supported` -> the client is missing `elicitation_callback=`.
* `Invalid or expired requestState` never says why on the wire. The server log does; `unknown key` means share `RequestStateSecurity(keys=[...])` across workers.
11 changes: 10 additions & 1 deletion src/mcp/server/lowlevel/server.py
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,12 @@ async def main():
from mcp.server.models import InitializationOptions
from mcp.server.runner import serve_dual_era_loop
from mcp.server.streamable_http import EventStore
from mcp.server.streamable_http_manager import StreamableHTTPASGIApp, StreamableHTTPSessionManager
from mcp.server.streamable_http_manager import (
DEFAULT_MAX_SESSIONS,
DEFAULT_SESSION_IDLE_TIMEOUT,
StreamableHTTPASGIApp,
StreamableHTTPSessionManager,
)
from mcp.server.transport_security import DEFAULT_MAX_REQUEST_BODY_SIZE, TransportSecuritySettings
from mcp.shared._stream_protocols import ReadStream, WriteStream
from mcp.shared.exceptions import MCPDeprecationWarning
Expand Down Expand Up @@ -722,6 +727,8 @@ def streamable_http_app(
event_store: EventStore | None = None,
retry_interval: int | None = None,
max_request_body_size: int = DEFAULT_MAX_REQUEST_BODY_SIZE,
session_idle_timeout: float | None = DEFAULT_SESSION_IDLE_TIMEOUT,
max_sessions: int | None = DEFAULT_MAX_SESSIONS,
transport_security: TransportSecuritySettings | None = None,
host: str = "127.0.0.1",
auth: AuthSettings | None = None,
Expand All @@ -747,6 +754,8 @@ def streamable_http_app(
stateless=stateless_http,
security_settings=transport_security,
max_request_body_size=max_request_body_size,
session_idle_timeout=session_idle_timeout,
max_sessions=max_sessions,
)
self._session_manager = session_manager

Expand Down
16 changes: 15 additions & 1 deletion src/mcp/server/mcpserver/server.py
Original file line number Diff line number Diff line change
Expand Up @@ -93,7 +93,11 @@
from mcp.server.sse import SseServerTransport
from mcp.server.stdio import stdio_server
from mcp.server.streamable_http import EventStore
from mcp.server.streamable_http_manager import StreamableHTTPSessionManager
from mcp.server.streamable_http_manager import (
DEFAULT_MAX_SESSIONS,
DEFAULT_SESSION_IDLE_TIMEOUT,
StreamableHTTPSessionManager,
)
from mcp.server.subscriptions import InMemorySubscriptionBus, ListenHandler, SubscriptionBus
from mcp.server.transport_security import DEFAULT_MAX_REQUEST_BODY_SIZE, TransportSecuritySettings
from mcp.shared.exceptions import MCPError
Expand Down Expand Up @@ -388,6 +392,8 @@ def run(
event_store: EventStore | None = ...,
retry_interval: int | None = ...,
max_request_body_size: int = ...,
session_idle_timeout: float | None = ...,
max_sessions: int | None = ...,
transport_security: TransportSecuritySettings | None = ...,
) -> None: ...

Expand Down Expand Up @@ -1106,6 +1112,8 @@ async def run_streamable_http_async( # pragma: no cover
event_store: EventStore | None = None,
retry_interval: int | None = None,
max_request_body_size: int = DEFAULT_MAX_REQUEST_BODY_SIZE,
session_idle_timeout: float | None = DEFAULT_SESSION_IDLE_TIMEOUT,
max_sessions: int | None = DEFAULT_MAX_SESSIONS,
transport_security: TransportSecuritySettings | None = None,
) -> None:
"""Run the server using StreamableHTTP transport."""
Expand All @@ -1118,6 +1126,8 @@ async def run_streamable_http_async( # pragma: no cover
event_store=event_store,
retry_interval=retry_interval,
max_request_body_size=max_request_body_size,
session_idle_timeout=session_idle_timeout,
max_sessions=max_sessions,
transport_security=transport_security,
host=host,
)
Expand Down Expand Up @@ -1270,6 +1280,8 @@ def streamable_http_app(
event_store: EventStore | None = None,
retry_interval: int | None = None,
max_request_body_size: int = DEFAULT_MAX_REQUEST_BODY_SIZE,
session_idle_timeout: float | None = DEFAULT_SESSION_IDLE_TIMEOUT,
max_sessions: int | None = DEFAULT_MAX_SESSIONS,
transport_security: TransportSecuritySettings | None = None,
host: str = "127.0.0.1",
) -> Starlette:
Expand All @@ -1281,6 +1293,8 @@ def streamable_http_app(
event_store=event_store,
retry_interval=retry_interval,
max_request_body_size=max_request_body_size,
session_idle_timeout=session_idle_timeout,
max_sessions=max_sessions,
transport_security=transport_security,
host=host,
auth=self.settings.auth,
Expand Down
42 changes: 26 additions & 16 deletions src/mcp/server/streamable_http_manager.py
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,9 @@
DEFAULT_SESSION_IDLE_TIMEOUT: Final = 30 * 60
"""Default idle period in seconds after which a stateful Streamable HTTP session is closed (30 minutes)."""

DEFAULT_MAX_SESSIONS: Final = 10_000
"""Default maximum number of concurrent stateful Streamable HTTP sessions per session manager."""


class StreamableHTTPSessionManager:
"""Manages StreamableHTTP sessions with optional resumability via event store.
Expand Down Expand Up @@ -73,6 +76,10 @@ class StreamableHTTPSessionManager:
sessions live until the client deletes them or the manager shuts down. Unused in stateless mode.
max_request_body_size: Maximum size in bytes for Streamable HTTP request bodies. Requests that
exceed this limit receive a 413 response before parsing or session creation. Defaults to 4 MiB.
max_sessions: Maximum number of concurrent stateful sessions. While that many sessions are open, a
request that would open another one receives a 503 response; existing sessions are unaffected and
room frees up as they end or expire. Defaults to 10 000; None removes the limit. Unused in stateless
mode.
"""

def __init__(
Expand All @@ -85,11 +92,14 @@ def __init__(
retry_interval: int | None = None,
session_idle_timeout: float | None = DEFAULT_SESSION_IDLE_TIMEOUT,
max_request_body_size: int = DEFAULT_MAX_REQUEST_BODY_SIZE,
max_sessions: int | None = DEFAULT_MAX_SESSIONS,
):
if session_idle_timeout is not None and session_idle_timeout <= 0:
raise ValueError("session_idle_timeout must be a positive number of seconds")
if max_request_body_size <= 0:
raise ValueError("max_request_body_size must be a positive number of bytes")
if max_sessions is not None and max_sessions <= 0:
raise ValueError("max_sessions must be a positive number of sessions or None")

self.app = app
self.event_store = event_store
Expand All @@ -99,6 +109,7 @@ def __init__(
self.retry_interval = retry_interval
self.session_idle_timeout = session_idle_timeout
self.max_request_body_size = max_request_body_size
self.max_sessions = max_sessions
self.asgi_app = RequestBodyLimitMiddleware(self._handle_request, max_request_body_size)

# Session tracking (only used if not stateless)
Expand Down Expand Up @@ -265,15 +276,7 @@ async def _handle_stateful_request(self, scope: Scope, receive: Receive, send: S
"Rejecting request for session %s: credential does not match the one that created the session",
request_mcp_session_id[:64],
)
body = JSONRPCError(
jsonrpc="2.0", id=None, error=ErrorData(code=INVALID_REQUEST, message="Session not found")
)
response = Response(
body.model_dump_json(by_alias=True, exclude_unset=True),
status_code=404,
media_type="application/json",
)
await response(scope, receive, send)
await _error_response("Session not found", 404)(scope, receive, send)
return
logger.debug("Session already exists, handling request directly")
await transport.handle_request(scope, receive, send)
Expand All @@ -287,6 +290,11 @@ async def _handle_stateful_request(self, scope: Scope, receive: Receive, send: S
# New session case
logger.debug("Creating new transport")
async with self._session_creation_lock:
if self.max_sessions is not None and len(self._server_instances) >= self.max_sessions:
logger.warning("Refusing to open a new session: %d sessions are already open", self.max_sessions)
await _error_response("Too many open sessions", 503)(scope, receive, send)
Comment thread
maxisbey marked this conversation as resolved.
Outdated
return

new_session_id = uuid4().hex
http_transport = StreamableHTTPServerTransport(
mcp_session_id=new_session_id,
Expand Down Expand Up @@ -365,20 +373,22 @@ async def run_server(*, task_status: TaskStatus[None] = anyio.TASK_STATUS_IGNORE
# TODO(L62): Align error code once spec clarifies
# See: https://github.com/modelcontextprotocol/python-sdk/issues/1821
logger.info(f"Rejected request with unknown or expired session ID: {request_mcp_session_id[:64]}")
body = JSONRPCError(
jsonrpc="2.0", id=None, error=ErrorData(code=INVALID_REQUEST, message="Session not found")
)
response = Response(
body.model_dump_json(by_alias=True, exclude_unset=True), status_code=404, media_type="application/json"
)
await response(scope, receive, send)
await _error_response("Session not found", 404)(scope, receive, send)

def _forget_session(self, session_id: str) -> None:
"""Stop tracking a session; requests naming it are answered 404 from then on."""
self._server_instances.pop(session_id, None)
self._session_owners.pop(session_id, None)


def _error_response(message: str, status_code: int) -> Response:
"""A JSON-RPC error body (no request id) with the given HTTP status."""
body = JSONRPCError(jsonrpc="2.0", id=None, error=ErrorData(code=INVALID_REQUEST, message=message))
return Response(
body.model_dump_json(by_alias=True, exclude_unset=True), status_code=status_code, media_type="application/json"
)


async def _send_and_report_status(app: ASGIApp, scope: Scope, receive: Receive, send: Send) -> int | None:
"""Run `app` for one request and return the HTTP status it answered with (None if it sent no response)."""
status: int | None = None
Expand Down
16 changes: 15 additions & 1 deletion tests/docs_src/test_asgi.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@

from docs_src.asgi import tutorial001, tutorial002, tutorial003, tutorial004, tutorial005, tutorial006
from mcp import Client
from mcp.server import MCPServer
from mcp.server import MCPServer, Server

# See test_index.py for why this is a per-module mark and not a conftest hook.
pytestmark = [pytest.mark.anyio, pytest.mark.filterwarnings("error::mcp.MCPDeprecationWarning")]
Expand Down Expand Up @@ -44,6 +44,8 @@ async def test_streamable_http_app_takes_runs_options_except_port() -> None:
"event_store",
"retry_interval",
"max_request_body_size",
"session_idle_timeout",
"max_sessions",
"transport_security",
"host",
}
Expand All @@ -68,6 +70,18 @@ async def test_streamable_http_app_applies_the_configured_request_body_limit() -
assert response.status_code == 413


async def test_streamable_http_app_applies_the_configured_session_limits() -> None:
"""The documented `session_idle_timeout` and `max_sessions` options reach the session manager, from
both the high-level and the low-level factory."""
server = MCPServer("Notes")
server.streamable_http_app(session_idle_timeout=5, max_sessions=7)
assert (server.session_manager.session_idle_timeout, server.session_manager.max_sessions) == (5, 7)

lowlevel = Server("Notes")
lowlevel.streamable_http_app(session_idle_timeout=None, max_sessions=None)
assert (lowlevel.session_manager.session_idle_timeout, lowlevel.session_manager.max_sessions) == (None, None)


async def test_mounting_at_the_root_keeps_the_default_path() -> None:
"""tutorial002: `Mount("/")` plus the default `streamable_http_path` leaves the endpoint at `/mcp`."""
(mount,) = tutorial002.app.routes
Expand Down
16 changes: 16 additions & 0 deletions tests/docs_src/test_legacy_clients.py
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,8 @@ def test_streamable_http_app_has_no_era_knob() -> None:
"event_store",
"retry_interval",
"max_request_body_size",
"session_idle_timeout",
"max_sessions",
"transport_security",
"host",
}
Expand All @@ -76,6 +78,20 @@ async def test_a_legacy_session_is_minted_in_process_and_a_stray_session_id_is_a
assert stray.status_code == 404


def test_legacy_sessions_expire_and_are_capped_by_default() -> None:
"""The cost section: a session record is dropped after 30 idle minutes and each worker process holds at most
10 000 of them, unless `run()` / `streamable_http_app()` say otherwise."""
server = MCPServer("Bookshop")
server.streamable_http_app()
assert server.session_manager.session_idle_timeout == 30 * 60
assert server.session_manager.max_sessions == 10_000

server = MCPServer("Bookshop")
server.streamable_http_app(session_idle_timeout=None, max_sessions=None)
assert server.session_manager.session_idle_timeout is None
assert server.session_manager.max_sessions is None


async def test_stateless_http_never_mints_a_session() -> None:
"""The `stateless_http=True` section: the same legacy `initialize` no longer gets an `Mcp-Session-Id`."""
app = MCPServer("Bookshop").streamable_http_app(stateless_http=True)
Expand Down
Loading
Loading