Fix proxy plugin config, logging and agent passthrough; port MCP to mcp 2.x (0.4.0) - #331

Fixes #330

Summary

Each problem reported in #330 was reproduced against a mock llama.cpp server before fixing.

DEBUG logging didn't work

• --log / OPTILLM_LOG only changed the optillm.server logger, so proxy/plugin DEBUG logs were dropped. It now sets the root logger and is case-insensitive (OPTILLM_LOG=DEBUG was silently ignored before).
• Removed forced log levels in proxy/routing.py and proxy_plugin.py. monitoring.log_level is now an optional proxy-only override.

Server target in the proxy config was ignored

• Without ~/.optillm/proxy_config.yaml, the bundled example_config.yaml was loaded, sending traffic to OpenAI/Azure/Together and using localhost:8080 only as fallback_only. It is no longer on the search path; a blank template is created instead.
• One failed request (e.g. a 400) marked the only provider unhealthy, so later requests silently went to the --base-url client. 4xx responses (other than 408/429) no longer mark providers unhealthy, and unhealthy providers are retried before the default client.
• /v1/models now lists the providers' models (plus model_map aliases) with --approach proxy.

Crush / agent clients failing

• The conversation was flattened into one user message ("User: ...\nAssistant: None"), dropping tool_calls and tool results. The proxy and none approaches now forward the original messages (with <optillm_approach> tags stripped).
• Streaming responses only emitted content, so a tool-call response became just data: [DONE]. Full completions are now streamed with tool_calls, finish_reason and usage.
• stream_options was forwarded upstream without stream, which providers reject. It is no longer forwarded.
• parse_conversation no longer crashes on content: null.

Docs: updated the proxy README (logging, config location, models, failover, local servers/agents).

MCP plugin ported to mcp 2.x (CI was failing on fresh installs: No module named 'mcp.client.websocket')

• McpError → MCPError; tool/result fields are snake_case in 2.x (input_schema, mime_type, is_error).
• New streamable_http transport (headers/timeouts supported). The websocket transport was removed from the SDK, so it now returns a clear error pointing to streamable_http or sse.
• The logging session handles 2.x request/notification objects; capabilities are detected by value; non-CallToolResult results are handled.
• Requires mcp>=2,<3. README MCP section updated.
• Verified against real mcp 2.2.0 servers over stdio, SSE and Streamable HTTP (connect, discover, call tool).

CI: tests/test_proxy_plugin.py now runs in the unit-test job.

Release: bumps version to 0.4.0 (breaking changes below).

Breaking changes

• MCP websocket transport removed (dropped from the MCP Python SDK in 2.x). Configs using "transport": "websocket" must switch to streamable_http or sse.
• Requires mcp>=2,<3. Environments that also need mcp<2 won't install.
• Proxy no longer loads the bundled example_config.yaml when ~/.optillm/proxy_config.yaml is missing. It creates an empty template and uses the server's default client (--base-url) until providers are added.
• OPTILLM_LOG_LEVEL removed. Use --log / OPTILLM_LOG (now case-insensitive); monitoring.log_level in the proxy config is an optional proxy-only override.

Behavior changes (not breaking): the none and proxy approaches now forward the full conversation (including tool calls) instead of one flattened message, and streamed responses carry tool calls, finish reason and usage in OpenAI chunk format.

Testing

• Frame SAST (--fail-on high, same pinned commit as CI) passes on all changed Python files.
• tests/test_mcp_plugin.py, tests/test_plugins.py, tests/test_proxy_plugin.py: 63 passed, 1 skipped with mcp 2.2.0.
• New tests/test_proxy_plugin.py (10 tests) covers passthrough, streaming tool calls, config loading, failover and /v1/models.
• test_proxy_plugin_token_counts now patches the config load. It previously read the developer's real ~/.optillm/proxy_config.yaml and made live calls.
• End-to-end check with the OpenAI SDK streaming parser against a mock upstream, for both --approach proxy and the default passthrough.
• test_basic_none_approach_logging fails on main as well (local-model tokenizer error); unchanged.

🤖 Generated with Claude Code