[Bugfix][Rust Frontend] Fail token-less streams in vllm-bench openai-chat - #59736

Purpose

Fixes #59722.

The Rust vllm-bench openai-chat backend set output.success = true whenever the stream ended cleanly, even if no token ever arrived. The openai (completions) backend and Python vllm bench serve only count a request as successful once a first token is received.

A token-less 200 response was therefore counted as a success with ttft, latency and output_tokens all at their defaults. As @aron-intframe pointed out in the issue, the impact goes beyond Mean TTFT:

• Mean TTFT and Mean E2EL are pulled down, because both are 0 for those requests.
• The 1 + output.itl.len() output-length fallback in metrics/calculator.rs counts each of them as one phantom output token, inflating total output tokens and output throughput.
• They pass every goodput SLO, since TTFT/TPOT/E2EL are all 0.

This PR gates success on the existing first_token_received flag and fails the request with the same error message as the completions backend and Python, so the two Rust backends behave the same.

Behavior change (multi-turn). multi_turn.rs branches on output.success. Today a token-less turn is appended to the conversation history as an empty assistant message and the run continues; after this change that turn is reported as failed, like any other failed turn. This matches how the completions backend and Python already treat it.

Not a duplicate

Claimed in #59722. I checked open PRs that touch rust/src/bench/src/backends/openai_chat.rs:

• [Bench] Save server queue time and TTFT in detailed results #55809 adds server queue time / TTFT to the detailed results; different change.
• [Bugfix][Rust Frontend] Use server-reported prompt_tokens in vllm-bench #59694 (mine, under review) changes the usage-chunk block a few lines above. No behavioral overlap, but whichever lands second will need a trivial rebase.
• [Bugfix][Benchmark] Fail streams that end in a server error event #57519 handles server error events, on the Python side only.

Test Plan

New test openai_chat::tests::test_tokenless_stream_is_failed: the existing local SSE test server streams zero token chunks followed by the usage chunk and [DONE]. The test asserts success == false and that the error reports the missing first chunk.

cargo test -p vllm-bench --lib
cargo clippy -p vllm-bench --all-targets --locked -- -D warnings
cargo fmt --all -- --check

End-to-end: a mock OpenAI-compatible server that returns 200 with only the usage chunk and [DONE] for every 8th request, --dataset-name random --random-input-len 100 --random-output-len 8 --num-prompts 32 --max-concurrency 8.

Test Result

• cargo test -p vllm-bench --lib: 169 passed. Clippy and fmt are clean.
• With the fix reverted, the new test fails (assertion failed: !output.success).

End-to-end against the mock server:

The before row shows the four phantom output tokens (228 vs 224) and the TTFT skew. After the fix, the chat backend matches the completions backend and Python.

AI assistance: prepared with Claude Code; I reviewed every changed line and ran the tests above locally.