fix(serve): a lone surrogate in the body was a 500 instead of a 400 - #454
Merged
Merged
Conversation
Owner
|
Thank you for finding this one, a lone surrogate turning a caller mistake into a 500 is a real bug and the emoji case in the test is well thought out. Two changes before merge. The recursive walk raises RecursionError on nested states that json.loads accepts (depth 600 now returns 500, while main returns 200), so please make it iterative with an explicit stack, or catch RecursionError as a 400 the way #401 does. Please also run it after the size checks, or use a C-speed test such as a precompiled regex over each string, since a near-cap body currently spends about 170 ms on the event loop. |
A `\udXXX` escape with no pair is legal JSON that `json.loads` accepts, and it builds a `str`
holding a code point that cannot be encoded as UTF-8. The tokenizer then raised from inside
`build_sequence`, and `serve` maps only `ValueError` to 422, so a malformed string in the
client's own body came back as a server fault:
```
{"state":"\ud800", ...} -> HTTP 500 {"detail":"inference failed"}
{"state":"ok", "instructions":"\udfff", ...} -> HTTP 500 {"detail":"inference failed"}
{"criteria":["\ud800","b"], ...} -> HTTP 500 {"detail":"inference failed"}
# the same request with a real emoji, which is a *paired* surrogate
{"state":"hi 😀", ...} -> HTTP 200
```
The 500 also wrote a traceback into the operator's log for every such request — `TypeError:
TextEncodeInput must be Union[TextInputSequence, ...]` from the tokenizer — so a caller could
fill the log with server-looking faults by sending one escape.
The body is now rejected at parse time with a message naming the cause:
```
{"state":"\ud800", ...} -> HTTP 400
{"detail":"request body contains an unpaired surrogate escape; those cannot be encoded as UTF-8"}
```
## The two problems the review found in the first version
**It recursed.** `json.loads` accepts nesting far deeper than Python's default recursion limit,
so a recursive walk turned a body the parser handles into a `RecursionError` — one 500 replaced
by another:
```
depth=200 -> HTTP 200
depth=600 -> HTTP 500 (before) -> HTTP 200 (after)
depth=1200 -> HTTP 500 (before) -> HTTP 200 (after)
```
The walk now uses an explicit stack, so the depth the parser accepts is the depth this handles,
and a test pins 200/600/1200.
**It was slow.** The per-character Python loop cost ~170 ms of event-loop time on a near-cap
body. It is now one precompiled regex scanned at C speed:
```
per-char loop 1.6 ms per 49k-char string
compiled regex 0.1 ms per 49k-char string (~16x)
```
and it runs **after** `_check_request_limits`, so an oversized body is refused before anything
walks it and `MAX_STATE_CHARS`/`MAX_QUESTIONS` bound what the walk can reach.
Deliberate choices tested:
- **Paired surrogates are left alone.** `json.loads` combines a `\ud83d\ude00` pair into one astral character before anything here sees it, so an emoji never reaches the check and still returns 200 — asserted, because a fix that rejected surrogates by code point rather than by unpairedness would break every emoji in a state.
- **The check walks the parsed body, not the raw bytes.** Testing the raw bytes for `\ud` would reject a request that merely *mentions* the escape in text; walking the parsed value tests what the tokenizer will actually receive. Dict keys are walked as well as values, since a criteria label can be a key.
- **400, at parse time, next to the other shape checks.** The neighbouring checks in the same handler use 400 for a malformed body.
- **No `encode("utf-8", "replace")` sanitising.** Replacing the character would answer a question about text the client did not send; a caller that sent a broken string should be told so.
```
$ python -m pytest tests/test_serve.py -q 32 passed
$ python -m ruff check laya/ --select=E9,F63,F7,F82,F401,F811 --line-length=120
All checks passed!
$ python -m compileall -q laya/ tests/
```
Known limitation: only `/v1/systemone` is checked. The OpenAI-compatible routes in PR NandhaKishorM#281 parse
their own bodies, so whether the same escape reaches the tokenizer there is that PR's question
and I have not verified it.
PerryLink
force-pushed
the
fix/serve-surrogate
branch
from
September 25, 2026 15:19
6dee5db to
06462bf
Compare
4 tasks done
Owner
|
Passed checks and merged. Thank you @PerryLink! It ships in 0.3.22. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
fix(serve): a lone surrogate in the body was a 500 instead of a 400
A
\udXXXescape with no pair is legal JSON thatjson.loadsaccepts, and it builds astrholding a code point that cannot be encoded as UTF-8. The tokenizer then raised from inside
build_sequence, andservemaps onlyValueErrorto 422, so a malformed string in theclient's own body came back as a server fault:
The 500 also wrote a traceback into the operator's log for every such request —
TypeError: TextEncodeInput must be Union[TextInputSequence, ...]from the tokenizer — so a caller couldfill the log with server-looking faults by sending one escape. This is the same class as the
nested-choice-label fix: a caller error reaching an HTTP client as a server fault because the
exception type is not one
serveclassifies.A body that survives
json.loadswith an unpaired surrogate is rejected at parse time, besidethe other body-shape checks, with a message naming the cause:
Deliberate choices tested:
json.loadscombines a\ud83d\ude00pair into one astral character before anything here sees it, so an emoji never reaches the check and still returns 200 — asserted in the test, because a fix that rejected surrogates by code point rather than by unpairedness would break every emoji in a state.\udwould reject a request that merely mentions the escape in text; walking the parsed value tests what the tokenizer will actually receive. Dict keys are walked as well as values, since a criteria label can be a key._check_request_limitsand_resolve_modelso nothing downstream sees the value.encode("utf-8", "replace")sanitising. Replacing the character would answer a question about text the client did not send, which is the failure fix(serve): a request with no state was answered about the literal text null #427 just fixed one case of; a caller that sent a broken string should be told so.Known limitation: only
/v1/systemoneis checked. The OpenAI-compatible routes in the openPR #281 parse their own bodies, so whether the same escape reaches the tokenizer there is that
PR's question and I have not verified it.