Repository navigation
http.client accepts Content-Length and chunk-size values that RFC 9112 forbids #150751
Description
http.client derives the response body framing from int() of the Content-Length header and the chunked chunk-size line:
HTTPResponse.begin:self.length = int(length)HTTPResponse._read_next_chunk_size:return int(line, 16)
RFC 9112 defines Content-Length = 1*DIGIT and chunk-size = 1*HEXDIG, but int() is more permissive: it accepts a leading +/-, underscores, surrounding whitespace and, in base 16, an 0x prefix and non-ASCII digits. So values like Content-Length: +5 / 5_0 and chunk sizes -5, +5, 0x5, 1_f are accepted and used to frame the body, while an RFC-compliant front end would reject them or frame the message differently (CWE-444).
Reproducer:
import http.client, io
class S:
def __init__(s, d): s.f = io.BytesIO(d)
def makefile(s, *a, **k): return s.f
def parse(raw):
r = http.client.HTTPResponse(S(raw)); r.begin(); return r
raw = b'HTTP/1.1 200 OK\r\nTransfer-Encoding: chunked\r\n\r\n+5\r\nHELLO\r\n0\r\n\r\n'
print(parse(raw).read()) # b'HELLO' -- '+5' is not a HEXDIG
raw = b'HTTP/1.1 200 OK\r\nContent-Length: 5_0\r\n\r\n' + b'A'*50
print(parse(raw).length) # 50 -- '5_0' is not 1*DIGITThe body-framing tokens should be validated against the grammar before being passed to int().
Linked PRs
- gh-150751: validate http.client Content-Length and chunk-size #150752
- [3.14] gh-150751: validate http.client Content-Length and chunk-size (GH-150752) #159020
- [3.15] gh-150751: validate http.client Content-Length and chunk-size (GH-150752) #159021
- gh-150751: Ignore optional whitespace around Content-Length in http.client #159156
Activity
devdanzin commentedon Aug 4, 2026
Three things that may be useful here.
1. There is an older issue covering part of this. gh-69926 ("multiple issues in http.client", open since 2015) has "negative chunk size accepted" as its first bullet. Worth linking so the two don't get fixed independently.
2. The negative chunk size has a concrete consequence beyond framing divergence, which may raise its priority.
_read_next_chunk_sizereturnsint(line, 16), andint(b"-1", 16)is-1with noValueError. That value then flows into_safe_read, where the pre-allocation clamp added by gh-119451 ismin(amt, _MIN_READ_BUF_SIZE)— andmin(-1, 1 << 20) == -1. So a negative chunk size disarms that clamp by sign: the read is no longer bounded by the buffer size. In_readinto_chunked,mvb[:chunk_left]likewise becomesmvb[:-1].The guarded twin is in the same file:
Content-Lengthhas carriedif self.length < 0: # ignore nonsensical negative lengthsatHTTPResponse.beginsince 2008. The chunk path never got the same check.3. A defect in the current PR. In gh-150752,
_read_next_chunk_sizebecomes:The
strip()is needed to remove the line terminator, but barebytes.strip()removesSP TAB LF CR VT FF— so it runs before the grammar check and re-admits the surrounding whitespace the PR's own comment says it is rejecting:Stripping only the terminator (
line.rstrip(b'\r\n'), or slicing it off) would keep the grammar check meaningful for whitespace as well as for sign and underscores.Written with AI assistance; the clamp arithmetic and the strip table above were run, and the PR diff was read.
encukou commentedon Oct 6, 2026
Thanks for the link!
gh-69926 lists several issues; solving one of them here is fine.
IMO, the divergence is worse.
Beware, we're stripping chunk-extensions, which might include space before the semicolon. We should strip space and tab in that case. (Or parse & validate chunk-ext, but I don't think that's necessary.
I don't see a harm in keeping the
strip()-- other parsers could reject extra whitespace but they shouldn't read a different value.gh-150751: validate http.client Content-Length and chunk-size (GH-150752
danilablagorodniy37 commentedon Oct 10, 2026
gh-150752 introduced a regression on
main: the compat32 header parser used byhttp.clientkeeps trailing SP/HTAB in a field value, soContent-Length: 5now fails the1*DIGITcheck and is treated as a missing header.HTTPResponse.read()then blocks until the server closes the connection, which never happens on a keep-alive connection (3.15.0 returns the 5-byte body). RFC 9112 §5.1 requires parsers to exclude the OWS around a field value, and Go, llhttp, aiohttp and h11 all frame such a message with the parsed digits.Fix in #159156. The open backports gh-159020 and gh-159021 carry the same regression.
gh-150751: Ignore optional whitespace around Content-Length in http.c…