Skip to content

Repository navigation

improve header parsing and performance - #3231

Merged
davidism merged 2 commits into
stablefrom
parsing
Aug 10, 2026
Merged

improve header parsing and performance#3231
davidism merged 2 commits into
stablefrom
parsing

Conversation

@davidism

@davidism davidism commented Aug 10, 2026

Copy link
Copy Markdown
Member
  • parse_options_header
    • Was doing a lot of string slicing. Replaced that with keeping track of the position in the full string and starting new matches from that position.
    • Replaced uses of str[-1] == "" with startswith and endswith, which are fast C calls that presumably don't create new strings.
    • Charset regex ensures the entire value is captured, rather than accepting up to an invalid token char.
    • Continuation charsets are tracked per key.
    • Slashs and quotes are not removed for charset values, as they are required to be token chars only. This also removes an ambiguity where unquoting could produce unexpected results if it was done before or after percent decoding.
    • A key specified without a continuation does not contribute to a continuation.
    • A continuation after a non-continuation key replaces it, and vice versa. This means the last value wins, as it would for multiple non-continuation items.
    • Continuations are accumulated and joined, rather than repeatedly concatenated. This probably doesn't matter, I'm pretty sure string concatenation is optimized in modern Python.
  • quote_etag does not create intermediate strings
  • unquote_etag does not use intermediate slicing.
  • parse_etags
    • Looks for * alone first, rather than accepting it within a list of other tags.
    • Still accepts invalid unquoted values, but requires at least one char and excludes *, ", space, and tab since those are part of valid parts or delimiters.
  • if_modified_since
    • Is always passed a valid quoted etag, and generates one when passed data.
    • if_range.etag is an unquoted value, and only ever one value, so it can be compared with == rather than in at the point it's used.
    • It currently accepts invalid unquoted etags, but in 3.2 we should ignore invalid unquoted etags rather than keep them. It's not clear why they were ever accepted, but no client should produce them.

@davidism davidism added this to the 3.1.9 milestone Aug 10, 2026
@davidism
davidism merged commit a180826 into stable Aug 10, 2026
14 checks passed
@davidism
davidism deleted the parsing branch August 10, 2026 05:18
@davidism

davidism commented Aug 10, 2026 •
edited
Loading

Copy link
Copy Markdown
Member Author

Some additional cleanup:

  • ad00b2d uses finditer instead of findall
  • c1a9de8 requires that each etag is only one etag separated by ,. The initial refactor made it possible to omit , between items, and I think even the original implementation allowed invalid prefix/suffix chars like "a"invalid, w/invalid"b"
  • 0ac9f12 stop parsing on unclosed quote

@davidism davidism mentioned this pull request Aug 10, 2026
Merged
@github-actions github-actions Bot locked as resolved and limited conversation to collaborators Aug 27, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

3.1.9

Development

Successfully merging this pull request may close these issues.

1 participant

@davidism