Repository navigation
Using the public PyUnicodeWriter C API made the json module slower #133968
Description
Activity
Could you try overoptimizing
PyUnicodeWriter_WriteUTF8and benchmarking that, to see how fast the existing function can “theoretically” be?
I'm thinking something like:- in the JSON module, force the constants (
"null"etc) to besize_t-aligned - add private (ABI-only) function
_PyUnicodeWriter_WriteUTF8_SmallAligned, which requiressize_t-aligned input and size <= sizeof(size_t) - make
PyUnicodeWriter_WriteUTF8a macro that calls_PyUnicodeWriter_WriteUTF8_SmallAlignedif the need are met, andPyUnicodeWriter_WriteUTF8otherwise
- in the JSON module, force the constants (
in the JSON module, force the constants ("null" etc) to be size_t-aligned
I don't know how to guarantee that.
add private (ABI-only) function _PyUnicodeWriter_WriteUTF8_SmallAligned, which requires size_t-aligned input and size <= sizeof(size_t)
Why do you want to limit the size to sizeof(size_t)? Which kind of optimization are you thinking of? find_first_nonascii() already takes care of the different cases with different optimizations.
ascii_decode() is already highly optimized.
ascii_decode() is already highly optimized.
Indeed it is! After playing around a bit, I could only optimize short-ASCII-strings at the expense of other strings, or a lot of code duplication.
Sorry for the noise!I don't know how to guarantee that.
For the record:
alignas; in old C it could beunion { char my_string[N]; size_t _align_me_please; }Reacted by Victor Stinner6 remaining items
I close the issue. Multiple changes optimized the
jsonmodule which should be "less slow" than 3.13 in 3.14 and main (3.15) branches now.If someone has more optimization ideas, I suggest opening a new issue.
- added 4 commits that reference this issue
on Jul 12, 2025 - added 4 commits that reference this issue
on Aug 4, 2025
I modified the json module to replace the private _PyUnicodeWriter C API with the public PyUnicodeWriter C API:
Problem: it made the json module slower. Let's investigate what's going on.
Linked PRs