Skip to content

native _json, _pickle, pyexpat, and _elementtree #8741

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
youknowone merged 21 commits into from
Sep 19, 2026
Merged

native _json, _pickle, pyexpat, and _elementtree #8741

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
d76be23
_json: add native make_encoder (c_make_encoder)
moreal Sep 7, 2026
4b4bf12
pyexpat: add UseForeignDTD, fix silently swallowed handler exceptions
moreal Sep 8, 2026
58e7dfe
pyexpat: support multi-chunk Parse()/ParseFile() feeds
moreal Sep 8, 2026
4592d32
test: drop stale expectedFailure markers fixed by the pyexpat changes
moreal Sep 8, 2026
a59da40
pyexpat: drive a single background EventReader for true incremental P…
moreal Sep 8, 2026
38795df
pyexpat: add a Sync fallback backend for targets without threads
moreal Sep 8, 2026
5f9c330
_pickle: native Pickler, Unpickler and PickleBuffer
moreal Sep 8, 2026
cfeaf81
_pickle: raise the pickle.find_class audit event
moreal Sep 8, 2026
b5fb603
_pickle: avoid temporary buffers on the save fast paths
moreal Sep 8, 2026
debdcff
pyexpat: make the threaded backend fork-safe
moreal Sep 8, 2026
95fc653
pyexpat: implement name interning via self.intern, matching libexpat
moreal Sep 8, 2026
152dd3f
_elementtree: native Element, SubElement and ParseError
moreal Sep 8, 2026
37e5e0b
_elementtree: native TreeBuilder and XMLParser
moreal Sep 8, 2026
04aee9e
pyexpat: stop rescanning the whole line per position query
moreal Sep 8, 2026
81dc2df
_elementtree: charge Element.__deepcopy__ to the recursion budget
moreal Sep 8, 2026
1e905ad
pyexpat: don't let a stale NeedMore end a Parse() early
moreal Sep 8, 2026
843e1e4
Restore xml.etree test markers instead of skipTest in bodies
youknowone Sep 19, 2026
6ee458f
Clone state before try_to_bool in pickle load_build
youknowone Sep 19, 2026
a440cd4
Drop decorative section-separator comments
youknowone Sep 19, 2026
90d1611
Represent pickle opcodes as an enum
youknowone Sep 19, 2026
40095e5
Use full paths for thread-only types in pyexpat
youknowone Sep 19, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
_elementtree: charge Element.__deepcopy__ to the recursion budget 
A chain of elements is copied by native recursion, so a deep enough tree
overflowed the real stack instead of raising. Wrap the copy in
with_recursion, as CPython does, and drop the skip on
test_deeply_nested_deepcopy for the accelerated run.

Assisted-by: Claude Code:claude-fable-5-1
  • Loading branch information
@moreal @youknowone
moreal authored and youknowone committed Sep 19, 2026
commit 81dc2dff3ae95073275ea5cfeb8843e8514c4349
6 changes: 5 additions & 1 deletion Lib/test/test_xml_etree.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3153,13 +3153,17 @@ def __deepcopy__(self, memo):
self.assertEqual([c.tag for c in children[3:]],
[a.tag, b.tag, a.tag, b.tag])

@unittest.skip("TODO: RUSTPYTHON; stack overflow")
@support.skip_if_unlimited_stack_size
@support.skip_emscripten_stack_overflow()
@support.skip_wasi_stack_overflow()
def test_deeply_nested_deepcopy(self):
# This should raise a RecursionError and not crash.
# See https://github.com/python/cpython/issues/148801.
if is_python_implementation():
# TODO: RUSTPYTHON; only the _elementtree accelerator charges the
# copy to the recursion budget, so the pure-Python one still
# overflows the stack.
self.skipTest("TODO: RUSTPYTHON")
root = cur = ET.Element('s')
for _ in range(500_000):
cur = ET.SubElement(cur, 'u')
Expand Down
13 changes: 13 additions & 0 deletions crates/stdlib/src/elementtree.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1043,6 +1043,19 @@ pub(crate) mod _elementtree {
}

fn deepcopy_element(zelf: &Py<PyElement>, memo: &Py<PyDict>, vm: &VirtualMachine) -> PyResult {
// A chain of elements is copied by native recursion, so the depth
// has to be charged to the interpreter's recursion budget or a deep
// enough tree overflows the real stack instead of raising.
vm.with_recursion(" in Element.__deepcopy__", || {
deepcopy_element_inner(zelf, memo, vm)
})
}

fn deepcopy_element_inner(
zelf: &Py<PyElement>,
memo: &Py<PyDict>,
vm: &VirtualMachine,
) -> PyResult {
let (tag, attrib, text, text_pending, tail, tail_pending, children) = {
let inner = zelf.inner.read();
(
Expand Down