marshal.dumps() crashes when a set is modified while it is being marshalled

Crash report

To sort the elements of a set, w_complex_object() reads the size of the
set, allocates a list of that size, then marshals each element on its own
in a loop. Marshalling an element can run Python code (an element's
__buffer__, an audit hook), and that code can change the set. If the set
grows, the loop writes past the end of the list. If it shrinks, the list
is left with NULL items and is then sorted.

import marshal, sys

class Item:
    def __hash__(self):
        return 0
    def __buffer__(self, flags):
        if sys.argv[1] == 'grow':
            s.update(range(1000, 1400))
        else:
            s.difference_update(range(1, 60))
        return memoryview(b'abc')

s = {Item()}
s.update(range(1, 60))
marshal.dumps(s)

On main:

shrink, release build: segmentation fault.
grow, release build: no error; writes past the list allocation.
grow, debug build:
PyList_SET_ITEM: Assertion 'index < list->allocated' failed.

On the free-threaded build another thread changing the set between the size
read and the critical section has the same effect.

Proposed fix

Stop the loop at the size that was read, and raise
RuntimeError("set changed size during iteration") if the number of
elements seen differs from it, as iterating a set does.

Discovered using Claude while doing a deep dive on pyc compilation determinism.

CPython versions tested on:

CPython main branch

Operating systems tested on:

No response

Output from running 'python -VV' on the command line:

No response

Linked PRs

gh-159185: Fix marshalling of a set modified while its elements are m…

Automated backports failed because main has gh-156942, which is not on 3.15 or 3.14. On main the marshal writer raises the exception where the error is detected. On the older branches it stores a WFERR_* code and turns it into an exception at the end, so an exception set inside the loop would be overwritten with ValueError("unmarshallable object").

The backport therefore follows the branch's own pattern: a new WFERR_SET_CHANGED_SIZE code, set where main raises directly, and a new case in the final switch that raises RuntimeError("set changed size during iteration"). The in-loop check and the test are the same as on main. The marshal.c change is identical on both branches.

One behaviour to know about

PyMarshal_WriteObjectToFile() still raises nothing on these branches. 😦 It ignores every marshalling error there; reporting them is gh-155907, which is not backported. With the fix it no longer crashes on a mutated set, but it returns silently after writing a truncated 5-byte file. marshal.dumps(), marshal.dump() and PyMarshal_WriteObjectToString() do raise. The NEWS entry only names dumps and dump, so it is still accurate.

... keeping them draft while considering if the above should be backported first so it is a happier situation. [edit: Verdict, not backporting those].

gh-159185: Document that PyMarshal_WriteObjectToFile can silently tru…

[3.14] gh-159185: Fix marshalling of a set modified while its element…

[3.15] gh-159185: Fix marshalling of a set modified while its element…

[3.15] gh-159185: Document that PyMarshal_WriteObjectToFile can silen…

the doc note I added isn't anything I'd call pretty but this shouldn't be a widely used API and it at least reflects the pre-existing reality of its deficiencies.

[3.14] gh-159185: Document that PyMarshal_WriteObjectToFile can silen…

Navigation Menu

`marshal.dumps()` crashes when a set is modified while it is being marshalled #159185

Crash report

Proposed fix

CPython versions tested on:

Operating systems tested on:

Output from running 'python -VV' on the command line:

Linked PRs

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

marshal.dumps() crashes when a set is modified while it is being marshalled #159185

Description

Crash report

Proposed fix

CPython versions tested on:

Operating systems tested on:

Output from running 'python -VV' on the command line:

Linked PRs

Activity

gpshead commented on Oct 11, 2026

gpshead commented on Oct 11, 2026

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions

`marshal.dumps()` crashes when a set is modified while it is being marshalled #159185