gh-159185: Fix marshalling of a set modified while its elements are marshalled - #159195

To order the elements of a set, marshal marshals each element on its own. That can run Python code (an element's __buffer__(), an audit hook), and that code can change the set. The number of elements was read once, before the loop.

class Item:
    def __hash__(self):
        return 0
    def __buffer__(self, flags):
        s.update(range(1000, 1400))     # or: s.difference_update(range(1, 60))
        return memoryview(b'abc')

s = {Item()}
s.update(range(1, 60))
marshal.dumps(s)

With this PR both raise RuntimeError: set changed size during iteration, as iterating the set would.

Change

flowchart TB
  subgraph before["main"]
    direction TB
    b1["n = size of the set"] --> b2["list of n slots"]
    b2 --> b3["for each element:<br>marshal it (may run Python code)<br>store in slot i++"]
    b3 -- "set grew" --> b4["writes past slot n"]
    b3 -- "set shrank" --> b5["slots left NULL,<br>then sorted"]
  end
  subgraph after["this PR"]
    direction TB
    a1["n = size of the set"] --> a2["list of n slots"]
    a2 --> a3["for each element:<br>stop if i == n<br>marshal it, store in slot i++"]
    a3 --> a4{"stopped early,<br>or i != n?"}
    a4 -- yes --> a5["RuntimeError"]
    a4 -- no --> a6["sort and write"]
  end

The change is two checks around the existing loop in w_complex_object(). On the free-threaded build the same checks cover another thread changing the set between the size being read and the critical section being entered. A set that is changed without changing its size is not detected; that is memory safe, and the output is whatever was iterated.

This touches the same block as the PR for gh-159184; whichever is merged second needs a one-line conflict resolved.

Tests

test_set_changed_size: grow and shrink, all marshal versions.

• Issue: marshal.dumps() crashes when a set is modified while it is being marshalled #159185