maximum recursion depth in deepcopy regression

@serhiy-storchaka

Bug report

Bug description:

The following code works on py313 and below (this is a very cut-down version of some code in Matplotilb (matplotlib.path.Path)) but hit the maximum recursion depth on main.

# Add a code block here, if required
import copy

class Test:
    def __init__(self, a, b):
        self.a = a
        self.b = a
        self._state = True

    def __deepcopy__(self, memo=None):
        p = copy.deepcopy(super(), memo)
        p._state = False
        return p

t = Test(1, 2)
t2 = copy.deepcopy(t)

I bisected this to #125781 and suspect the problem is that because it is returning a new object it is escaping the memoization and looping.

I'm happy to be told we should not be doing this fix it on the Matplotlib side, but I would like to be sure that this was an expected consequence of the change.

attn @serhiy-storchaka

CPython versions tested on:

3.14, CPython main branch

Operating systems tested on:

Linux

Interesting. I did not foresee such use case.

In general, such code worked by accident, because the code in the copy module is more permissive than in the pickle module. Although it does not work if the reducing function returns a string (enums or other singletons). It does not work if copying in the parent class was implemented by the __copy__ and __deepcopy__ methods.

The question is how to support that use case -- copying an object ignoring the special methods in the current class. Should we use super() for this? How to resolve conflict with different semantic for pickling? 🤔

sounds like we should find another way downstream no matter which way CPython goes on this question?

Another example from Matplotlib that I suspect is coming from the same cause (but have not actually bisected) is

https://github.com/matplotlib/matplotlib/blob/f2717a5486f782baca9fedd5fdb28f73e6c366d0/lib/matplotlib/transforms.py#L143-L153

    def __copy__(self):
        other = copy.copy(super())
        # If `c = a + b; a1 = copy(a)`, then modifications to `a1` do not
        # propagate back to `c`, i.e. we need to clear the parents of `a1`.
        other._parents = {}
        # If `c = a + b; c1 = copy(c)`, then modifications to `a` also need to
        # be propagated to `c1`.
        for key, val in vars(self).items():
            if isinstance(val, TransformNode) and id(self) in val._parents:
                other.set_children(val)  # val == getattr(other, key)
        return other

That is now failing with

E       AttributeError: 'super' object has no attribute '_parents' and no __dict__ for setting new attributes

It seems right to be to be able to access "copy/deepcopy this as if the object were its parent class" via copy.deepcopy(super()) because for all other methods that is what super().foo() does. A lazy (but apparently wrong) way to think about copy/deepcopy is "please invoke __copy__ / __deepcopy__ for me".

I'm indifferent on if pickling super objects should work or not (I think a compelling case could be made for pickle simply refusing to tray), but feel pretty strongly that copy.deepcopy(super()) when used in __deepcopy__ should return a copy of the calling object as if it were the next class in the mro. Otherwise it is not clear how we should be extending a base classes copying implementation.