Cannot pickle a class with a metaclass #35791

Activity

1. mathematician commented on Dec 19, 2001

   

   mathematicianmannequin

   on Dec 19, 2001

   MannequinAuthor

   More actions

   when pickle retrieves the __reduce__ method of a new
   style class that has a metaclass, instead of
   returning the metaclass's __reduce__ method bound to
   the class, it returns an unbound __reduce__ method of
   that class.

   >>> class metaclass(type):
   ... 	def __reduce__(self):
   ... 		"""This is metaclass.__reduce__
   ... 		"""
   ... 		return type.__reduce__(self)
   ... 
   >>> class newclass(object):
   ... 	__metaclass__ = metaclass
   ... 	def __reduce__(self):
   ... 		"""This is newclass.__reduce__
   ... 		"""
   ... 		return object.__reduce__(self)
   ... 
   >>> print newclass.__reduce__.__doc__
   This is newclass.__reduce__

   when pickle calls object.__reduce__ on newclass, it
   returns an unbound newclass.__reduce__ and not a
   bound metaclass.__reduce__. This has the unfortunate
   side effect of not correctly 'reducing' the class.
   I'm trying to figure out a solution.
2. mathematician

   closed this as completedon Dec 19, 2001
3. mathematician

   assigned

   gvanrossum

   on Dec 19, 2001
4. mathematician

   added

   interpreter-core(Objects, Python, Grammar, and Parser dirs)(Objects, Python, Grammar, and Parser dirs)

   on Dec 19, 2001
5. mathematician

   closed this as completedon Dec 19, 2001
6. mathematician

   assigned

   gvanrossum

   on Dec 19, 2001
7. mathematician

   added

   interpreter-core(Objects, Python, Grammar, and Parser dirs)(Objects, Python, Grammar, and Parser dirs)

   on Dec 19, 2001
8. mathematician commented on Dec 19, 2001

   

   mathematicianmannequin

   on Dec 19, 2001

   MannequinAuthor

   More actions

   Logged In: YES
   user_id=118203

   it's a shame the tabs do not appear above...
   --- Solution

   class metaclass(type):
       def __getattribute__(self, name):
           if name in ["__reduce__", "__getstate__", 
   "__setstate__"]:
               return lambda s=self, f=getattr(type(self), 
   name): f(s)
           return type.__getattribute__(self, name)

   this fixed my bug, but it may not work for everybody. My
   suggestion is if you are to pickle a new style class, you
   should call
   type(new_style_class).__reduce__(new_style_class) instead
   of new_style_class.__reduce__()
9. tim-one commented on Dec 19, 2001

   

   tim-one

   on Dec 19, 2001

   Member

   More actions

   Logged In: YES
   user_id=31435

   Assigned to Guido.

   Dan, leading tabs vanish from the web view, but are visible
   in auto-emailed versions (it's a display issue, it's not
   that the database lost them) -- so don't worry about that.
10. gvanrossum commented on Dec 19, 2001

    

    gvanrossum

    on Dec 19, 2001

    Member

    More actions

    Logged In: YES
    user_id=6380

    Very interesting! Good analysis too (took me a while with
    pdb to come to verify it :-). But neither class needs to
    define __reduce__ -- the one they inherit from their bases,
    type and object, will do the trick just as well.

    The problem is that there are two things competing to be
    newclass.__reduce__: on the one hand, the unbound method
    __reduce__ of newclass instances; on the other hand, the
    bound method __reduce__ of newclass, seen as a metaclass
    instance. Unfortunately, the unbound method wins, but pickle
    is expecting the bound method.

    I've tried experimenting with a few ways of fixing this
    (e.g. forcing pickle to get the bound __reduce__ method) but
    there seem to be powerful forces preventing me from getting
    this to work (and I don't mean just a screaming baby :-). I
    think maybe the right solution is to make pickle realize
    that newclass is a class, and prevent it from pickling it at
    all -- it should just pickle a reference to it (i.e. the
    module and class name) rather than attempting to pickle its
    contents.

    Stay tuned.
11. gvanrossum commented on Dec 19, 2001

    

    gvanrossum

    on Dec 19, 2001

    Member

    More actions

    Logged In: YES
    user_id=6380

    I've got a patch for pickle.py; a similar patch needs to be
    developed for cPickle.c. I can't find a way to fix this
    purely by fixing the type object implementation -- pickle.py
    and cPickle.py just don't recognize classes with a custom
    metaclass as classes, because of their roots in pre-2.2
    patterns. :-(

    Should we risk fixing this before 2.2 goes out?
12. tim-one commented on Dec 19, 2001

    

    tim-one

    on Dec 19, 2001

    Member

    More actions

    Logged In: YES
    user_id=31435

    Sounds like I'd risk it, Guido: it's not like you could
    break 2.1's pickle behavior for new-style classes with a
    custom metaclass <wink>.
13. gvanrossum commented on Dec 19, 2001

    

    gvanrossum

    on Dec 19, 2001

    Member

    More actions

    Logged In: YES
    user_id=6380

    Thanks for reporting! Fixed in CVS, with a simpler version
    of the patch below; also for cPickle. Adding tests too...
14. ezio-melotti

    transferred this issue fromon Apr 9, 2022