Skip to content

enum with inherited type won't pickle #88508

Closed
Closed
enum with inherited type won't pickle#88508
Assignees
ethanfurman
Labels
3.10 (EOL)end of life3.11only security fixes3.8 (EOL)end of life3.9 (EOL)end of lifestdlibStandard Library Python modules in the Lib/ directorytype-bugAn unexpected behavior, bug, or error

Description

@TomBrown
TomBrown
mannequin
opened on Jun 8, 2021
BPO 44342
Nosy @ethanfurman, @miss-islington
PRs
  • bpo-44342: [Enum] change pickling from by-value to by-name #26658
  • [3.10] bpo-44342: [Enum] changed pickling from by-value to by-name (GH-26658) #26660
  • [3.9] bpo-44342: [Enum] improve pickle support #26666
  • bpo-44342: [Enum] fix data type search #26667
  • bpo-44342: [Enum] improve test, add andrei kulakov to ACKS #26726
  • [3.10] bpo-44342: [Enum] fix data type search (GH-26667) #26746
  • [3.10] bpo-44342: [Enum] improve test, add andrei kulakov to ACKS (GH-26726) #26747
  • bpo-44342: [Enum] sync current docs to 3.10 #26750
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/ethanfurman'
closed_at = <Date 2021-06-19.21:53:03.719>
created_at = <Date 2021-06-08.00:56:53.414>
labels = ['type-bug', '3.8', '3.9', '3.10', '3.11', 'library']
title = "enum with inherited type won't pickle"
updated_at = <Date 2021-06-19.21:53:03.718>
user = 'https://bugs.python.org/TomBrown'

    bugs.python.org fields:

    activity = <Date 2021-06-19.21:53:03.718>
actor = 'ethan.furman'
assignee = 'ethan.furman'
closed = True
closed_date = <Date 2021-06-19.21:53:03.719>
closer = 'ethan.furman'
components = ['Library (Lib)']
creation = <Date 2021-06-08.00:56:53.414>
creator = 'Tom.Brown'
dependencies = []
files = []
hgrepos = []
issue_num = 44342
keywords = ['patch']
message_count = 10.0
messages = ['395300', '395578', '395585', '395589', '395609', '395613', '395615', '395898', '395902', '395903']
nosy_count = 3.0
nosy_names = ['ethan.furman', 'Tom.Brown', 'miss-islington']
pr_nums = ['26658', '26660', '26666', '26667', '26726', '26746', '26747', '26750']
priority = 'high'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue44342'
versions = ['Python 3.8', 'Python 3.9', 'Python 3.10', 'Python 3.11']

    Activity

    1. TomBrown commented on Jun 8, 2021

      @TomBrown
      TomBrownmannequin
      on Jun 8, 2021
      MannequinAuthor

      The following script runs without error in 3.8.5 and raises an error in 3.8.6, 3.9.5 and 3.10.0b1.

      Source:

      import enum, pickle

class MyInt(int):
    pass
    # work-around: __reduce_ex__ = int.__reduce_ex__

class MyEnum(MyInt, enum.Enum):
    A = 1

pickle.dumps(MyEnum.A)

      Error (same in 3.8.6, 3.9.5 and 3.10.0b1):

      Traceback (most recent call last):
  File "/home/thecap/projects/covid-data-model/./enum-pickle.py", line 12, in <module>
    pickle.dumps(MyEnum.A)
  File "/home/thecap/.pyenv/versions/3.10.0b1/lib/python3.10/enum.py", line 83, in _break_on_call_reduce
    raise TypeError('%r cannot be pickled' % self)
TypeError: MyEnum.A cannot be pickled

      Like https://bugs.python.org/issue41889 this seems to be related to the fix for https://bugs.python.org/issue39587 which changes member_type from int to MyInt. A work-around is in the comment above.

    2. TomBrown
      added
      3.10 (EOL)end of life
      3.8 (EOL)end of life
      3.9 (EOL)end of life
      stdlibStandard Library Python modules in the Lib/ directory
      on Jun 8, 2021
    3. ethanfurman
      assigned
      ethanfurman
      on Jun 8, 2021
    4. ethanfurman
      added
      3.11only security fixes
      on Jun 10, 2021

    5. ethanfurman commented on Jun 10, 2021

      @ethanfurman
      ethanfurman
      on Jun 10, 2021
      Member

      Looking into this I think the root of the problem is the way reduce is handled -- currently, Enum's __reduce_ex__ works by returning the class, and the value to use to lookup the member. Because that lookup can fail with complex enums, EnumType will sabotage reduce if it can't find support in the new enum class.

      However, if __reduce_ex__ working by returning

      `getattr, (self.__class, self._name_)`

      then we should be fine, as that should never fail.

    6. ethanfurman commented on Jun 10, 2021

      @ethanfurman
      ethanfurman
      on Jun 10, 2021
      Member

      New changeset 62f1d2b by Ethan Furman in branch 'main':
      bpo-44342: [Enum] changed pickling from by-value to by-name (GH-26658)
      62f1d2b

    7. ethanfurman commented on Jun 10, 2021

      @ethanfurman
      ethanfurman
      on Jun 10, 2021
      Member

      New changeset b613132 by Miss Islington (bot) in branch '3.10':
      bpo-44342: [Enum] changed pickling from by-value to by-name (GH-26658) (GH-26660)
      b613132

    8. ethanfurman commented on Jun 11, 2021

      @ethanfurman
      ethanfurman
      on Jun 11, 2021
      Member

      Changing __reduce_ex__ is too big a change for 3.9, so making the search for pickle support more robust there.

    9. ethanfurman commented on Jun 11, 2021

      @ethanfurman
      ethanfurman
      on Jun 11, 2021
      Member

      New changeset 3a7cccf by Ethan Furman in branch 'main':
      bpo-44342: [Enum] fix data type search (GH-26667)
      3a7cccf

    10. ethanfurman commented on Jun 11, 2021

      @ethanfurman
      ethanfurman
      on Jun 11, 2021
      Member

      Also fixing the search for the data type so that in

      A -> B -> C

      the data type is C and not B.

    11. ethanfurman commented on Jun 15, 2021

      @ethanfurman
      ethanfurman
      on Jun 15, 2021
      Member

      New changeset 0f99324 by Miss Islington (bot) in branch '3.10':
      bpo-44342: [Enum] fix data type search (GH-26667)
      0f99324

    12. ethanfurman commented on Jun 16, 2021

      @ethanfurman
      ethanfurman
      on Jun 16, 2021
      Member

      New changeset 41c2a4a by Ethan Furman in branch '3.10':
      [3.10] bpo-44342: [Enum] improve test, add andrei kulakov to ACKS (GH-26726)
      41c2a4a

    13. ethanfurman commented on Jun 16, 2021

      @ethanfurman
      ethanfurman
      on Jun 16, 2021
      Member

      New changeset 741b8ae by Ethan Furman in branch 'main':
      bpo-44342: [Enum] sync current docs to 3.10 (GH-26750)
      741b8ae

    14. ethanfurman
      closed this as completedon Jun 19, 2021
    15. ethanfurman
      added
      type-bugAn unexpected behavior, bug, or error
      on Jun 19, 2021
    16. ethanfurman
      closed this as completedon Jun 19, 2021
    17. ethanfurman
      added
      type-bugAn unexpected behavior, bug, or error
      on Jun 19, 2021
    18. ezio-melotti
      transferred this issue fromon Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

    Metadata

    Metadata

    Assignees

    Labels

    3.10 (EOL)end of life3.11only security fixes3.8 (EOL)end of life3.9 (EOL)end of lifestdlibStandard Library Python modules in the Lib/ directorytype-bugAn unexpected behavior, bug, or error

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions