Skip to content

Crash when importing an extension after Py_Initialize, Py_Finalize and Py_Initialize #53443

Closed
Closed
Crash when importing an extension after Py_Initialize, Py_Finalize and Py_Initialize#53443
Assignees
abalkin
Labels
OS-mactype-crashA hard crash of the interpreter, possibly with a core dump

Description

@mpalluat
mpalluat
mannequin
opened on Jul 8, 2010
BPO 9197
Nosy @loewis, @ronaldoussoren, @amauryfa, @ncoghlan, @abalkin
Superseder
  • bpo-11321: 9th import of module _pickle always crashes
    		•
    Files
  • OSDP2.zip: Mac project that highlights the crash
  • simpletest.zip
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/abalkin'
closed_at = <Date 2011-02-28.01:15:21.385>
created_at = <Date 2010-07-08.08:50:50.638>
labels = ['OS-mac', 'type-crash']
title = 'Crash when importing an extension after Py_Initialize, Py_Finalize and Py_Initialize'
updated_at = <Date 2011-02-28.01:15:21.383>
user = 'https://bugs.python.org/mpalluat'

    bugs.python.org fields:

    activity = <Date 2011-02-28.01:15:21.383>
actor = 'ncoghlan'
assignee = 'belopolsky'
closed = True
closed_date = <Date 2011-02-28.01:15:21.385>
closer = 'ncoghlan'
components = ['macOS']
creation = <Date 2010-07-08.08:50:50.638>
creator = 'mpalluat'
dependencies = []
files = ['17898', '20631']
hgrepos = []
issue_num = 9197
keywords = []
message_count = 12.0
messages = ['109521', '109524', '109525', '109537', '109539', '109540', '109974', '111452', '127634', '127642', '129669', '129676']
nosy_count = 6.0
nosy_names = ['loewis', 'ronaldoussoren', 'amaury.forgeotdarc', 'ncoghlan', 'belopolsky', 'mpalluat']
pr_nums = []
priority = 'normal'
resolution = 'duplicate'
stage = None
status = 'closed'
superseder = '11321'
type = 'crash'
url = 'https://bugs.python.org/issue9197'
versions = ['Python 3.1']

    Activity

    1. mpalluat
      assigned
      ronaldoussoren
      on Jul 8, 2010
    2. mpalluat
      added
      OS-mac
      type-crashA hard crash of the interpreter, possibly with a core dump
      on Jul 8, 2010

    3. mpalluat commented on Jul 8, 2010

      @mpalluat
      mpalluatmannequin
      on Jul 8, 2010
      MannequinAuthor

      there is a crash log inside the archive

    4. ronaldoussoren commented on Jul 8, 2010

      @ronaldoussoren
      ronaldoussoren
      on Jul 8, 2010
      Contributor

      The title is somewhat misleading, this is a C program that uses dlopen to load the python framework.

      I will look into this, but it is just as likely that code that loads or uses the python framework has a problem.

    5. mpalluat commented on Jul 8, 2010

      @mpalluat
      mpalluatmannequin
      on Jul 8, 2010
      MannequinAuthor

      Hi Ronald,

      Thank you for looking into it, and sorry for the misleading title.

      You will find a crash log and some instructions on how to reproduce the problem inside the archive.

      Thanks,

      Marc

      On 8 Jul 2010, at 10:18, Ronald Oussoren wrote:

      Ronald Oussoren <ronaldoussoren@mac.com<mailto:ronaldoussoren@mac.com>> added the comment:

      The title is somewhat misleading, this is a C program that uses dlopen to load the python framework.

      I will look into this, but it is just as likely that code that loads or uses the python framework has a problem.

      ----------

      Python tracker <report@bugs.python.org<mailto:report@bugs.python.org>>
      <http://bugs.python.org/issue9197\>

      Marc Palluat de Besset
      Mac Software Engineer

      email : marc.palluatdebesset@sophos.com<mailto:marc.palluatdebesset@sophos.com>
      tel : +44 1235 544056
      loc : E2 floor 2

      SOPHOS - simply secure

      Sophos Plc, The Pentagon, Abingdon Science Park, Abingdon, OX14 3YP, United Kingdom.
      Company Reg No 2096520. VAT Reg No GB 348 3873 20.

    6. amauryfa commented on Jul 8, 2010

      @amauryfa
      amauryfa
      on Jul 8, 2010
      Contributor

      I reproduce the problem on Linux (./configure --enable-shared), after I modified the source code a bit to directly use Python.h and to link with libpython3.2.so (no call to dlopen). In gdb the stack trace has exactly the same symbols as the attached crash log, so the issue is not specific to Mac frameworks.

      The crash is inside _PyImport_LoadDynamicModule("_pickle"), when _PyImport_FixupExtension() calls Py_DECREF(def->m_base.m_copy), this certainly frees objects allocated in a previous incarnation of the interpreter, and segfaults in type_dealloc(): _PyObject_GC_UNTRACK(type).

      In import.c::_PyImport_FixupExtension(), the Py_DECREF is preceded by a comment: """ /* Somebody already imported the module, 
   likely under a different name.
   XXX this should really not happen. */
Py_DECREF(def->m_base.m_copy);
"""
I removed this statement, and the program now runs correctly. I don't know about memory leaks though...

    7. ronaldoussoren commented on Jul 8, 2010

      @ronaldoussoren
      ronaldoussoren
      on Jul 8, 2010
      Contributor

      Removing the call to Py_DECREF is fishy.

      The cleanup of the interpreter state in Py_Finalize doesn't clean up all state when m_copy refers to a version from a previous instance of the interpreter.

      Maybe the tp_dealloc of a module object clean up m_copy? I don't know enough of this code and its intended behavior to know if that would be a valid change.

    8. amauryfa commented on Jul 8, 2010

      @amauryfa
      amauryfa
      on Jul 8, 2010
      Contributor

      We definitely need unit tests about embedded python interpreter, I think there are none.

    9. ronaldoussoren commented on Jul 11, 2010

      @ronaldoussoren
      ronaldoussoren
      on Jul 11, 2010
      Contributor

      Unassigning, this is not mac-specific and I don't plan to work on this in the near future.

    10. ronaldoussoren
      removed their assignment
      on Jul 11, 2010

    11. ronaldoussoren commented on Jul 24, 2010

      @ronaldoussoren
      ronaldoussoren
      on Jul 24, 2010
      Contributor

      (changing subject to better tell what this issue is about)

    12. ronaldoussoren
      changed the title [-]subprocess module causing crash[/-] [+]Crash when importer an extension after Py_Initialize, Py_Finalize and Py_Initialize[/+] on Jul 24, 2010

    13. abalkin commented on Jan 31, 2011

      @abalkin
      abalkin
      on Jan 31, 2011
      Member

      Amaury,

      Can you post your Linux code reproducing the issue? I would like to take a look, but don't want to dig through an XCode project.

    14. abalkin
      changed the title [-]Crash when importer an extension after Py_Initialize, Py_Finalize and Py_Initialize[/-] [+]Crash when importing an extension after Py_Initialize, Py_Finalize and Py_Initialize[/+] on Jan 31, 2011
    15. abalkin
      self-assigned this
      on Jan 31, 2011

    16. amauryfa commented on Jan 31, 2011

      @amauryfa
      amauryfa
      on Jan 31, 2011
      Contributor

      Here are the files I used to reproduce the crash.
      See "build.sh" for the build command, there may be better ways...

    17. amauryfa commented on Feb 27, 2011

      @amauryfa
      amauryfa
      on Feb 27, 2011
      Contributor

      Does the patch for bpo-11321 fix this issue?

    18. ncoghlan commented on Feb 28, 2011

      @ncoghlan
      ncoghlan
      on Feb 28, 2011
      Contributor

      Merging this into bpo-11321 since that relates to the same underlying problem and includes a simpler test case.

    19. ncoghlan
      closed this as completedon Feb 28, 2011
    20. ezio-melotti
      transferred this issue fromon Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

    Metadata

    Metadata

    Assignees

    Labels

    OS-mactype-crashA hard crash of the interpreter, possibly with a core dump

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions