Skip to content

Repeated Py_Initialize and Py_Finalize usage brings fatal error? #54037

Closed
Closed
Repeated Py_Initialize and Py_Finalize usage brings fatal error?#54037
Assignees
pitrou
Labels
interpreter-core(Objects, Python, Grammar, and Parser dirs)

Description

@ocean-city
ocean-city
mannequin
opened on Sep 11, 2010
BPO 9828
Nosy @pitrou
Files
  • gildestroy.patch
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/pitrou'
closed_at = <Date 2010-09-13.14:17:26.853>
created_at = <Date 2010-09-11.04:37:45.040>
labels = ['interpreter-core']
title = 'Repeated Py_Initialize and Py_Finalize usage brings fatal error?'
updated_at = <Date 2010-09-13.14:17:26.851>
user = 'https://bugs.python.org/ocean-city'

    bugs.python.org fields:

    activity = <Date 2010-09-13.14:17:26.851>
actor = 'pitrou'
assignee = 'pitrou'
closed = True
closed_date = <Date 2010-09-13.14:17:26.853>
closer = 'pitrou'
components = ['Interpreter Core']
creation = <Date 2010-09-11.04:37:45.040>
creator = 'ocean-city'
dependencies = []
files = ['18861']
hgrepos = []
issue_num = 9828
keywords = ['patch']
message_count = 8.0
messages = ['116075', '116079', '116174', '116194', '116208', '116217', '116241', '116308']
nosy_count = 2.0
nosy_names = ['pitrou', 'ocean-city']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue9828'
versions = ['Python 3.2']

    Activity

    1. ocean-city commented on Sep 11, 2010

      @ocean-city
      ocean-citymannequin
      on Sep 11, 2010
      MannequinAuthor

      When I ran the code in bpo-6869 (msg95444) on py3k, I noticed
      Fatal Python error occurs.

      # Actually, I need to copy created executable file into
      # PC/VC6 directoly where python32_d.dll exists. Otherwise,
      # the error "encoding.utf-8 was not found" occured even if
      # PC/VC6 was added to environment variable PATH. But probably
      # it is another issue.

      E:\python-dev\py3k\PC\VC6>foo
      --- 0 ---
      [34859 refs]
      --- 1 ---
      Fatal Python error: drop_gil: wrong thread state

      ... And here is stacktrace.

      _NMSG_WRITE(int 10) line 221
      abort() line 44 + 7 bytes
      Py_FatalError(const char * 0x1e371878) line 2109
      drop_gil(_ts * 0x00920d90) line 321 + 10 bytes
      PyEval_SaveThread() line 424 + 6 bytes
      0012ff30()
      PyCFunction_Call(_object * 0x00b99378, _object * 0x00afcc40, _object * 0x00000000) line 89 + 9 bytes
      PyObject_Call(_object * 0x00b99378, _object * 0x00afcc40, _object * 0x00000000) line 2149 + 15 bytes
      call_function_tail(_object * 0x00b99378, _object * 0x00afcc40) line 2181 + 15 bytes
      _PyObject_CallMethod_SizeT(_object * 0x00bf6838, char * 0x1e332898, char * 0x00000000) line 2297 + 13 bytes
      io_open(_object * 0x00c17f78, _object * 0x00ae0f88, _object * 0x00000000) line 423 + 16 bytes
      PyCFunction_Call(_object * 0x00b0cab8, _object * 0x00ae0f88, _object * 0x00000000) line 84 + 15 bytes
      PyObject_Call(_object * 0x00b0cab8, _object * 0x00ae0f88, _object * 0x00000000) line 2149 + 15 bytes
      call_function_tail(_object * 0x00b0cab8, _object * 0x00ae0f88) line 2181 + 15 bytes
      PyObject_CallMethod(_object * 0x00bf6138, char * 0x1e3d6abc, char * 0x1e3d6ab4) line 2258 + 13 bytes
      create_stdio(_object * 0x00bf6138, int 0, int 0, char * 0x1e3d6968, char * 0x00000000, char * 0x00000000) line 808 + 48 bytes
      initstdio() line 931 + 28 bytes
      Py_InitializeEx(int 1) line 303 + 5 bytes
      Py_Initialize() line 314 + 7 bytes
      main() line 10 + 8 bytes
      mainCRTStartup() line 206 + 25 bytes
      KERNEL32! 77e789d5()

      Because first run succeeded, Py_Finalize might not do enough
      work to reset thread state, I guess.

      I didn't try this on other platforms. (thread_nt.h was used here)

    2. ocean-city
      added
      interpreter-core(Objects, Python, Grammar, and Parser dirs)
      on Sep 11, 2010

    3. ocean-city commented on Sep 11, 2010

      @ocean-city
      ocean-citymannequin
      on Sep 11, 2010
      MannequinAuthor

      Sorry, msg92444

    4. birkenfeld
      assigned
      pitrou
      on Sep 12, 2010

    5. pitrou commented on Sep 12, 2010

      @pitrou
      pitrou
      on Sep 12, 2010
      Member

      I can't reproduce under Linux. Is 3.1 also affected?
      Can you try the following patch:

      Index: Python/pythonrun.c
      ===================================================================

      --- Python/pythonrun.c	(révision 84718)
+++ Python/pythonrun.c	(copie de travail)
@@ -219,6 +219,8 @@
 
     /* auto-thread-state API, if available */
 #ifdef WITH_THREAD
+    if (PyEval_ThreadsInitialized())
+        PyEval_ReInitThreads();
     _PyGILState_Init(interp, tstate);
 #endif /* WITH_THREAD */

    6. pitrou commented on Sep 12, 2010

      @pitrou
      pitrou
      on Sep 12, 2010
      Member

      It should be noted that repeatedly calling Py_Initialize / Py_Finalize leaks a lot (even without the "import _ctypes"):

      --- 0 ---
      [34543 refs]
      --- 1 ---
      [37540 refs]
      --- 2 ---
      [40504 refs]
      --- 3 ---
      [43468 refs]
      --- 4 ---
      [46432 refs]
      --- 5 ---
      [49396 refs]
      --- 6 ---
      [52360 refs]
      --- 7 ---
      [55324 refs]
      --- 8 ---
      [58288 refs]
      --- 9 ---
      [61252 refs]
      --- 10 ---
      [64216 refs]

      And it also translates into memory consumption...

    7. ocean-city commented on Sep 12, 2010

      @ocean-city
      ocean-citymannequin
      on Sep 12, 2010
      MannequinAuthor

      Is 3.1 also affected?
      Sorry, 3.1 is not affected.

      Can you try the following patch:
      Your patch works. :-)

      E:\python-dev\py3k\PC\VC6>ctypes_crash.exe
      --- 0 ---
      [36242 refs]
      --- 1 ---
      [39140 refs]
      --- 2 ---
      [41538 refs]
      --- 3 ---
      [43930 refs]
      --- 4 ---
      [46322 refs]
      --- 5 ---
      [48714 refs]
      --- 6 ---
      [51106 refs]
      --- 7 ---
      [53498 refs]
      --- 8 ---
      [55890 refs]
      --- 9 ---
      [58282 refs]

    8. pitrou commented on Sep 12, 2010

      @pitrou
      pitrou
      on Sep 12, 2010
      Member

      Ok. How about this patch now?
      (I'm trying to solve this in a clean way :))

    9. ocean-city commented on Sep 12, 2010

      @ocean-city
      ocean-citymannequin
      on Sep 12, 2010
      MannequinAuthor

      gildestroy.patch also works.

      E:\PYTHON~1\py3k\PC\VC6>ctypes_crash.exe
      --- 0 ---
      [36303 refs]
      --- 1 ---
      [39184 refs]
      --- 2 ---
      [41582 refs]
      --- 3 ---
      [43974 refs]
      --- 4 ---
      [46366 refs]
      --- 5 ---
      [48758 refs]
      --- 6 ---
      [51150 refs]
      --- 7 ---
      [53542 refs]
      --- 8 ---
      [55934 refs]
      --- 9 ---
      [58326 refs]

    10. pitrou commented on Sep 13, 2010

      @pitrou
      pitrou
      on Sep 13, 2010
      Member

      Fix committed in r84774. Thank you!

    11. pitrou
      closed this as completedon Sep 13, 2010
    12. ezio-melotti
      transferred this issue fromon Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

    Metadata

    Metadata

    Assignees

    Labels

    interpreter-core(Objects, Python, Grammar, and Parser dirs)

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions