Skip to content

Subinterpreters hang in GIL adquisition if an extension module calls PyGILState_Ensure. #90453

Description

@pablogsal
BPO 46295
Nosy @vstinner, @ericsnowcurrently, @pablogsal
Superseder
  • bpo-15751: [subinterpreters] Make the PyGILState API compatible with subinterpreters
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2022-01-07.17:41:11.910>
    created_at = <Date 2022-01-07.15:18:15.098>
    labels = ['3.9', '3.10', '3.11']
    title = 'Subinterpreters hang in GIL adquisition if an extension module calls PyGILState_Ensure.'
    updated_at = <Date 2022-01-07.17:41:32.956>
    user = 'https://github.com/pablogsal'

    bugs.python.org fields:

    activity = <Date 2022-01-07.17:41:32.956>
    actor = 'pablogsal'
    assignee = 'none'
    closed = True
    closed_date = <Date 2022-01-07.17:41:11.910>
    closer = 'pablogsal'
    components = []
    creation = <Date 2022-01-07.15:18:15.098>
    creator = 'pablogsal'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 46295
    keywords = []
    message_count = 3.0
    messages = ['409975', '409987', '409990']
    nosy_count = 3.0
    nosy_names = ['vstinner', 'eric.snow', 'pablogsal']
    pr_nums = []
    priority = 'normal'
    resolution = 'duplicate'
    stage = 'resolved'
    status = 'closed'
    superseder = '15751'
    type = None
    url = 'https://bugs.python.org/issue46295'
    versions = ['Python 3.9', 'Python 3.10', 'Python 3.11']

    Activity

    1. pablogsal commented on Jan 7, 2022

      @pablogsal
      MemberAuthor

      Reproducer:

      Code for native_ext.cpp:

      #define PY_SSIZE_T_CLEAN
      #include <Python.h>
      
      #include <assert.h>
      #include <pthread.h>
      #include <malloc.h>

      #pragma GCC push_options
      #pragma GCC optimize ("O0")

      PyObject*
      run_simple(PyObject*, PyObject*)
      {
      	PyGILState_STATE gstate;
      	gstate = PyGILState_Ensure();
      	PyGILState_Release(gstate);
          Py_RETURN_NONE;
      }

      static PyMethodDef methods[] = {
      {"run_simple", run_simple, METH_NOARGS, "Execute a chain of native functions"},
      {NULL, NULL, 0, NULL},
      };

      #if PY_MAJOR_VERSION >= 3
      static struct PyModuleDef moduledef = {PyModuleDef_HEAD_INIT, "native_ext", "", -1, methods};
      
      PyMODINIT_FUNC
      PyInit_native_ext(void)
      {
          return PyModule_Create(&moduledef);
      }
      #else
      PyMODINIT_FUNC
      initnative_ext(void)
      {
          Py_InitModule("native_ext", methods);
      }
      #endif

      Code for setup.py:

      import os
      from distutils.core import Extension
      from distutils.core import setup
      
      ROOT = os.path.realpath(os.path.dirname(__file__))

      setup(
      name="native_ext",
      version="0.0",
      ext_modules=[
      Extension(
      "native_ext",
      language="c++",
      sources=[os.path.join(ROOT, "native_ext.cpp")],
      ),
      ],
      zip_safe=False,
      )

      Compile extension:

      python setup.py build_ext --inplace

      Execute the following script:

      from test.support import run_in_subinterp
      run_in_subinterp("""
      import sys
      sys.path.append(".")
      import native_ext
      native_ext.run_simple()
      """)

      The script deadlocks

    2. ericsnowcurrently commented on Jan 7, 2022

      @ericsnowcurrently
      Member

      Is this a duplicate of bpo-15751? (also see bpo-10915)

    3. pablogsal commented on Jan 7, 2022

      @pablogsal
      MemberAuthor

      Yeah, this is indeed related/duplicate of bpo-15751

    4. transferred this issue fromon Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

    Metadata

    Metadata

    Assignees

    No one assigned

      Labels

      Projects

      No projects

        Milestone

        No milestone

        Relationships

        None yet

        Development

        No branches or pull requests

        Issue actions