Skip to content

PyThreadState_Next not thread safe #40857

Open
Open
PyThreadState_Next not thread safe#40857
Labels
interpreter-core(Objects, Python, Grammar, and Parser dirs)topic-C-API

Description

@jpe
jpe
mannequin
opened on Sep 2, 2004
BPO 1021318
Nosy @pitrou, @vstinner, @devdanzin, @jd, @ericsnowcurrently, @serhiy-storchaka

Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2004-09-02.16:56:47.000>
labels = ['interpreter-core', '3.9']
title = 'PyThreadState_Next not thread safe'
updated_at = <Date 2020-05-15.01:09:40.629>
user = 'https://bugs.python.org/jpe'

bugs.python.org fields:

activity = <Date 2020-05-15.01:09:40.629>
actor = 'vstinner'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Interpreter Core']
creation = <Date 2004-09-02.16:56:47.000>
creator = 'jpe'
dependencies = []
files = []
hgrepos = []
issue_num = 1021318
keywords = []
message_count = 14.0
messages = ['60562', '82072', '114376', '358187', '358188', '358189', '358190', '358191', '358192', '358261', '358262', '358347', '364094', '368895']
nosy_count = 7.0
nosy_names = ['jpe', 'pitrou', 'vstinner', 'ajaksu2', 'jd', 'eric.snow', 'serhiy.storchaka']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = None
url = 'https://bugs.python.org/issue1021318'
versions = ['Python 3.9']

Activity

  1. jpe commented on Sep 2, 2004

    @jpe
    jpemannequin
    on Sep 2, 2004
    MannequinAuthor

    I can't convince myself that PyThreadState_Next is
    thread safe even if the GIL is held because
    PyGILState_Release can delete a thread state after
    releasing the GIL if the gilstate_counter reaches 0.
    So another thread could be deleting a thread state my
    thread is referencing.

    If the GIL was held when the thread state was deleted,
    the I think the next function would be thread safe
    because new thread states are always added to the front
    of the list.

    If these functions should work when the GIL is not
    held, the head_mutex in pystate.c probably needs to be
    exposed in some manner so code can lock it, iterate
    through the threads, and then unlock it.

  2. jpe
    added
    interpreter-core(Objects, Python, Grammar, and Parser dirs)
    on Sep 2, 2004

  3. devdanzin commented on Feb 14, 2009

    @devdanzin
    devdanzinmannequin
    on Feb 14, 2009
    Mannequin

    Can someone take a look a this and either close or propose what to do?

  4. BreamoreBoy commented on Aug 19, 2010

    @BreamoreBoy
    BreamoreBoymannequin
    on Aug 19, 2010
    Mannequin

    No reply to msg82072.

  5. BreamoreBoy
    closed this as completedon Aug 19, 2010
  6. BreamoreBoy
    closed this as completedon Aug 19, 2010

  7. jd commented on Dec 10, 2019

    @jd
    jdmannequin
    on Dec 10, 2019
    Mannequin

    This is actually true and it's quite easy to produce a bug that triggers a segmentation fault when iterating over a large list of threads create dynamically.

    That makes PyThreadState_Next barely usable for any program that has a few threads. The head mutex needs to be locked to be able to iterate safely over the list of thread.

  8. jd
    added
    3.7 (EOL)end of life
    3.8 (EOL)end of life
    3.9 (EOL)end of life
    on Dec 10, 2019
  9. jd
    changed the title [-]PyThreadState_Next not thread safe?[/-] [+]PyThreadState_Next not thread safe[/+] on Dec 10, 2019
  10. jd
    added
    3.7 (EOL)end of life
    3.8 (EOL)end of life
    3.9 (EOL)end of life
    on Dec 10, 2019

  11. 10 remaining items

  12. vstinner commented on Dec 10, 2019

    @vstinner
    vstinner
    on Dec 10, 2019
    Member

    We'd simply need to provide a new API to lock/unlock the interpreter's mutex.

    Are you talking about this lock?

    #define HEAD_LOCK(runtime) \
    PyThread_acquire_lock((runtime)->interpreters.mutex, WAIT_LOCK)
#define HEAD_UNLOCK(runtime) \
    PyThread_release_lock((runtime)->interpreters.mutex)

  13. jd commented on Dec 10, 2019

    @jd
    jdmannequin
    on Dec 10, 2019
    Mannequin

    Yes, that's the one 👍

  14. jd commented on Dec 11, 2019

    @jd
    jdmannequin
    on Dec 11, 2019
    Mannequin

    Victor, do you have an idea of how you would like this mutex to be exposed?

    I see it only recently changed in Python 3.7 to be part of a structure, though this structure is not public AFAIU.

    Is adding 2 new lock/unlock C functions good enough?

  15. vstinner commented on Dec 11, 2019

    @vstinner
    vstinner
    on Dec 11, 2019
    Member

    I'm not sure of what is the best approach. Currently, bpo-10915 is blocking different projects: "Make the PyGILState API compatible with multiple interpreters". We may have to fix this API first, and clarify the scope of the different locks.

  16. ericsnowcurrently commented on Dec 13, 2019

    @ericsnowcurrently
    ericsnowcurrently
    on Dec 13, 2019
    Member

    On Wed, Dec 11, 2019 at 7:02 AM STINNER Victor <report@bugs.python.org> wrote:

    We may have to fix this API first, and clarify the scope of the different locks.

    +1

  17. vstinner commented on Mar 13, 2020

    @vstinner
    vstinner
    on Mar 13, 2020
    Member

    See also bpo-35370: "Add _PyEval_SetTrace(tstate, func, arg) function".

  18. vstinner commented on May 15, 2020

    @vstinner
    vstinner
    on May 15, 2020
    Member

    I marked bpo-26461 as a duplicate of this issue.

  19. ezio-melotti
    transferred this issue fromon Apr 9, 2022
  20. serhiy-storchaka
    added this to Threading issues 🧵on May 20, 2022
  21. iritkatriel
    added
    3.12only security fixes
    and removed
    3.9 (EOL)end of life
    on Jun 15, 2022
  22. iritkatriel
    added
    topic-C-API
    on May 14, 2024
  23. erlend-aasland
    removed
    3.12only security fixes
    on May 14, 2024

  24. a7a-dar commented on Aug 23, 2024

    @a7a-dar
    a7a-dar
    on Aug 23, 2024 · Hidden as spam

  25. ashm-dev commented on Nov 13, 2025

    @ashm-dev
    ashm-dev
    on Nov 13, 2025
    Contributor

    Ping. Is this issue still relevant?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    interpreter-core(Objects, Python, Grammar, and Parser dirs)topic-C-API

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions