Repository navigation
An interpreter's initial thread can be accessed while finalizing #126914
Description
Crash report
What happened?
While working on gh-126644, I came across an elusive bug that seems to occur when a lot of threads are trying to create a thread state for the same interpreter. Initially, I thought it was an issue with free-threading, but it occurs on the default build as well.
Here's a small reproducer:
from threading import Thread
import _interpreters
interp = _interpreters.create()
def run():
this_interp = _interpreters.create()
_interpreters.run_string(this_interp, f"import _interpreters; _interpreters.run_string({interp}, '1')")
threads = [Thread(target=run) for _ in range(1000)]
for thread in threads:
thread.start()
for thread in threads:
thread.join()As it turns out, this is because of this check: https://github.com/python/cpython/blob/main/Python/pystate.c#L1531
Inside tstate_delete_common, the runtime lock is released after the threads.head has already been set to NULL, so other threads think erroneously think it's OK to try and use _initial_thread while it's still getting finalized. Possibly related, free_threadstate tries to reset _initial_thread back to the default settings without the runtime lock, which is possibly racy.
There's a few ways to fix this, but I think the easiest (and has the least amount of risk for backporting) is to just hold the runtime lock for all of thread state deletion. This will introduce a very slight slowdown due to lock contention, but that should get better once #114940 lands. I have a PR ready.
CPython versions tested on:
3.12, 3.13, 3.14, CPython main branch
Operating systems tested on:
Linux
Output from running 'python -VV' on the command line:
No response
Linked PRs
Metadata
Metadata
Assignees
Labels
Projects
- Status
Activity
ZeroIntensity commentedon Nov 17, 2024
Apparently, this is problematic for the free-threaded build, because QSBR and thread-local refcounting initialization cause a lock-ordering deadlock in some cases. So, instead, my solution is to just add a new field to the interpreter state that indicates when it's safe to access
_initial_thread, and that field is only used atomically (unlike something liketstate->_status). I'm hoping that should fix it without issues.gh-126914: Store the Preallocated Thread State's Pointer in a PyInter…
pythongh-126914: Store the Preallocated Thread State's Pointer in a P…
[3.13] gh-126914: Store the Preallocated Thread State's Pointer in a …
pythongh-126914: Store the Preallocated Thread State's Pointer in a P…