Repository navigation
Add _at_fork_reinit() method to locks #84270
Description
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = None
closed_at = <Date 2020-04-21.01:07:34.430>
created_at = <Date 2020-03-27.15:51:37.958>
labels = ['library', '3.9']
title = 'Add _at_fork_reinit() method to locks'
updated_at = <Date 2020-04-21.01:07:34.430>
user = 'https://github.com/vstinner'bugs.python.org fields:
activity = <Date 2020-04-21.01:07:34.430>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2020-04-21.01:07:34.430>
closer = 'vstinner'
components = ['Library (Lib)']
creation = <Date 2020-03-27.15:51:37.958>
creator = 'vstinner'
dependencies = []
files = []
hgrepos = []
issue_num = 40089
keywords = ['patch']
message_count = 8.0
messages = ['365157', '365165', '365168', '365171', '365176', '365945', '365947', '365948']
nosy_count = 3.0
nosy_names = ['vstinner', 'miss-islington', 'Batuhan Taskaya']
pr_nums = ['19191', '19193', '19194', '19195', '19196']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue40089'
versions = ['Python 3.9']
Activity
vstinner commentedon Mar 27, 2020
Using a lock after fork() is unsafe and can crash.
Example of a crash in logging after a fork on AIX:
https://bugs.python.org/issue40068#msg365028
This problem is explained in length in bpo-6721: "Locks in the standard library should be sanitized on fork".
The threading module registers an "at fork" callback: Thread._reset_internal_locks() is called to reset self._started (threading.Event) and self._tstate_lock. The current implementation creates new Python lock objects and forgets about the old ones.
I propose to add a new _at_fork_reinit() method to Python lock objects which reinitializes the native lock internally without having to create a new Python object.
Currently, my implementation basically creates a new native lock object and forgets about the old new (don't call PyThread_free_lock()).
Tomorrow, we can imagine a more efficient impementation using platform specific function to handle this case without having to forget about the old lock.
vstinner commentedon Mar 27, 2020
_at_fork() has a bug: it creates a _DummyThread instead of a _MainThread. Example:
---
print(f"parent process: {threading.current_thread()=}")
print(f"parent process: {threading._main_thread=}")
Output:
---
parent process: threading.current_thread()=<_MainThread(MainThread, started 139879200077632)>
parent process: threading._main_thread=<_MainThread(MainThread, started 139879200077632)>
child process: threading.current_thread()=<_DummyThread(Dummy-1, started daemon 139878980245248)>
child process: threading._main_thread=<_DummyThread(Dummy-1, started daemon 139878980245248)>
---
My PR 19191 fix the issue:
---
parent process: threading.current_thread()=<_MainThread(MainThread, started 140583665170240)>
parent process: threading._main_thread=<_MainThread(MainThread, started 140583665170240)>
child process: threading.current_thread()=<_MainThread(MainThread, started 140583445075712)>
child process: threading._main_thread=<_MainThread(MainThread, started 140583445075712)>
---
vstinner commentedon Mar 27, 2020
New changeset d8ff44c by Victor Stinner in branch 'master':
bpo-40089: Fix threading._after_fork() (GH-19191)
d8ff44c
vstinner commentedon Mar 27, 2020
See also bpo-40091 "Crash in logging._after_at_fork_child_reinit_locks()" that I just created.
vstinner commentedon Mar 27, 2020
See also bpo-40092: "Crash in _PyThreadState_DeleteExcept() at fork in the process child".
vstinner commentedon Apr 7, 2020
New changeset 87255be by Victor Stinner in branch 'master':
bpo-40089: Add _at_fork_reinit() method to locks (GH-19195)
87255be
vstinner commentedon Apr 7, 2020
New changeset 6318e45 by Miss Islington (bot) in branch '3.8':
bpo-40089: Fix threading._after_fork() (GH-19191) (GH-19194)
6318e45
vstinner commentedon Apr 7, 2020
New changeset a514ccb by Miss Islington (bot) in branch '3.7':
bpo-40089: Fix threading._after_fork() (GH-19191) (GH-19193)
a514ccb