Bug report
Bug description:
PyMember_GetOne does not use an atomic read when reading a _Py_T_OBJECT member, even though PyMember_SetOne uses an atomic write when updating such a member.
To demonstrate this, I'm using builtin_function_or_method's __module__, which is an example of a writeable _Py_T_OBJECT member in CPython's codebase. I don't know what field exactly was the source of the race when I found it in another codebase; it may or may not have been the same.
To reproduce, run the following code under CPython 3.13.1t with thread-sanitizer enabled:
import concurrent.futures
import functools
import threading
num_threads = 10
def closure(b, c):
b.wait()
for _ in range(100):
x = c.__module__
c.__module__ = 42
with concurrent.futures.ThreadPoolExecutor(max_workers=num_threads) as executor:
for i in range(100):
b = threading.Barrier(num_threads)
c = [].append
for _ in range(num_threads):
executor.submit(functools.partial(closure, b, c))and you'll get this race:
WARNING: ThreadSanitizer: data race (pid=350546)
Atomic write of size 8 at 0x7f03d4a7d1e0 by thread T1:
#0 _Py_atomic_store_ptr_release /usr/local/google/home/phawkins/p/cpython/./Include/cpython/pyatomic_gcc.h:501:3 (python3.13+0x4a73a9) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
#1 PyMember_SetOne /usr/local/google/home/phawkins/p/cpython/Python/structmember.c:308:9 (python3.13+0x4a73a9)
#2 member_set /usr/local/google/home/phawkins/p/cpython/Objects/descrobject.c:238:12 (python3.13+0x1ff85a) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
#3 _PyObject_GenericSetAttrWithDict /usr/local/google/home/phawkins/p/cpython/Objects/object.c:1748:19 (python3.13+0x297b4e) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
#4 PyObject_GenericSetAttr /usr/local/google/home/phawkins/p/cpython/Objects/object.c:1819:12 (python3.13+0x298667) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
#5 PyObject_SetAttr /usr/local/google/home/phawkins/p/cpython/Objects/object.c:1385:15 (python3.13+0x294f8a) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
#6 _PyEval_EvalFrameDefault /usr/local/google/home/phawkins/p/cpython/Python/generated_cases.c.h:5488:27 (python3.13+0x3f3f6d) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
#7 _PyEval_EvalFrame /usr/local/google/home/phawkins/p/cpython/./Include/internal/pycore_ceval.h:119:16 (python3.13+0x3de62a) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
#8 _PyEval_Vector /usr/local/google/home/phawkins/p/cpython/Python/ceval.c:1811:12 (python3.13+0x3de62a)
#9 _PyFunction_Vectorcall /usr/local/google/home/phawkins/p/cpython/Objects/call.c (python3.13+0x1eb61f) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
#10 _PyObject_VectorcallTstate /usr/local/google/home/phawkins/p/cpython/./Include/internal/pycore_call.h:168:11 (python3.13+0x571bb2) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
#11 partial_vectorcall /usr/local/google/home/phawkins/p/cpython/./Modules/_functoolsmodule.c:252:16 (python3.13+0x571bb2)
#12 _PyVectorcall_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:273:16 (python3.13+0x1eb293) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
#13 _PyObject_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:348:16 (python3.13+0x1eb293)
#14 PyObject_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:373:12 (python3.13+0x1eb315) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
#15 _PyEval_EvalFrameDefault /usr/local/google/home/phawkins/p/cpython/Python/generated_cases.c.h:1355:26 (python3.13+0x3e46e2) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
#16 _PyEval_EvalFrame /usr/local/google/home/phawkins/p/cpython/./Include/internal/pycore_ceval.h:119:16 (python3.13+0x3de62a) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
#17 _PyEval_Vector /usr/local/google/home/phawkins/p/cpython/Python/ceval.c:1811:12 (python3.13+0x3de62a)
#18 _PyFunction_Vectorcall /usr/local/google/home/phawkins/p/cpython/Objects/call.c (python3.13+0x1eb61f) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
#19 _PyObject_VectorcallTstate /usr/local/google/home/phawkins/p/cpython/./Include/internal/pycore_call.h:168:11 (python3.13+0x1ef5ef) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
#20 method_vectorcall /usr/local/google/home/phawkins/p/cpython/Objects/classobject.c:70:20 (python3.13+0x1ef5ef)
#21 _PyVectorcall_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:273:16 (python3.13+0x1eb293) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
#22 _PyObject_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:348:16 (python3.13+0x1eb293)
#23 PyObject_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:373:12 (python3.13+0x1eb315) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
#24 thread_run /usr/local/google/home/phawkins/p/cpython/./Modules/_threadmodule.c:337:21 (python3.13+0x564292) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
#25 pythread_wrapper /usr/local/google/home/phawkins/p/cpython/Python/thread_pthread.h:243:5 (python3.13+0x4bd637) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
Previous read of size 8 at 0x7f03d4a7d1e0 by thread T10:
#0 PyMember_GetOne /usr/local/google/home/phawkins/p/cpython/Python/structmember.c:86:13 (python3.13+0x4a6c64) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
#1 member_get /usr/local/google/home/phawkins/p/cpython/Objects/descrobject.c:179:12 (python3.13+0x1ff6ca) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
#2 _PyObject_GenericGetAttrWithDict /usr/local/google/home/phawkins/p/cpython/Objects/object.c:1635:19 (python3.13+0x295de1) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
#3 PyObject_GenericGetAttr /usr/local/google/home/phawkins/p/cpython/Objects/object.c:1717:12 (python3.13+0x295c32) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
#4 PyObject_GetAttr /usr/local/google/home/phawkins/p/cpython/Objects/object.c:1231:18 (python3.13+0x294597) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
#5 _PyEval_EvalFrameDefault /usr/local/google/home/phawkins/p/cpython/Python/generated_cases.c.h:3766:28 (python3.13+0x3eddf0) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
#6 _PyEval_EvalFrame /usr/local/google/home/phawkins/p/cpython/./Include/internal/pycore_ceval.h:119:16 (python3.13+0x3de62a) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
#7 _PyEval_Vector /usr/local/google/home/phawkins/p/cpython/Python/ceval.c:1811:12 (python3.13+0x3de62a)
#8 _PyFunction_Vectorcall /usr/local/google/home/phawkins/p/cpython/Objects/call.c (python3.13+0x1eb61f) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
#9 _PyObject_VectorcallTstate /usr/local/google/home/phawkins/p/cpython/./Include/internal/pycore_call.h:168:11 (python3.13+0x571bb2) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
#10 partial_vectorcall /usr/local/google/home/phawkins/p/cpython/./Modules/_functoolsmodule.c:252:16 (python3.13+0x571bb2)
#11 _PyVectorcall_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:273:16 (python3.13+0x1eb293) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
#12 _PyObject_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:348:16 (python3.13+0x1eb293)
#13 PyObject_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:373:12 (python3.13+0x1eb315) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
#14 _PyEval_EvalFrameDefault /usr/local/google/home/phawkins/p/cpython/Python/generated_cases.c.h:1355:26 (python3.13+0x3e46e2) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
#15 _PyEval_EvalFrame /usr/local/google/home/phawkins/p/cpython/./Include/internal/pycore_ceval.h:119:16 (python3.13+0x3de62a) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
#16 _PyEval_Vector /usr/local/google/home/phawkins/p/cpython/Python/ceval.c:1811:12 (python3.13+0x3de62a)
#17 _PyFunction_Vectorcall /usr/local/google/home/phawkins/p/cpython/Objects/call.c (python3.13+0x1eb61f) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
#18 _PyObject_VectorcallTstate /usr/local/google/home/phawkins/p/cpython/./Include/internal/pycore_call.h:168:11 (python3.13+0x1ef5ef) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
#19 method_vectorcall /usr/local/google/home/phawkins/p/cpython/Objects/classobject.c:70:20 (python3.13+0x1ef5ef)
#20 _PyVectorcall_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:273:16 (python3.13+0x1eb293) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
#21 _PyObject_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:348:16 (python3.13+0x1eb293)
#22 PyObject_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:373:12 (python3.13+0x1eb315) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
#23 thread_run /usr/local/google/home/phawkins/p/cpython/./Modules/_threadmodule.c:337:21 (python3.13+0x564292) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
#24 pythread_wrapper /usr/local/google/home/phawkins/p/cpython/Python/thread_pthread.h:243:5 (python3.13+0x4bd637) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
The fix is presumably to make the read atomic as well.
CPython versions tested on:
3.13
Operating systems tested on:
Linux
Bug report
Bug description:
PyMember_GetOnedoes not use an atomic read when reading a_Py_T_OBJECTmember, even thoughPyMember_SetOneuses an atomic write when updating such a member.To demonstrate this, I'm using
builtin_function_or_method's__module__, which is an example of a writeable_Py_T_OBJECTmember in CPython's codebase. I don't know what field exactly was the source of the race when I found it in another codebase; it may or may not have been the same.To reproduce, run the following code under CPython 3.13.1t with thread-sanitizer enabled:
and you'll get this race:
The fix is presumably to make the read atomic as well.
CPython versions tested on:
3.13
Operating systems tested on:
Linux