Bug report
Bug description:
How to Reproduce
CC=clang CXX=clang++ ./configure --with-thread-sanitizer --with-pydebug --with-lto=full && make -j8
TSAN_OPTIONS=handle_segv=0 ./python -X dev -m test test_multiprocessing_forkserver.test_manager -j8
ThreadSanitizer Output
WARNING: ThreadSanitizer: thread leak (pid=191832)
Thread T13 'Thread-13 (hand' (tid=192190, finished) created by thread T1 at:
#0 pthread_create <null> (python+0xf821e) (BuildId: 8cf5a2f67fcf002f5a851af0abefbcffdc70b4d7)
#1 do_start_joinable_thread /home/shamil/oss/cpython/main/Python/thread_pthread.h:281:14 (python+0x694fe1)
#2 PyThread_start_joinable_thread /home/shamil/oss/cpython/main/Python/thread_pthread.h:323:9 (python+0x7b3fb1)
#3 ThreadHandle_start /home/shamil/oss/cpython/main/./Modules/_threadmodule.c:474:9 (python+0x7b3fb1)
#4 do_start_new_thread /home/shamil/oss/cpython/main/./Modules/_threadmodule.c:1920:9 (python+0x7b3fb1)
#5 thread_PyThread_start_joinable_thread /home/shamil/oss/cpython/main/./Modules/_threadmodule.c:2043:14 (python+0x7b1df8)
[... rest of stack trace ...]
SUMMARY: ThreadSanitizer: thread leak
Reproduces: Every run
Observations
- Current TSAN suppressions (
Tools/tsan/suppressions.txt):
# This file contains suppressions for the default (with GIL) build.
# reference: https://github.com/google/sanitizers/wiki/ThreadSanitizerSuppressions
# https://gist.github.com/mpage/daaf32b39180c1989572957b943eb665
thread:pthread_create
This suppression masks all thread leaks starting from pthread_create.
- Code analysis of
Lib/multiprocessing/managers.py:
The Server.accepter thread runs an infinite loop:
def accepter(self):
while True: # Line ~197
try:
c = self.listener.accept()
except OSError:
continue
t = threading.Thread(target=self.handle_request, args=(c,))
t.daemon = True
t.start()The Server.serve_forever method sets self.stop_event:
def serve_forever(self):
self.stop_event = threading.Event()
# ...
accepter = threading.Thread(target=self.accepter)
accepter.daemon = True
accepter.start()
try:
while not self.stop_event.is_set(): # Line ~189
self.stop_event.wait(1)The Server.shutdown method sets the stop event:
def shutdown(self, c):
# ...
finally:
self.stop_event.set() # Line ~235
- Handler threads (created in
accepter) check stop_event:
def serve_client(self, conn):
while not self.stop_event.is_set(): # Line ~259
# handle requestsAnalysis
The accepter thread uses while True and does not check self.stop_event, unlike:
- The main loop in
serve_forever (line ~189)
- The handler threads in
serve_client (line ~259)
When shutdown() is called and stop_event is set, the accepter thread continues running because it never checks the event.
The test teardown (Lib/test/_test_multiprocessing.py, ManagerMixin.tearDownClass) calls:
cls.manager.shutdown()
cls.manager.join()
ThreadSanitizer detects that the daemon accepter thread was created but not properly joined/terminated.
Additional Context
- The broad
thread:pthread_create suppression in Tools/tsan/suppressions.txt currently masks this issue
- Similar test cleanup code exists in
BaseMixin.tearDownClass which warns about dangling threads
- The thread name in the TSAN output
'Thread-13 (hand' corresponds to handler threads created by the accepter
CPython versions tested on:
3.15
Operating systems tested on:
Linux
Linked PRs
Bug report
Bug description:
How to Reproduce
CC=clang CXX=clang++ ./configure --with-thread-sanitizer --with-pydebug --with-lto=full && make -j8TSAN_OPTIONS=handle_segv=0 ./python -X dev -m test test_multiprocessing_forkserver.test_manager -j8ThreadSanitizer Output
Reproduces: Every run
Observations
Tools/tsan/suppressions.txt):This suppression masks all thread leaks starting from
pthread_create.Lib/multiprocessing/managers.py:The
Server.accepterthread runs an infinite loop:The
Server.serve_forevermethod setsself.stop_event:The
Server.shutdownmethod sets the stop event:accepter) checkstop_event:Analysis
The
accepterthread useswhile Trueand does not checkself.stop_event, unlike:serve_forever(line ~189)serve_client(line ~259)When
shutdown()is called andstop_eventis set, theaccepterthread continues running because it never checks the event.The test teardown (
Lib/test/_test_multiprocessing.py,ManagerMixin.tearDownClass) calls:ThreadSanitizer detects that the daemon accepter thread was created but not properly joined/terminated.
Additional Context
thread:pthread_createsuppression inTools/tsan/suppressions.txtcurrently masks this issueBaseMixin.tearDownClasswhich warns about dangling threads'Thread-13 (hand'corresponds to handler threads created by the accepterCPython versions tested on:
3.15
Operating systems tested on:
Linux
Linked PRs