Daemonic threads not killed in some circumstances in python 3.13

Bug report

Bug description:

I got a bug report on python-zlib-ng where using the threaded opening the program would hang if an exception occurred and a context manager was not used.

I fixed it by changing the threads to be daemonic: pycompression/python-zlib-ng#54.

This fixes the issue on python 3.9, 3.10, 3.11 and 3.12, but not 3.13-rc2. Also the latest main branch is affected. Unfortunately I could not create a minimal reproducer, other than running the test directly. Daemonic threads seem not to work in this particular case, where there are some BufferedIO streams and queues involved.

git clone -b issue53 https://github.com/pycompression/python-zlib-ng
# Build Cpython
./python -m ensurepip
./python -m pip install pytest pytest-timeout <path to python-zlib-ng repo>
./python -m pytest <path to python-zlib-ng repo>/tests/test_gzip_ng_threaded.py::test_threaded_program_can_exit_on_error

Since there are many code changes between 3.12 in 3.13 the underlying cause is surely there, but it is hard to pinpoint.

CPython versions tested on:

3.13, CPython main branch

Operating systems tested on:

Linux

Linked PRs

gh-123940: Ensure force-terminated daemon threads can be joined #124150

Joining daemon threads at finalizer of _ThreadedGzipReader and _ThreadedGzipWriter (i.e. close()) gives a deadlock in recent versions of Python. I am still investigating on this and trying to give an MRE. The following patch makes it pass the test, but I am not sure whether it will break other versions (tested on Python 3.12 and latest).

diff --git a/src/zlib_ng/gzip_ng_threaded.py b/src/zlib_ng/gzip_ng_threaded.py
index 7fa7249..65bd95c 100644
--- a/src/zlib_ng/gzip_ng_threaded.py
+++ b/src/zlib_ng/gzip_ng_threaded.py
@@ -156,7 +156,6 @@ class _ThreadedGzipReader(io.RawIOBase):
         if self._closed:
             return
         self.running = False
-        self.worker.join()
         self.fileobj.close()
         if self.closefd:
             self.raw.close()
@@ -329,7 +328,7 @@ class _ThreadedGzipWriter(io.RawIOBase):
         if self._closed:
             return
         self.flush()
-        self.stop()
+        self.running = False
         if self.exception:
             self.raw.close()
             self._closed = True

Is it possible to workaround it by starting the worker in first call to read() / write() and using non-daemon threads?

@y5c4l3

@y5c4l3 Thanks! I will investigate and report back.

Reproducible:

# mwe.py
import io
import threading
from spawner import ModuleSpawnerIO

class SpawnerIO(io.RawIOBase):
    def __init__(self):
        self.task = threading.Thread(target=self._process, daemon=True)
        self.task.start()
    def _process(self):
        while True:
            pass
    def readable(self):
        return True
    def close(self):
        print('finalizing...')
        self.task.join()

# exit as expected, not calling finalizer
# r = io.BufferedReader(SpawnerIO())

# finalized and stuck
r = io.BufferedReader(ModuleSpawnerIO())

# spawner.py
import io
import threading

class ModuleSpawnerIO(io.RawIOBase):
    def __init__(self):
        self.task = threading.Thread(target=self._process, daemon=True)
        self.task.start()
    def _process(self):
        while True:
            pass
    def readable(self):
        return True
    def close(self):
        print('finalizing...')
        self.task.join()

Wrapping ModuleSpawnerIO from another module inside BufferedReader ensures that the finalizer will be called. The spawned thread dies without notifying the main thread of thread_is_exiting:

cpython/Python/ceval_gil.c

Lines 294 to 302 in 1de4613

    
               if (_PyThreadState_MustExit(tstate)) { 
        
                   /* bpo-39877: If Py_Finalize() has been called and tstate is not the 
        
                      thread which called Py_Finalize(), exit immediately the thread. 
        
                      This code path can be reached by a daemon thread after Py_Finalize() 
        
                      completes. In this case, tstate is a dangling pointer: points to 
        
                      PyThreadState freed memory. */ 
        
                   PyThread_exit_thread(); 
        
               }

Simpler Reproducible with `atexit`

import io
import threading
import atexit

def process():
    while True:
        pass

t1 = threading.Thread(target=process, daemon=True)
t1.start()
atexit.register(lambda: t1.join())

@mpage

Thanks for the repro! Bisects to #114839
cc @mpage in case you have time to take a look :-)

Thanks for the minimal repo! I can take a look on Monday.

@hauntsaninja

@hauntsaninja - What were you using as the test case for bisection? The simpler reproducer using atexit has the same behavior on 3.12.4 and main. Both hang until interrupted, which is what I would expect to happen.

I was using the mwe.py + spawner.py repro (which exits in 3.12.5). I think y5c4l3 edited in the atexit example after I posted.

Merge branch 'main' into pythongh-123940

Don't join threads in destructors. That seems likely to cause problems because now you have blocking synchronization potentially at arbitrary places. In general, you can't be sure which thread will call destructors due to GC.

There is no custom destructors in python-zlib-ng that join threads. Where did you see this code? In fact, for the classes that cause the issue, there are no custom destructors.

Don't rely on deamon threads being joinable at shutdown [...] not a robust approach.

I really appreciate your explanation. What patterns do you recommend to avoid this sort of behaviour? I have a thread that does the IO. Are there any well documented patterns to prevent this from blocking?

There should be at least some documentation on this in the release notes if it is going to break currently used code. Python-isal uses the same code and is marked as an essential package for the python ecosystem. I'd like it to be able to work with python 3.13 when it comes out.

There is no custom destructors in python-zlib-ng that join threads. Where did you see this code?

The __del__ for RawIOBase calls close() and you have a thread.join() called from your close() (via stop()).

What patterns do you recommend to avoid this sort of behaviour?

Don't call thread.join() in close() if it's being called from __del__(). Something like:

class _ThreadedGzipReader(io.RawIOBase):
    def stop(self):
        if self.running:
            self.running = False
            if not self._finalizing:
                for worker in self.compression_workers:
                    worker.join()
                self.output_worker.join()

    def __del__(self):
        self._finalizing = True
        super().__del__()

@colesbury

@colesbury Thank you so much for this additional information. Now I understand the problem and can fix it appropriately.

People are leaning towards not treating this as blocking for 3.13.0 (x-ref colesbury's message on internal Discord)

colesbury also mentioned #116514 as a reference for hard crashes on 3.11/3.12 (I wasn't able to reproduce #124150 (comment) )

pythongh-87135: Raise PythonFinalizationError when joining a blocked …

gh-87135: Raise PythonFinalizationError when joining a blocked daemon…

In 3.14 daemon threads will hang if they attempt to call Python API after finalization. Also, joining such a thread will raise PythonFinalizationError.
This will not be backported to 3.13. The change is too invasive. See also this topic: “In general, avoiding new bugs or changes in behaviour is more important than fixing existing bugs.”

AFAICS, the immediate issue in zlib-ng can be (and was?) fixed by avoiding thread.join() being called (indirectly) from finalizers.

Looks like there's nothing more to do for this issue. I'll close it next week if there are no objections.

Navigation Menu

Repository navigation

Daemonic threads not killed in some circumstances in python 3.13 #123940

Bug report

Bug description:

CPython versions tested on:

Operating systems tested on:

Linked PRs

Simpler Reproducible with `atexit`

5 remaining items

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

	if (_PyThreadState_MustExit(tstate)) {
	/* bpo-39877: If Py_Finalize() has been called and tstate is not the
	thread which called Py_Finalize(), exit immediately the thread.

	This code path can be reached by a daemon thread after Py_Finalize()
	completes. In this case, tstate is a dangling pointer: points to
	PyThreadState freed memory. */
	PyThread_exit_thread();
	}

Daemonic threads not killed in some circumstances in python 3.13 #123940

Description

Bug report

Bug description:

CPython versions tested on:

Operating systems tested on:

Linked PRs

Activity

y5c4l3 commented on Sep 12, 2024

rhpvorderman commented on Sep 13, 2024

y5c4l3 commented on Sep 14, 2024

Simpler Reproducible with atexit

hauntsaninja commented on Sep 15, 2024

mpage commented on Sep 15, 2024

mpage commented on Sep 15, 2024

hauntsaninja commented on Sep 15, 2024

5 remaining items

rhpvorderman commented on Sep 20, 2024

colesbury commented on Sep 20, 2024

rhpvorderman commented on Sep 23, 2024

hauntsaninja commented on Sep 23, 2024

encukou commented on Jun 16, 2025

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions

Simpler Reproducible with `atexit`