Can not communicate with subprocess during interpreter shutdown on windows

Bug report

Bug description:

In #104826 (backported to 3.12 in #105277) to fix #104690 disallowed creating new threads during interpreter finalization. This poses a problem on windows because the Popen._communicate uses a thread (see matplotlib/matplotlib#27437 (comment) for discussion).

In Matplotlib we are holding onto a latex process and triggering this via a weakref finalize, but a minimal reproducer is:

from subprocess import Popen, PIPE
import atexit

# anything that holds stdin and stdout should work
proc = Popen(['powershell'], stdin=PIPE, stdout=PIPE)

def cleanup():
    print('hi bob')
    proc.kill()
    proc.communicate()

atexit.register(cleanup)

xref matplotlib/matplotlib#27437

CPython versions tested on:

3.12

Operating systems tested on:

Windows

@chgnrdv

cc @chgnrdv, @gpshead

@ericsnowcurrently

(cc @ericsnowcurrently)

One possibility is to transform the Windows code here into a purely native thread, rather than a Python thread, but that is likely to break people who are passing file objects to stdout=/stderr= besides our constants. That's probably unlikely, but we also shouldn't break it.

What might be better, and why I pinged Eric, is to find a finalization ordering where we can run atexit callbacks and then require all threads to be exited. That may not help with the similar GC issues, but it seems like atexit ought to be okay with running a bit earlier.

See gh-116677.

This subprocess on Windows atexit or from a non-daemon thread use case is a good one. I'm going to mark this a duplicate of our more general issue #113964 on the topic where we've got a PR and likely a second one in the works to address this.

Navigation Menu

Repository navigation

Can not communicate with subprocess during interpreter shutdown on windows #115219

Bug report

Bug description:

CPython versions tested on:

Operating systems tested on:

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Can not communicate with subprocess during interpreter shutdown on windows #115219

Description

Bug report

Bug description:

CPython versions tested on:

Operating systems tested on:

Activity

serhiy-storchaka commented on Feb 9, 2024

zooba commented on Feb 12, 2024

ericsnowcurrently commented on Mar 18, 2024

gpshead commented on Mar 19, 2024

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions