Repository navigation
PidfdChildWatcher removed without deprecation #130915
Description
Activity
That's caused by this issue removing a publicly documented API which (as per the current official documentation) was not deprecated or slated for removal:
The whole child watcher subsystem of asyncio was deprecated and slated for removal in 3.14. Specifically the functions to the set and get the child watchers i.e.
get_child_watcherandset_child_watcherwere deprecated and removed.The code in question is incorrect, it should check the presence of
set_child_watcheralong side checking for watcher as just creating a watcher without being able to set it has no use.The whole child watcher subsystem of asyncio was deprecated and slated for removal in 3.14.
Do you mean that it was both deprecated and removed in the 3.14 development cycle (ie: without advance warning)?
Specifically the functions to the set and get the child watchers i.e. get_child_watcher and set_child_watcher were deprecated and removed.
We're not using either of those functions. We're just creating a
PidfdChildWatcherand calling.add_child_handler()on it. It was working nicely for us that way before it was removed.Do you mean that it was both deprecated and removed in the 3.14 development cycle (ie: without advance warning)?
It was deprecated since 3.12 and removed in 3.14
We're not using either of those functions. We're just creating a PidfdChildWatcher and calling .add_child_handler() on it. It was working nicely for us that way before it was removed.
The way to use a child watcher is to set it after creating it as per documented here https://docs.python.org/3.13/library/asyncio-policy.html#process-watchers
FWIW the easiest fix would be to avoid creating child watcher at all as in versions 3.12+ the best child watcher is used automatically as suggested here https://docs.python.org/3/whatsnew/3.12.html#asyncio
Our issue is that we still need to retain compatibility all the way back to Python 3.6 and to kernel versions where pidfds are not available.
I'm also concerned about people trying to use existing version of our software with new versions of Python: even if we add a workaround now, we're not going to backport that workaround to previous versions. Older versions of our software are packaged in a large number of Linux distributions. One of the features of our software involves connecting to remote systems and running
python3from thePATHand injecting itself into that system. This has the potential to get pretty messy if thatpython3ends up being 3.14...Our issue is that we still need to retain compatibility all the way back to Python 3.6 and to kernel versions where pidfds are not available.
That's what I suggested above to configure child watcher in <3.12, on 3.12+ let asyncio handle it and it will do the right thing if kernel supports it.
I'm also concerned about people trying to use existing version of our software with new versions of Python: even if we add a workaround now, we're not going to backport that workaround to previous versions. Older versions of our software are packaged in a large number of Linux distributions. One of the features of our software involves connecting to remote systems and running python3 from the PATH and injecting itself into that system. This has the potential to get pretty messy if that python3 ends up being 3.14...
Your code also uses SafeChildWatcher as fallback which is also removed so your code will not work anyways on newer Python versions.
Your code also uses SafeChildWatcher as fallback which is also removed so your code will not work anyways on newer Python versions.
We only try to use SafeChildWatcher if PidfdChildWatcher is missing (ie: for very old Python versions).
We specifically did that because SafeChildWatcher was deprecated and slated for removal...
We only try to use SafeChildWatcher if PidfdChildWatcher is missing (ie: for very old Python versions).
We specifically did that because SafeChildWatcher was deprecated and slated for removal...The version of Python doesn't matter, if the kernel is old enough, then it would fallback to
SafeChildWatcherwhich would break anyways.Although that's strictly true, we don't have any supported systems with Python 3.14 and a kernel not supporting pidfd (and I doubt that any such systems exist anywhere...).
I am closing this as it is the intended behavior.
Reacted by Allison KarlitskayaI am closing this as it is the intended behavior.
I'm sorry to keep nagging on a closed issue, but I want to make sure I fully understand the justification behind this decision, because it has the potential to cause serious problems for us (Cockpit).
I've been doing a bit of research and I came across https://peps.python.org/pep-0387/#basic-policy-for-backwards-compatibility
This is (I think) the current official policy about backwards compatibility between stable Python releases. It pretty clearly states that unless an exception has been granted by the steering council for "extreme situations" (eg. dangerously broken or insecure features, obsolete platforms, etc.) then "Unless it is going through the deprecation process below, the behavior of an API must not change in an incompatible fashion between any two consecutive releases" and "the deprecation period must last at least two years".
To reiterate: this is a public API (
async.PidfdChildWatcher), which was documented (and continues to be documented) in the current official stable version of the docs on docs.python.org, without deprecation notice, which is being removed from one release to the next.There is also no deprecation notice generated at run time:
Python 3.13.2 (main, Feb 6 2025, 00:00:00) [GCC 15.0.1 20250204 (Red Hat 15.0.1-0)] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import asyncio >>> asyncio.PidfdChildWatcher() <asyncio.unix_events.PidfdChildWatcher object at 0x7fe9b1e21fd0>(API not deprecated, no notice is given)
Compare with:
Python 3.13.2 (main, Feb 6 2025, 00:00:00) [GCC 15.0.1 20250204 (Red Hat 15.0.1-0)] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import asyncio >>> asyncio.SafeChildWatcher() <python-input-1>:1: DeprecationWarning: 'SafeChildWatcher' is deprecated as of Python 3.12 and will be removed in Python 3.14. asyncio.SafeChildWatcher() <asyncio.unix_events.SafeChildWatcher object at 0x7ff958cbdfd0>(deprecated API gives a deprecation notice on usage)
Was this an exception granted by the steering council, or has there been some miscommunication along the way? What does "intended behavior" mean in this context?
Sorry again for writing on a closed issue, and thanks in advance for any further consideration.
@kumaraditya303 : I am currently looking into this, and I'm stumped how the API is actually supposed to work in current Python (i.e. >= 3.12) (nevermind old versions, I know how to add the fallbacks):
- https://docs.python.org/3/whatsnew/3.12.html#asyncio and PidfdChildWatcher removed without deprecation #130915 (comment) mention that the entire child watcher API is deprecated.
- https://docs.python.org/3.13/library/asyncio-policy.html#process-watchers also has everything deprecated, i.e. it doesn't mention any current API
- So maybe this cannot be used with arbitrary processes any more, but just with the "Process" class that asyncio.create_subprocess_exec() returns? But https://docs.python.org/3.13/library/asyncio-subprocess.html#asyncio.subprocess.Process also doesn't have anything like "invoke this callback then the process exits". The only thing it has is
async wait, which could possibly be run in a task?
I cannot find any documentation how to achieve this now --- i.e. what we need is
something.call_this_when_pid_exits(pid, callback). Thanks for any help!This is (I think) the current official policy about backwards compatibility between stable Python releases. It pretty clearly states that unless an exception has been granted by the steering council for "extreme situations" (eg. dangerously broken or insecure features, obsolete platforms, etc.) then "Unless it is going through the deprecation process below, the behavior of an API must not change in an incompatible fashion between any two consecutive releases" and "the deprecation period must last at least two years".
The deprecation warning is raised when you try to set the child watcher using
set_child_watcherwhich is the correct way to use it as I explained before. Just creating a child watcher without setting it for the loop is not correct as I said. It is not practical to add deprecation warning to each and every function and odd ways to use a child watcher. Your code uses a child watcher without setting it which is broken because asyncio expects only one child watcher per loop, otherwise asyncio has no way to know if any child watcher is set or overridden.The following correct code raises the necessary warnings:
import asyncio watcher = asyncio.PidfdChildWatcher() # do something with watcher # now set it as the watcher and in policy registry asyncio.set_child_watcher(watcher)
/home/realkumaraditya/cpython/Lib/asyncio/events.py:834: DeprecationWarning: 'set_child_watcher' is deprecated as of Python 3.12 and will be removed in Python 3.14. return get_event_loop_policy().set_child_watcher(watcher)
I cannot find any documentation how to achieve this now --- i.e. what we need is something.call_this_when_pid_exits(pid, callback). Thanks for any help!
If you want to add some callback for when process exits you can add that as done callback to a task calling
Process.wait.@kumaraditya303 OK, thanks for clarifying! We can't (or at least couldn't in the past) use
AbstractEventLoop.subprocess_execas that didn't support PTYs, but we can (1) build our own child watch API relatively easily, and (2) re-evaluate the asyncio API for subprocesses again.OK, thanks for clarifying! We can't (or at least couldn't in the past) use AbstractEventLoop.subprocess_exec as that didn't support PTYs, but we can (1) build our own child watch API relatively easily, and (2) re-evaluate the asyncio API for subprocesses again.
Building a child watcher API is not difficult considering your project already implements most of subprocess transport from scratch. You can even copy the impl from now private https://github.com/python/cpython/blob/main/Lib/asyncio/unix_events.py and use it which I wrote for 3.14 main branch.
Building a child watcher API is not difficult considering your project already implements most of subprocess transport from scratch. You can even copy the impl from now private https://github.com/python/cpython/blob/main/Lib/asyncio/unix_events.py and use it which I wrote for 3.14 main branch.
_UnixSelectorEventLoopnow always has a private_watcherattribute. It's tempting to use it to watch child processes, but the attribute is private :-(I understand that the customization of the child watcher is gone. But what about reusing
_UnixSelectorEventLoop._watcher? Is it ok? Can this attribute be made public so it's possible to calladd_child_handler()on it?I proposed a fix for cockpit using this private attribute: cockpit-project/cockpit#21787.
Or is there another recommended way to use child watchers on Python 3.14? Is it ok to have two
_PidfdChildWatcher()instances and attach them to the same event loop? I don't think that it would work well :-(I understand that the customization of the child watcher is gone. But what about reusing _UnixSelectorEventLoop._watcher? Is it ok? Can this attribute be made public so it's possible to call add_child_handler() on it?
It is private API and that would probably be gone in Python 3.15.
Or is there another recommended way to use child watchers on Python 3.14? Is it ok to have two _PidfdChildWatcher() instances and attach them to the same event loop? I don't think that it would work well :-(
Child watchers are gone in 3.14 and there is not such API now, if you want to add pid exit callback then you can do with Process.wait + done callback as I suggested above, otherwise you can copy _PidfdChildWatcher for your use case if you can't do the former.
Reacted by Martin Pitt and Victor Stinner- added a commit that references this issue
on Mar 15, 2026
Bug report
Bug description:
hi
We got a bug report that our software isn't working with Python 3.14 prereleases: https://bugzilla.redhat.com/show_bug.cgi?id=2350287
That's caused by this issue removing a publicly documented API which (as per the current official documentation) was not deprecated or slated for removal:
#120893
https://docs.python.org/3/library/asyncio-policy.html#asyncio.PidfdChildWatcher
I kinda get the intent that the entire abstract child watcher approach was intended to be deprecated, but that's not how it was advertised in the official docs. We went out of our way to try to use the new API if available and fall back to the old deprecated API only if the new one wasn't available. Having both pulled out in the same release has just broken our code.
Please add this back.
CPython versions tested on:
3.14
Operating systems tested on:
Linux