Cancellation ignored by asyncio.wait_for can hang application #87555

Activity

1. nmatravolgyi commented on Mar 3, 2021

   

   nmatravolgyimannequin

   on Mar 3, 2021

   MannequinAuthor

   More actions

   I have found myself debugging a *very* not intuitive behavior regarding asyncio.wait_for that I'd consider a bug/deficiency. The problem very simply put: wait_for will return the wrapped future's result even when it is being cancelled, ignoring the cancellation as it has never existed.

   This will make parallel execution-waits hang forever if some simple conditions are met. From the perspective of this snippet every task must exit so it just needs to wait. I know cancellation *can* be ignored, but it is discouraged by the documentation for this reason exactly.

   tasks = [...]
   for t in tasks:
       t.cancel()
   results = await asyncio.gather(*tasks, return_exceptions=True)

   I already know that this behavior has been chosen because otherwise the returned value would be lost. But for many applications, losing an explicit cancellation error/event is just as bad.

   The reason why ignoring the cancellation is critical is because the cancelling (arbiter) task cannot reliably solve it. In most cases having repeated cancellations in a polling wait can solve this, but it is ugly and does not work if the original wait_for construct is in a loop and will always ignore the cancellation.

   The most sensible solution would be to allow the user to handle both the return value and the cancellation if they do happen at once. This can be done by subclassing the CancelledError as CancelledWithResultError and raising that instead. If the user code does not handle that exception specifically then the user "chose" to ignore the result. Even if this is not intuitive, it would give the user the control over what really is happening. Right now, the user cannot prefer to handle the cancellation or both.

   Lastly, I may have overlooked something trivial to make this work well. Right now I'm considering replacing all of the asyncio.wait_for constructs with asyncio.wait constructs. I can fully control all tasks and cancellations with that. I've made a simple demonstration of my problem, maybe someone can shed some light onto it.
2. nmatravolgyi

   added

   3.8 (EOL)end of lifeend of life

   3.9 (EOL)end of lifeend of life

   topic-asyncio

   type-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error

   on Mar 3, 2021
3. nmatravolgyi commented on Mar 3, 2021

   

   nmatravolgyimannequin

   on Mar 3, 2021

   MannequinAuthor

   More actions

   I've quickly wanted to create a suitable solution for myself. I made a small library with a asyncio.wait_for()-like function using asyncio.wait(). The prototype worked, so I put together a small project. When I ran tox and realized that this issue with wait_for is only present on py38 and py39 (possibly py310). The wait_for does not get stuck with py36, py37 and pypy3.

   The repo is a little bare bones, but you can run tox after checkout: https://github.com/Traktormaster/wait-for2

   Right now the tests are set-up that they expect wait_for to get stuck so only py38 and py39 passes.

   I'm pretty sure the side-effect of returning the future's result when handling cancellation is not desired. However I'm not sure how to handle it correctly. The repo holds a demo of what I suggested in the beginning of this thread (CancelledWithResultError). It works but it is limited.
4. nmatravolgyi commented on Mar 3, 2021

   

   nmatravolgyimannequin

   on Mar 3, 2021

   Last edited by gvanrossum

   MannequinAuthor

   More actions

   One more thing. I've figured out that I can fix the cancellation around the asyncio.wait_for() with asyncio.shield() like:

   try:
       await asyncio.shield(wf := asyncio.ensure_future(asyncio.wait_for(self.event.wait(), timeout=60.0)))
   except asyncio.CancelledError:
       wf.cancel()
       result = await asyncio.gather(wf, return_exceptions=True)
       # here I know there is a cancellation AND I might have a result as well!
       raise

   However I don't like the idea of writing all that boilerplate for every wait_for usage. I still might be overlooking something, but at least I have adequate workarounds.

   I'm curious what the consensus will be on this issue. I'm certain it should be documented though. Right now there is no mention of ignoring/eating a cancellation.
5. nmatravolgyi commented on May 9, 2021

   

   nmatravolgyimannequin

   on May 9, 2021

   MannequinAuthor

   More actions

   Closing as duplicate, because there was already an issue for this bug: https://bugs.python.org/issue42130
6. nmatravolgyi

   closed this as completedon May 9, 2021
7. nmatravolgyi

   closed this as completedon May 9, 2021
8. ezio-melotti

   transferred this issue fromon Apr 10, 2022