concurrent.futures.wait returns cancelled as "not done" #109934
Copy link
Copy link
Open
Labels
stdlibStandard Library Python modules in the Lib/ directoryStandard Library Python modules in the Lib/ directorytype-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error
Description
Bug report
Bug description:
According to the documentation for concurrent.futures.wait, it:
Returns a named 2-tuple of sets. The first set, named done, contains the futures that completed (finished or cancelled futures) before the wait completed. The second set, named not_done, contains the futures that did not complete (pending or running futures).
In this example, however, it returns cancelled futures in not_done. Note the wait=True in the preceding executor.shutdown call, which should ensure futures are not cancelled between wait and print. task calls sleep to ensure at least one thread is pending when executor.shutdown is called, even on especially performant systems.
from concurrent.futures import ThreadPoolExecutor, wait
from time import sleep
def task():
sleep(1)
executor = ThreadPoolExecutor(max_workers=1)
futures = [executor.submit(task), executor.submit(task)]
executor.shutdown(wait=True, cancel_futures=True)
wait_return = wait(futures, timeout=0)
print(wait_return.not_done)Considerations
- If we instead call
waitwithtimeout=None, the script hangs indefinitely. - If we replace the call to
executor.shutdownwithfutures[1].cancel()(leavetimeout=0), the cancelled future is similarly included innot_done. We can addprint(futures[1].cancelled())to ensure the cancellation is successful before the call towait. - Replacing both, however, produces the expected result.
I couldn't find an existing report; apologies if I missed it.
CPython versions tested on:
3.11
Operating systems tested on:
Linux, macOS
Linked PRs
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
stdlibStandard Library Python modules in the Lib/ directoryStandard Library Python modules in the Lib/ directorytype-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error
Activity
duaneg commentedon May 24, 2025
This appears to be a bug in
concurrent.futures.ThreadPoolExecutor: when the pool is shutdown it cancels all pending work items, however it does not callset_running_or_notify_cancelon the future, so it stays in theCANCELLEDstate instead of being moved to theCANCELLED_AND_NOTIFIEDstate.Aside from the task being incorrectly reported as "not done", this means code that is blocking waiting on the future to complete will never be awakened (e.g. if passing
wait=Trueto shutdown, as the OP notes, or if usingas_completedetc).pythongh-109934: notify cancelled futures on thread pool shutdown
Merge branch 'main' into pythongh-109934
pythongh-136655: ensure cancelled futures are notified on process poo…