Repository navigation
InterpreterPoolExecutor Failure When __name__ == '__main__': Idiom Not Used #136659
Description
Activity
Does this supersede #136603?
I think subinterpreters are trickier than that, consider this example:>>> from concurrent import futures, interpreters >>> from math import ceil >>> >>> def a(): ... return ceil(0) >>> >>> interp = interpreters.create() >>> interp.call_in_thread(a) ... concurrent.interpreters.NotShareableError: func not shareable
You can make it work just by moving the import into the function itself.
Docs on multiprocessing you are referring to, say:
However, global variables which are just module level constants cause no problems.
That seems false with subinterpreters. We should define a preferred way of handling these cases (maybe there are even more of them).P.s. there's an issue #136603 regarding this problem.
Upd:
if __name__ == "__main__"solves this problem too, there was just an issue with my tests.- changed the title
[-]InterpreterPoolExecutor Failure When "__name__ == '__main__:' Idiom Not Used[/-][+]InterpreterPoolExecutor Failure When "`__name__` == '`__main__`:" Idiom Not Used[/+]on Jul 14, 2025 - changed the title
[-]InterpreterPoolExecutor Failure When "`__name__` == '`__main__`:" Idiom Not Used[/-][+]InterpreterPoolExecutor Failure When `__name__ == __main__:` Idiom Not Used[/+]on Jul 14, 2025 - changed the title
[-]InterpreterPoolExecutor Failure When `__name__ == __main__:` Idiom Not Used[/-][+]InterpreterPoolExecutor Failure When `__name__ == '__main__'`: Idiom Not Used[/+]on Jul 14, 2025 Interesting, it's fine at the main branch as you said but when I execute the same script at 3.14b3 which is installed by uv it's fails :(
Traceback (most recent call last): File "/Users/xxxx/.local/share/uv/python/cpython-3.14-macos-aarch64-none/lib/python3.14/concurrent/futures/interpreter.py", line 155, in run self._exec(script) ~~~~~~~~~~^^^^^^^^ File "/Users/xxxx/.local/share/uv/python/cpython-3.14-macos-aarch64-none/lib/python3.14/concurrent/futures/interpreter.py", line 111, in _exec raise ExecutionFailed(excinfo) concurrent.futures.interpreter.ExecutionFailed: AttributeError: module '__main__' has no attribute 'fib' Uncaught in the interpreter: Traceback (most recent call last): File "<script>", line 1, in <module> File "/Users/xxxx/.local/share/uv/python/cpython-3.14-macos-aarch64-none/lib/python3.14/concurrent/futures/interpreter.py", line 94, in _call_pickled fn, args, kwargs = pickle.loads(pickled) ~~~~~~~~~~~~^^^^^^^^^ AttributeError: module '__main__' has no attribute 'fib' The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/Users/xxxx/oss/cpython/op.py", line 16, in <module> data = future.result() File "/Users/xxxx/.local/share/uv/python/cpython-3.14-macos-aarch64-none/lib/python3.14/concurrent/futures/_base.py", line 443, in result return self.__get_result() ~~~~~~~~~~~~~~~~~^^ File "/Users/xxxx/.local/share/uv/python/cpython-3.14-macos-aarch64-none/lib/python3.14/concurrent/futures/_base.py", line 395, in __get_result raise self._exception File "/Users/xxxx/.local/share/uv/python/cpython-3.14-macos-aarch64-none/lib/python3.14/concurrent/futures/thread.py", line 86, in run result = ctx.run(self.task) File "/Users/xxxx/.local/share/uv/python/cpython-3.14-macos-aarch64-none/lib/python3.14/concurrent/futures/interpreter.py", line 180, in run raise exc from exc_wrapper AttributeError: module '__main__' has no attribute 'fib'But let me test with 3.14 branch right now.
Ah okay.
- main branch (o)
- 3.14b4 tag (o)
- 3.14b3 tag (x)
So seems like it was fixed between 3.14b3 and 3.14b4. I met this issue even with that installed by UV 3.14 (currently 3.14b3), so I thought that this was overall a failure to execute, but it looks not now.
The solution here is probably the same: document that users should apply if name == 'main':. We may also be able to accommodate this in the code somewhat, since we know when we are running the script in the other interpreter. However, there's a limit to how much we can do so without introducing nontrivial complexity.
I think a good approach for 3.14 would be to update the documentation or improve the error message to suggest using the
__name__ == '__main__'idiom to reduce confusion.Reacted by Eric Snow, Ilya Solin and Xuanteng HuangThis is a duplicate of #136603. Sorry I didn't see it first. Looks the other was already closed so we can continue here.
Metadata
Metadata
Assignees
Projects
- Status
- Status
Bug report
Bug description:
From @corona10:
This fails with:
The failure goes away if the "executable" part of the script is wrapped in an
if __name__ == '__main__':block.This is because of how
Interpreter.call()sends functions between interpreters. It mostly pickles them. On the other side, functions are looked up by name in the defining module. For the__main__module, this is a problem.The original script only runs in the main interpreter, so the function will not be found in the other interpreter's
__main__module. We must run the script in the other interpreter, but without polluting__main__and avoiding executing anything but the desired function definition. We partially solve this by callingrunpy.run_path()with a different module name than__main__.multiprocessingfaces a similar situation. See the "Safe importing of main module" entry in the https://docs.python.org/3/library/multiprocessing.html#the-spawn-and-forkserver-start-methods section.The solution here is probably the same: document that users should apply
if __name__ == '__main__':. We may also be able to accommodate this in the code somewhat, since we know when we are running the script in the other interpreter. However, there's a limit to how much we can do so without introducing nontrivial complexity.CPython versions tested on:
CPython main branch
Operating systems tested on:
No response