Skip to content

REPL: AttributeError: module __mp_main__ has no attribute is_prime in ProcessPoolExecutor example #132898

Description

@wenming21

Bug report

Bug description:

python version:3.12.9
An error occurred when running as a standalone script.
An error message is reported when running the sample code of the python document:

Traceback (most recent call last):
File "C:\Users\wen\Desktop\test.py", line 32, in
main()
File "C:\Users\wen\Desktop\test.py", line 28, in main
for number, prime in zip(PRIMES, executor.map(is_prime, PRIMES)):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Miniconda3\Lib\concurrent\futures\process.py", line 636, in _chain_from_iterable_of_lists
for element in iterable:
^^^^^^^^
File "C:\Miniconda3\Lib\concurrent\futures_base.py", line 619, in result_iterator
yield _result_or_cancel(fs.pop())
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Miniconda3\Lib\concurrent\futures_base.py", line 317, in _result_or_cancel
return fut.result(timeout)
^^^^^^^^^^^^^^^^^^^
File "C:\Miniconda3\Lib\concurrent\futures_base.py", line 456, in result
return self.__get_result()
^^^^^^^^^^^^^^^^^^^
File "C:\Miniconda3\Lib\concurrent\futures_base.py", line 401, in __get_result
raise self._exception
concurrent.futures.process.BrokenProcessPool: A process in the process pool was terminated abruptly while the future was running or pending.

import concurrent.futures
import math

PRIMES = [
    112272535095293,
    112582705942171,
    112272535095293,
    115280095190773,
    115797848077099,
    1099726899285419]

def is_prime(n):
    if n < 2:
        return False
    if n == 2:
        return True
    if n % 2 == 0:
        return False

    sqrt_n = int(math.floor(math.sqrt(n)))
    for i in range(3, sqrt_n + 1, 2):
        if n % i == 0:
            return False
    return True

def main():
    with concurrent.futures.ProcessPoolExecutor() as executor:
        for number, prime in zip(PRIMES, executor.map(is_prime, PRIMES)):
            print('%d is prime: %s' % (number, prime))

if __name__ == '__main__':
    main()

CPython versions tested on:

3.12

Operating systems tested on:

Windows 10

Linked PRs

Activity

  1. picnixz commented on Apr 25, 2025

    @picnixz
    Member

    Question: did this occur because you hit Ctrl+C or killed the process somehow?

    Nevermind, it occurred on Linux as well with just running the example.

  2. changed the title [-]A process in the process pool was terminated abruptly while the future was running or pending[/-] [+]AttributeError: module `__mp_main__` has no attribute `is_prime` in `ProcessPoolExecutor` example[/+] on Apr 25, 2025
  3. changed the title [-]AttributeError: module `__mp_main__` has no attribute `is_prime` in `ProcessPoolExecutor` example[/-] [+]REPL: AttributeError: module `__mp_main__` has no attribute `is_prime` in `ProcessPoolExecutor` example[/+] on Apr 25, 2025
  4. picnixz commented on Apr 25, 2025

    @picnixz
    Member

    Ok, I only found the issue when the above code is being tested inside the REPL and not as a script (namely python test.py should work but not when I C/C the above code). The best fix for now would be to just indicate in the docs "run it as a standalone script, not inside the REPL", but I don't know the exact reason why (well I can guess: it has something with pickling and probably globals that are not made available because they are only available in the REPL and ProcessPoolExecutor is not able to find the correct "main" module).

    cc @pablogsal @gpshead

  5. picnixz commented on Apr 25, 2025

    @picnixz
    Member

    @wenming21 Your latest edit indicates that the error also occurs when running the example as a standalone script. What is the full traceback? (namely, what's above the "Traceback (most recent call last):"? because I do have other stuff written when I encounter the bug)

  6. StanFromIreland commented on Apr 25, 2025

    @StanFromIreland
    Member

    Why was this closed? @wenming21

  7. StanFromIreland commented on Apr 25, 2025

    @StanFromIreland
    Member

    I can also add that this only occurs on the main branch, on 3.13 it works fine.

  8. picnixz commented on Apr 25, 2025

    @picnixz
    Member

    3.14 has seen changes with the default start method so this could be the cause by the way (at least on Linux)

  9. YvesDup commented on Apr 30, 2025

    @YvesDup
    Contributor

    What's happen when you change the start_method ?
    I am on Mac, with a 3.12.8 version and I ran that script:
    /bin/python3 -i /Users/yves/Desktop/Bugs/gh-132898.py and in REPL, I call main(), I have got errors:

    Ouptput
    multiprocessing.get_start_method() = 'spawn' # I print the current `start_method` and keep only two numbers to check
    112272535095293 is prime: True
    1099726899285419 is prime: False
    >>> main()
    Process SpawnProcess-4:
    Traceback (most recent call last):
      File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/multiprocessing/process.py", line 314, in _bootstrap
        self.run()
      File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/multiprocessing/process.py", line 108, in run
        self._target(*self._args, **self._kwargs)
      File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/concurrent/futures/process.py", line 252, in _process_worker
        call_item = call_queue.get(block=True)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/multiprocessing/queues.py", line 122, in get
        return _ForkingPickler.loads(res)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^
    AttributeError: Can't get attribute 'is_prime' on <module '__main__' (<class '_frozen_importlib.BuiltinImporter'>)>
    Process SpawnProcess-3:
    Traceback (most recent call last):
      File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/multiprocessing/process.py", line 314, in _bootstrap
        self.run()
      File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/multiprocessing/process.py", line 108, in run
        self._target(*self._args, **self._kwargs)
      File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/concurrent/futures/process.py", line 252, in _process_worker
        call_item = call_queue.get(block=True)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/multiprocessing/queues.py", line 122, in get
        return _ForkingPickler.loads(res)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^
    AttributeError: Can't get attribute 'is_prime' on <module '__main__' (<class '_frozen_importlib.BuiltinImporter'>)>
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "/Users/yves/Desktop/Bugs/gh-132898.py", line 31, in main
        for number, prime in zip(PRIMES, executor.map(is_prime, PRIMES)):
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/concurrent/futures/process.py", line 636, in _chain_from_iterable_of_lists
        for element in iterable:
                       ^^^^^^^^
      File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/concurrent/futures/_base.py", line 619, in result_iterator
        yield _result_or_cancel(fs.pop())
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/concurrent/futures/_base.py", line 317, in _result_or_cancel
        return fut.result(timeout)
               ^^^^^^^^^^^^^^^^^^^
      File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/concurrent/futures/_base.py", line 456, in result
        return self.__get_result()
               ^^^^^^^^^^^^^^^^^^^
      File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result
        raise self._exception
    concurrent.futures.process.BrokenProcessPool: A process in the process pool was terminated abruptly while

    if I changed the start_method to forkserver, I have got the same errors.

    And finally if the start method is fork everything is fine.
    /usr/local/bin/python3 -i /Users/yves/Desktop/Bugs/gh-132898.py

    multiprocessing.get_start_method() = 'fork'
    112272535095293 is prime: True
    1099726899285419 is prime: False
    >>> main()
    112272535095293 is prime: True
    1099726899285419 is prime: False
    >>> 
    

    IMO, this issue is also causing by start_method but I don't figure out how REPL is involved.
    And a last remark, if you replace the is_prime function with math.sqrt, the script succeeds in any start_method case .

  10. 27 remaining items

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

3.12only security fixes3.13only security fixes3.14bugs and security fixesdocsDocumentation in the Doc dirstdlibStandard Library Python modules in the Lib/ directorytopic-multiprocessingtype-bugAn unexpected behavior, bug, or error

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions