Skip to content

test_multiprocessing.test_notify_all() hangs on "AMD64 Snow Leopard 02 03.x" #57774

Closed
Closed
test_multiprocessing.test_notify_all() hangs on "AMD64 Snow Leopard 02 03.x"#57774
Labels
stdlibStandard Library Python modules in the Lib/ directory

Description

@vstinner
vstinner
opened on Dec 9, 2011
BPO 13565
Nosy @vstinner
Files
  • test_backlog.py
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = <Date 2012-01-02.12:34:53.270>
created_at = <Date 2011-12-09.12:58:58.571>
labels = ['library']
title = 'test_multiprocessing.test_notify_all() hangs on "AMD64 Snow Leopard 02 03.x"'
updated_at = <Date 2012-01-02.12:34:53.269>
user = 'https://github.com/vstinner'

    bugs.python.org fields:

    activity = <Date 2012-01-02.12:34:53.269>
actor = 'neologix'
assignee = 'none'
closed = True
closed_date = <Date 2012-01-02.12:34:53.270>
closer = 'neologix'
components = ['Library (Lib)']
creation = <Date 2011-12-09.12:58:58.571>
creator = 'vstinner'
dependencies = []
files = ['24048']
hgrepos = []
issue_num = 13565
keywords = []
message_count = 7.0
messages = ['149089', '149824', '150120', '150189', '150190', '150191', '150453']
nosy_count = 3.0
nosy_names = ['vstinner', 'neologix', 'python-dev']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue13565'
versions = ['Python 3.3']

    Activity

    1. vstinner commented on Dec 9, 2011

      @vstinner
      vstinner
      on Dec 9, 2011
      MemberAuthor

      [333/363] test_multiprocessing
      Timeout (1:00:00)!
      Thread 0x0000000112d0b000:
      File "/Users/buildbot/buildarea/3.x.parc-snowleopard-1/build/Lib/multiprocessing/connection.py", line 411 in _recv
      File "/Users/buildbot/buildarea/3.x.parc-snowleopard-1/build/Lib/multiprocessing/connection.py", line 432 in _recv_bytes
      File "/Users/buildbot/buildarea/3.x.parc-snowleopard-1/build/Lib/multiprocessing/connection.py", line 275 in recv
      File "/Users/buildbot/buildarea/3.x.parc-snowleopard-1/build/Lib/multiprocessing/managers.py", line 758 in _callmethod
      File "/Users/buildbot/buildarea/3.x.parc-snowleopard-1/build/Lib/multiprocessing/managers.py", line 994 in wait
      File "/Users/buildbot/buildarea/3.x.parc-snowleopard-1/build/Lib/test/test_multiprocessing.py", line 734 in f
      File "/Users/buildbot/buildarea/3.x.parc-snowleopard-1/build/Lib/threading.py", line 682 in run
      File "/Users/buildbot/buildarea/3.x.parc-snowleopard-1/build/Lib/threading.py", line 729 in _bootstrap_inner
      File "/Users/buildbot/buildarea/3.x.parc-snowleopard-1/build/Lib/threading.py", line 702 in _bootstrap

      Thread 0x0000000112908000:
      File "/Users/buildbot/buildarea/3.x.parc-snowleopard-1/build/Lib/multiprocessing/connection.py", line 411 in _recv
      File "/Users/buildbot/buildarea/3.x.parc-snowleopard-1/build/Lib/multiprocessing/connection.py", line 432 in _recv_bytes
      File "/Users/buildbot/buildarea/3.x.parc-snowleopard-1/build/Lib/multiprocessing/connection.py", line 275 in recv
      File "/Users/buildbot/buildarea/3.x.parc-snowleopard-1/build/Lib/multiprocessing/managers.py", line 758 in _callmethod
      File "/Users/buildbot/buildarea/3.x.parc-snowleopard-1/build/Lib/multiprocessing/managers.py", line 994 in wait
      File "/Users/buildbot/buildarea/3.x.parc-snowleopard-1/build/Lib/test/test_multiprocessing.py", line 734 in f
      File "/Users/buildbot/buildarea/3.x.parc-snowleopard-1/build/Lib/threading.py", line 682 in run
      File "/Users/buildbot/buildarea/3.x.parc-snowleopard-1/build/Lib/threading.py", line 729 in _bootstrap_inner
      File "/Users/buildbot/buildarea/3.x.parc-snowleopard-1/build/Lib/threading.py", line 702 in _bootstrap

      Thread 0x00007fff7022ccc0:
      File "/Users/buildbot/buildarea/3.x.parc-snowleopard-1/build/Lib/multiprocessing/connection.py", line 411 in _recv
      File "/Users/buildbot/buildarea/3.x.parc-snowleopard-1/build/Lib/multiprocessing/connection.py", line 432 in _recv_bytes
      File "/Users/buildbot/buildarea/3.x.parc-snowleopard-1/build/Lib/multiprocessing/connection.py", line 275 in recv
      File "/Users/buildbot/buildarea/3.x.parc-snowleopard-1/build/Lib/multiprocessing/managers.py", line 758 in _callmethod
      File "/Users/buildbot/buildarea/3.x.parc-snowleopard-1/build/Lib/multiprocessing/managers.py", line 982 in acquire
      File "/Users/buildbot/buildarea/3.x.parc-snowleopard-1/build/Lib/test/test_multiprocessing.py", line 833 in test_notify_all
      File "/Users/buildbot/buildarea/3.x.parc-snowleopard-1/build/Lib/unittest/case.py", line 385 in _executeTestPart
      File "/Users/buildbot/buildarea/3.x.parc-snowleopard-1/build/Lib/unittest/case.py", line 440 in run
      File "/Users/buildbot/buildarea/3.x.parc-snowleopard-1/build/Lib/unittest/case.py", line 492 in __call__
      File "/Users/buildbot/buildarea/3.x.parc-snowleopard-1/build/Lib/unittest/suite.py", line 105 in run
      File "/Users/buildbot/buildarea/3.x.parc-snowleopard-1/build/Lib/unittest/suite.py", line 67 in __call__
      File "/Users/buildbot/buildarea/3.x.parc-snowleopard-1/build/Lib/unittest/suite.py", line 105 in run
      File "/Users/buildbot/buildarea/3.x.parc-snowleopard-1/build/Lib/unittest/suite.py", line 67 in __call__
      File "/Users/buildbot/buildarea/3.x.parc-snowleopard-1/build/Lib/unittest/suite.py", line 105 in run
      File "/Users/buildbot/buildarea/3.x.parc-snowleopard-1/build/Lib/unittest/suite.py", line 67 in __call__
      File "/Users/buildbot/buildarea/3.x.parc-snowleopard-1/build/Lib/unittest/runner.py", line 168 in run
      File "/Users/buildbot/buildarea/3.x.parc-snowleopard-1/build/Lib/test/support.py", line 1368 in _run_suite
      File "/Users/buildbot/buildarea/3.x.parc-snowleopard-1/build/Lib/test/support.py", line 1402 in run_unittest
      File "/Users/buildbot/buildarea/3.x.parc-snowleopard-1/build/Lib/test/test_multiprocessing.py", line 2392 in test_main
      File "/Users/buildbot/buildarea/3.x.parc-snowleopard-1/build/Lib/test/regrtest.py", line 1221 in runtest_inner
      File "/Users/buildbot/buildarea/3.x.parc-snowleopard-1/build/Lib/test/regrtest.py", line 907 in runtest
      File "/Users/buildbot/buildarea/3.x.parc-snowleopard-1/build/Lib/test/regrtest.py", line 710 in main
      File "/Users/buildbot/buildarea/3.x.parc-snowleopard-1/build/Lib/test/main.py", line 13 in <module>
      File "/Users/buildbot/buildarea/3.x.parc-snowleopard-1/build/Lib/runpy.py", line 73 in _run_code
      File "/Users/buildbot/buildarea/3.x.parc-snowleopard-1/build/Lib/runpy.py", line 160 in _run_module_as_main
      make: *** [buildbottest] Error 1

      command timed out: 3900 seconds without output, attempting to kill
      program finished with exit code 2
      elapsedTime=9486.934561

    2. vstinner
      added
      stdlibStandard Library Python modules in the Lib/ directory
      on Dec 9, 2011

    3. neologix commented on Dec 19, 2011

      @neologix
      neologixmannequin
      on Dec 19, 2011
      Mannequin

      I think this could be due to the multiprocessing manager's server socket backlog value, which is a little too low: by default, it's set to 5, and the tests launch up to 3 threads and 3 processes in parallel, so if we're unlucky with the scheduling, we could get some ECONNREFUSED.
      Unless otherwise specified, the server uses Unix domain sockets: on Linux, when the server's socket backlog is full, connect() blocks, which could explain why it doesn't happen on Linux. It would be nice to check the behavior in case of socket backlog full on affected OSes (for example OS X or FreeBSD).

      Here's a run on Linux:
      """
      $ ./python ~/test_backlog.py
      0
      1
      2
      3
      4
      [blocks]
      """

      If we get ECONNREFUSED on OS X or FreeBSD, then there's a chance it's the culprit. If not, well, no idea what's going on :-)

    4. neologix commented on Dec 22, 2011

      @neologix
      neologixmannequin
      on Dec 22, 2011
      Mannequin

      Victor, could you try the attached script on FreeBSD, to see if you get ECONNREFUSED?

    5. vstinner commented on Dec 23, 2011

      @vstinner
      vstinner
      on Dec 23, 2011
      MemberAuthor

      Victor, could you try the attached script on FreeBSD,
      to see if you get ECONNREFUSED?

      Yes, I get a ECONNREFUSED. I tested backlog.py on FreeBSD 8.2.

    6. python-dev commented on Dec 23, 2011

      @python-dev
      python-devmannequin
      on Dec 23, 2011
      Mannequin

      New changeset 94494a779c20 by Charles-François Natali in branch '2.7':
      Issue bpo-13565: Increase multiprocessing's server socket backlog, to avoid
      http://hg.python.org/cpython/rev/94494a779c20

      New changeset 9b99adef3c78 by Charles-François Natali in branch '3.2':
      Issue bpo-13565: Increase multiprocessing's server socket backlog, to avoid
      http://hg.python.org/cpython/rev/9b99adef3c78

      New changeset 29cad1ac828c by Charles-François Natali in branch 'default':
      Issue bpo-13565: Increase multiprocessing's server socket backlog, to avoid
      http://hg.python.org/cpython/rev/29cad1ac828c

    7. neologix commented on Dec 23, 2011

      @neologix
      neologixmannequin
      on Dec 23, 2011
      Mannequin

      Yes, I get a ECONNREFUSED. I tested backlog.py on FreeBSD 8.2.

      Thanks.
      I bumped the backlog, I hope it will fix this.
      We can leave this report open for a couple days, to see how the buildbots behave.

    8. neologix commented on Jan 2, 2012

      @neologix
      neologixmannequin
      on Jan 2, 2012
      Mannequin

      Alright, it seems to be fixed.
      We can still reopen if this happens again.

    9. neologix
      closed this as completedon Jan 2, 2012
    10. ezio-melotti
      transferred this issue fromon Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

    Metadata

    Metadata

    Assignees

    No one assigned

      Labels

      stdlibStandard Library Python modules in the Lib/ directory

      Projects

      No projects

        Milestone

        No milestone

        Relationships

        None yet

        Development

        No branches or pull requests

        Issue actions