Skip to content

test_getcwd_long_pathnames (in test_posix) kills NetBSD #59969

Closed
Closed
test_getcwd_long_pathnames (in test_posix) kills NetBSD#59969
Assignees
tpn
Labels
type-bugAn unexpected behavior, bug, or error

Description

@tpn
tpn
opened on Aug 22, 2012
BPO 15765
Nosy @vstinner, @tpn, @skrah
Files
  • getcwd_netbsd.patch
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/tpn'
closed_at = <Date 2012-08-30.05:32:25.552>
created_at = <Date 2012-08-22.14:12:15.496>
labels = ['type-bug']
title = 'test_getcwd_long_pathnames (in test_posix) kills NetBSD'
updated_at = <Date 2012-08-30.05:32:25.551>
user = 'https://github.com/tpn'

    bugs.python.org fields:

    activity = <Date 2012-08-30.05:32:25.551>
actor = 'trent'
assignee = 'trent'
closed = True
closed_date = <Date 2012-08-30.05:32:25.552>
closer = 'trent'
components = []
creation = <Date 2012-08-22.14:12:15.496>
creator = 'trent'
dependencies = []
files = ['26987']
hgrepos = []
issue_num = 15765
keywords = ['patch']
message_count = 7.0
messages = ['168885', '168902', '168994', '169012', '169027', '169119', '169375']
nosy_count = 5.0
nosy_names = ['vstinner', 'trent', 'skrah', 'neologix', 'python-dev']
pr_nums = []
priority = 'normal'
resolution = 'duplicate'
stage = None
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue15765'
versions = ['Python 2.7']

    Activity

    1. tpn commented on Aug 22, 2012

      @tpn
      tpn
      on Aug 22, 2012
      MemberAuthor

      The 2.7 builds of both my NetBSD slaves keep hanging on test_posix (and eventually time out after an hour). Running the test manually:

      netbsd51-x86-1$ pwd
      /home/cpython/2.7
      netbsd51-x86-1$ b/test/regrtest.py -uall -rwW -v test_posix <
      == CPython 2.7.3+ (2.7:c1c45755397b, Aug 22 2012, 14:05:02) [GCC 4.1.3 20080704 prerelease (NetBSD nb2 2008112)]
      == NetBSD-5.1.2-i386-32bit-ELF little-endian
      == /home/cpython/2.7/build/test_python_10630
      Testing with flags: sys.flags(debug=0, py3k_warning=1, division_warning=1, division_new=0, inspect=0, interactive=0, optimize=0, dont_write_bytecode=0, no_user_site=0, no_site=0, ignore_environment=1, tabcheck=2, verbose=0, unicode=0, bytes_warning=0, hash_randomization=1)
      Using random seed 2040018
      test_posix
      testNoArgFunctions (test.test_posix.PosixTester) ... ok
      test_access (test.test_posix.PosixTester) ... ok
      test_chdir (test.test_posix.PosixTester) ... ok
      test_chflags (test.test_posix.PosixTester) ... ok
      test_chown (test.test_posix.PosixTester) ... ok
      test_confstr (test.test_posix.PosixTester) ... ok
      test_dup (test.test_posix.PosixTester) ... ok
      test_dup2 (test.test_posix.PosixTester) ... ok
      test_fchown (test.test_posix.PosixTester) ... ok
      test_fdopen (test.test_posix.PosixTester) ... ok
      test_fstat (test.test_posix.PosixTester) ... ok
      test_fstatvfs (test.test_posix.PosixTester) ... ok
      test_ftruncate (test.test_posix.PosixTester) ... ok
      test_getcwd_long_pathnames (test.test_posix.PosixTester) ... load: 0.50 cmd: python 10630 [runnable] 0.27u 13.07s 46% 10976k
      load: 0.87 cmd: python 10630 [runnable] 0.27u 94.24s 98% 10976k
      ^C^C

      Placeholder issue at the moment.

      ....although this looks pretty dodgy:

                          os.chdir(dirname)
                    try:
                        os.getcwd()
                        if current_path_length < 4099:
                            _create_and_do_getcwd(dirname, current_path_length + len(dirname) + 1)
    2. tpn
      self-assigned this
      on Aug 22, 2012
    3. tpn
      added
      type-bugAn unexpected behavior, bug, or error
      on Aug 22, 2012

    4. skrah commented on Aug 22, 2012

      @skrah
      skrahmannequin
      on Aug 22, 2012
      Mannequin

      The timeouts on NetBSD and OpenBSD are very likely related to
      bugs in the user-space thread libraries. See also bpo-8714.

      I would recommend running these bots --without-threads.

    5. neologix commented on Aug 24, 2012

      @neologix
      neologixmannequin
      on Aug 24, 2012
      Mannequin

      This doesn't look thread-related:
      """
      load: 0.50 cmd: python 10630 [runnable] 0.27u 13.07s 46% 10976k
      load: 0.87 cmd: python 10630 [runnable] 0.27u 94.24s 98% 10976k
      """

      See the huge system time?
      This probably means that the kernel is struggling with path name
      resolution, although that's surprising because 4099 is a reasonable
      limit.
      Do you have something like strace or truss on NetBSD?
      The first thing to try could be to lower the limit, and see what
      happens, but it's really likely a FS scalibility limit you're hitting
      (altough, once again, that's really surprising given the reasonable
      limit). Are you running those under VMs?

    6. tpn commented on Aug 24, 2012

      @tpn
      tpn
      on Aug 24, 2012
      MemberAuthor

      Hacked the test to print out info each loop:

      test_getcwd_long_pathnames (test.test_posix.PosixTester) ... [0/0] getcwd-test-directory-0123456789abcdef-01234567890abcdef
      [1/57] getcwd-test-directory-0123456789abcdef-01234567890abcdef
      [2/114] getcwd-test-directory-0123456789abcdef-01234567890abcdef
      [3/171] getcwd-test-directory-0123456789abcdef-01234567890abcdef
      [4/228] getcwd-test-directory-0123456789abcdef-01234567890abcdef
      [5/285] getcwd-test-directory-0123456789abcdef-01234567890abcdef
      [6/342] getcwd-test-directory-0123456789abcdef-01234567890abcdef
      [7/399] getcwd-test-directory-0123456789abcdef-01234567890abcdef
      [8/456] getcwd-test-directory-0123456789abcdef-01234567890abcdef
      [9/513] getcwd-test-directory-0123456789abcdef-01234567890abcdef
      [10/570] getcwd-test-directory-0123456789abcdef-01234567890abcdef
      [11/627] getcwd-test-directory-0123456789abcdef-01234567890abcdef
      [12/684] getcwd-test-directory-0123456789abcdef-01234567890abcdef
      [13/741] getcwd-test-directory-0123456789abcdef-01234567890abcdef
      [14/798] getcwd-test-directory-0123456789abcdef-01234567890abcdef
      [15/855] getcwd-test-directory-0123456789abcdef-01234567890abcdef
      [16/912] getcwd-test-directory-0123456789abcdef-01234567890abcdef
      [17/969] getcwd-test-directory-0123456789abcdef-01234567890abcdef
      [18/1026] getcwd-test-directory-0123456789abcdef-01234567890abcdef
      [19/1083] getcwd-test-directory-0123456789abcdef-01234567890abcdef
      [20/1140] getcwd-test-directory-0123456789abcdef-01234567890abcdef
      [21/1197] getcwd-test-directory-0123456789abcdef-01234567890abcdef
      [22/1254] getcwd-test-directory-0123456789abcdef-01234567890abcdef
      [23/1311] getcwd-test-directory-0123456789abcdef-01234567890abcdef
      [24/1368] getcwd-test-directory-0123456789abcdef-01234567890abcdef
      [25/1425] getcwd-test-directory-0123456789abcdef-01234567890abcdef
      [26/1482] getcwd-test-directory-0123456789abcdef-01234567890abcdef
      [27/1539] getcwd-test-directory-0123456789abcdef-01234567890abcdef
      [28/1596] getcwd-test-directory-0123456789abcdef-01234567890abcdef
      [29/1653] getcwd-test-directory-0123456789abcdef-01234567890abcdef
      [30/1710] getcwd-test-directory-0123456789abcdef-01234567890abcdef
      [31/1767] getcwd-test-directory-0123456789abcdef-01234567890abcdef
      [32/1824] getcwd-test-directory-0123456789abcdef-01234567890abcdef
      [33/1881] getcwd-test-directory-0123456789abcdef-01234567890abcdef
      [34/1938] getcwd-test-directory-0123456789abcdef-01234567890abcdef
      [35/1995] getcwd-test-directory-0123456789abcdef-01234567890abcdef
      [36/2052] getcwd-test-directory-0123456789abcdef-01234567890abcdef
      [37/2109] getcwd-test-directory-0123456789abcdef-01234567890abcdef
      [38/2166] getcwd-test-directory-0123456789abcdef-01234567890abcdef
      [39/2223] getcwd-test-directory-0123456789abcdef-01234567890abcdef
      [40/2280] getcwd-test-directory-0123456789abcdef-01234567890abcdef
      [41/2337] getcwd-test-directory-0123456789abcdef-01234567890abcdef
      [42/2394] getcwd-test-directory-0123456789abcdef-01234567890abcdef
      [43/2451] getcwd-test-directory-0123456789abcdef-01234567890abcdef
      [44/2508] getcwd-test-directory-0123456789abcdef-01234567890abcdef
      [45/2565] getcwd-test-directory-0123456789abcdef-01234567890abcdef
      [46/2622] getcwd-test-directory-0123456789abcdef-01234567890abcdef
      [47/2679] getcwd-test-directory-0123456789abcdef-01234567890abcdef
      [48/2736] getcwd-test-directory-0123456789abcdef-01234567890abcdef
      [49/2793] getcwd-test-directory-0123456789abcdef-01234567890abcdef
      [50/2850] getcwd-test-directory-0123456789abcdef-01234567890abcdef
      [51/2907] getcwd-test-directory-0123456789abcdef-01234567890abcdef
      [52/2964] getcwd-test-directory-0123456789abcdef-01234567890abcdef
      [53/3021] getcwd-test-directory-0123456789abcdef-01234567890abcdef
      [54/3078] getcwd-test-directory-0123456789abcdef-01234567890abcdef
      [55/3135] getcwd-test-directory-0123456789abcdef-01234567890abcdef
      [56/3192] getcwd-test-directory-0123456789abcdef-01234567890abcdef
      [57/3249] getcwd-test-directory-0123456789abcdef-01234567890abcdef
      [58/3306] getcwd-test-directory-0123456789abcdef-01234567890abcdef
      [59/3363] getcwd-test-directory-0123456789abcdef-01234567890abcdef
      [60/3420] getcwd-test-directory-0123456789abcdef-01234567890abcdef
      [61/3477] getcwd-test-directory-0123456789abcdef-01234567890abcdef
      [62/3534] getcwd-test-directory-0123456789abcdef-01234567890abcdef
      [63/3591] getcwd-test-directory-0123456789abcdef-01234567890abcdef
      [64/3648] getcwd-test-directory-0123456789abcdef-01234567890abcdef
      [65/3705] getcwd-test-directory-0123456789abcdef-01234567890abcdef
      [66/3762] getcwd-test-directory-0123456789abcdef-01234567890abcdef
      [67/3819] getcwd-test-directory-0123456789abcdef-01234567890abcdef
      [68/3876] getcwd-test-directory-0123456789abcdef-01234567890abcdef
      [69/3933] getcwd-test-directory-0123456789abcdef-01234567890abcdef
      [70/3990] getcwd-test-directory-0123456789abcdef-01234567890abcdef
      ^C^C
      load: 1.08 cmd: python 4486 [runnable] 0.07u 14.38s 40% 10012k
      load: 1.08 cmd: python 4486 [runnable] 0.07u 14.91s 40% 10012k
      load: 1.08 cmd: python 4486 [runnable] 0.07u 15.24s 42% 10012k
      load: 1.08 cmd: python 4486 [runnable] 0.07u 15.40s 42% 10012k
      Terminated

      (Had to kill it from a different process, mind you; the ctrl-Cs never get caught.)

      Yeah, it's a VM. Also, it looks like 3.x uses 1027 instead of 4099.

      It's definitely odd that it ends up spinning in the kernel at ~3990. I'll do some more digging.

    7. tpn commented on Aug 24, 2012

      @tpn
      tpn
      on Aug 24, 2012
      MemberAuthor

      Looks like this is a duplicate of bpo-9185. The fix used for that just needs to be extended to cover NetBSD as well. See proposed patch. I'll commit this to 2.7 in a day or two if there are no objections.

      Antoine: added you as nosy as you were involved with the original patch.

    8. skrah commented on Aug 25, 2012

      @skrah
      skrahmannequin
      on Aug 25, 2012
      Mannequin

      I completely forgot about that issue. The patch looks good to me.

    9. python-dev commented on Aug 29, 2012

      @python-dev
      python-devmannequin
      on Aug 29, 2012
      Mannequin

      New changeset bfa5d0ddfbeb by Trent Nelson in branch '2.7':
      Issue bpo-15765: Fix quirky NetBSD getcwd() behaviour.
      http://hg.python.org/cpython/rev/bfa5d0ddfbeb

    10. tpn
      closed this as completedon Aug 30, 2012
    11. ezio-melotti
      transferred this issue fromon Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

    Metadata

    Metadata

    Assignees

    Labels

    type-bugAn unexpected behavior, bug, or error

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions