Skip to content

Running python test suite fails on macOS 10.14.4 with resource.RLIMIT_STACK error #80613

Closed
Closed
Running python test suite fails on macOS 10.14.4 with resource.RLIMIT_STACK error#80613
Assignees
ned-deily
Labels
3.7 (EOL)end of life3.8 (EOL)end of lifeOS-mactype-bugAn unexpected behavior, bug, or error

Description

@ned-deily
ned-deily
opened on Mar 26, 2019
BPO 36432
Nosy @ronaldoussoren, @ned-deily, @methane, @berkerpeksag, @yan12125, @remilapeyre, @dimpase
Superseder
  • bpo-34602: python3 resource.setrlimit strange behaviour under macOS
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/ned-deily'
closed_at = <Date 2019-04-29.19:53:25.019>
created_at = <Date 2019-03-26.07:09:11.590>
labels = ['OS-mac', '3.8', 'type-bug', '3.7']
title = 'Running python test suite fails on macOS 10.14.4 with resource.RLIMIT_STACK error'
updated_at = <Date 2019-07-02.07:57:03.169>
user = 'https://github.com/ned-deily'

    bugs.python.org fields:

    activity = <Date 2019-07-02.07:57:03.169>
actor = 'ned.deily'
assignee = 'ned.deily'
closed = True
closed_date = <Date 2019-04-29.19:53:25.019>
closer = 'ned.deily'
components = ['macOS']
creation = <Date 2019-03-26.07:09:11.590>
creator = 'ned.deily'
dependencies = []
files = []
hgrepos = []
issue_num = 36432
keywords = []
message_count = 9.0
messages = ['338858', '338862', '338865', '338867', '338941', '339307', '340369', '341116', '347120']
nosy_count = 7.0
nosy_names = ['ronaldoussoren', 'ned.deily', 'methane', 'berker.peksag', 'yan12125', 'remi.lapeyre', 'dimpase']
pr_nums = []
priority = 'critical'
resolution = 'duplicate'
stage = 'resolved'
status = 'closed'
superseder = '34602'
type = 'behavior'
url = 'https://bugs.python.org/issue36432'
versions = ['Python 3.7', 'Python 3.8']

    Activity

    1. ned-deily commented on Mar 26, 2019

      @ned-deily
      ned-deily
      on Mar 26, 2019
      MemberAuthor

      After upgrading my first macOS system to the newly released macOS 10.14.4 update, attempts to run the Python test suite via regrtest fail:

      $ /usr/local/bin/python3.7 -m test -uall -j3 -w
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/test/__main__.py", line 2, in <module>
    main()
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/test/libregrtest/main.py", line 640, in main
    Regrtest().main(tests=tests, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/test/libregrtest/main.py", line 586, in main
    self._main(tests, kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/test/libregrtest/main.py", line 607, in _main
    setup_tests(self.ns)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/test/libregrtest/setup.py", line 77, in setup_tests
    resource.setrlimit(resource.RLIMIT_STACK, (newsoft, hard))
ValueError: current limit exceeds maximum limit

      The error is during regrtest initialization when it is trying to increase the process stack size to avoid previously seen problems when running tests. This code has been unchanged for a long time.

      Stepping through the code in the REPL on 10.14.4:
>>> import resource
>>> resource.getrlimit(resource.RLIMIT_STACK)
(8388608, 67104768)
>>> soft, hard = resource.getrlimit(resource.RLIMIT_STACK)
>>> newsoft = min(hard, max(soft, 1024*2048))
>>> newsoft
8388608
>>> resource.setrlimit(resource.RLIMIT_STACK, (newsoft, hard))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: current limit exceeds maximum limit

      The same code run on a macOS system still running 10.14.3 gives the same values from getrlimit but does not fail when calling setrlimit.

      I will investigate further tomorrow; if this is a general problem with macOS 10.14.4, we'll need to provide a workaround and possibly open an incident report with Apple.

    2. ned-deily
      added
      3.7 (EOL)end of life
      3.8 (EOL)end of life
      on Mar 26, 2019
    3. ned-deily
      self-assigned this
      on Mar 26, 2019
    4. ned-deily
      added
      OS-mac
      on Mar 26, 2019

    5. methane commented on Mar 26, 2019

      @methane
      methane
      on Mar 26, 2019
      Member

      My mac is also 10.14.4. I don't have older macOS now. Could someone test this?

      $ ulimit -Sa
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
file size               (blocks, -f) unlimited
max locked memory       (kbytes, -l) unlimited
max memory size         (kbytes, -m) unlimited
open files                      (-n) 4864
pipe size            (512 bytes, -p) 1
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) 1418
virtual memory          (kbytes, -v) unlimited

$ ulimit -Ha
core file size          (blocks, -c) unlimited
data seg size           (kbytes, -d) unlimited
file size               (blocks, -f) unlimited
max locked memory       (kbytes, -l) unlimited
max memory size         (kbytes, -m) unlimited
open files                      (-n) unlimited
pipe size            (512 bytes, -p) 1
stack size              (kbytes, -s) 65532
cpu time               (seconds, -t) unlimited
max user processes              (-u) 2128
virtual memory          (kbytes, -v) unlimited

    6. kakshay21 commented on Mar 26, 2019

      @kakshay21
      kakshay21mannequin
      on Mar 26, 2019
      Mannequin

      I'm on macOS 10.14.3 and running those commands gives me this ->

      ~ ulimit -Sa
      -t: cpu time (seconds) unlimited
      -f: file size (blocks) unlimited
      -d: data seg size (kbytes) unlimited
      -s: stack size (kbytes) 8192
      -c: core file size (blocks) 0
      -v: address space (kbytes) unlimited
      -l: locked-in-memory size (kbytes) unlimited
      -u: processes 709
      -n: file descriptors 4864
      ➜ ~ ulimit -Ha
      -t: cpu time (seconds) unlimited
      -f: file size (blocks) unlimited
      -d: data seg size (kbytes) unlimited
      -s: stack size (kbytes) 65532
      -c: core file size (blocks) unlimited
      -v: address space (kbytes) unlimited
      -l: locked-in-memory size (kbytes) unlimited
      -u: processes 1064
      -n: file descriptors unlimited

    7. methane commented on Mar 26, 2019

      @methane
      methane
      on Mar 26, 2019
      Member

      I created simple program calling setrlimit and it succeeds.
      I confirmed setrlimit argument is exactly same.
      It's very curious why same Python code fails...

      == c code
#include <sys/resource.h>
#include <stdio.h>

int
main(int argc, char *argv[])
{
    struct rlimit rl;
    int err;

    err = getrlimit(RLIMIT_STACK, &rl);
    if (err < 0) {
        perror("getrlimit");
        return err;
    }

    printf("%d, soft=%llu, hard=%llu\n", RLIMIT_STACK, rl.rlim_cur, rl.rlim_max);

    err = setrlimit(RLIMIT_STACK, &rl);
    if (err < 0) {
        perror("setrlimit");
        return err;
    }

    return 0;
}

      == Python code
      import resource
      soft, hard = resource.getrlimit(resource.RLIMIT_STACK)
      print("limits=", soft, hard)
      resource.setrlimit(resource.RLIMIT_STACK, (soft, hard))

      == fails
Traceback (most recent call last):
  File "x.py", line 4, in <module>
    resource.setrlimit(resource.RLIMIT_STACK, (soft, hard))
ValueError: current limit exceeds maximum limit

    8. methane commented on Mar 27, 2019

      @methane
      methane
      on Mar 27, 2019
      Member

      I think this issue is duplicate of https://bugs.python.org/issue34602

    9. remilapeyre commented on Apr 1, 2019

      @remilapeyre
      remilapeyremannequin
      on Apr 1, 2019
      Mannequin

      I confirm that reverting 335ab5b work as a workaround, and does not seem to create regressions.

    10. berkerpeksag commented on Apr 16, 2019

      @berkerpeksag
      berkerpeksag
      on Apr 16, 2019
      Member

      I can also confirm that reverting 335ab5b fixes the problem.

    11. berkerpeksag
      added
      type-bugAn unexpected behavior, bug, or error
      on Apr 16, 2019

    12. ned-deily commented on Apr 29, 2019

      @ned-deily
      ned-deily
      on Apr 29, 2019
      MemberAuthor

      Thanks, Inada-san, for the pointer. Closing this as a duplicate of bpo-34602

    13. ned-deily
      closed this as completedon Apr 29, 2019

    14. ned-deily commented on Jul 2, 2019

      @ned-deily
      ned-deily
      on Jul 2, 2019
      MemberAuthor

      See updated bpo-34602 discussion for why reverting 335ab5b causes other problems and for a different fix for this issue.

    15. ezio-melotti
      transferred this issue fromon Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

    Metadata

    Metadata

    Assignees

    Labels

    3.7 (EOL)end of life3.8 (EOL)end of lifeOS-mactype-bugAn unexpected behavior, bug, or error

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions