Skip to content

test_run_fileexflags() crash on PPC64LE RHEL7 LTO + PGO 3.x #118422

Closed
Closed
test_run_fileexflags() crash on PPC64LE RHEL7 LTO + PGO 3.x#118422
Labels
testsTests in the Lib/test dirtype-bugAn unexpected behavior, bug, or error

Description

@vstinner
vstinner
opened on Apr 30, 2024

Bug report

build: https://buildbot.python.org/all/#/builders/43/builds/5305/steps/5/logs/stdio

Logs:

./python -E  -m test --slow-ci --timeout=1200 -j2 --junit-xml test-results.xml -j10 
+ ./python -u -W default -bb -E -m test --slow-ci --timeout=1200 -j2 --junit-xml test-results.xml -j10 --dont-add-python-opts
== CPython 3.13.0a6+ (heads/main:8b56d82, Apr 29 2024, 18:49:30) [GCC 8.3.1 20190311 (Red Hat 8.3.1-3)]
== Linux-3.10.0-1160.114.2.el7.ppc64le-ppc64le-with-glibc2.17 little-endian
== Python build: release LTO+PGO
== cwd: /home/buildbot/buildarea/3.x.cstratak-RHEL7-ppc64le.lto-pgo/build/build/test_python_worker_24730æ
== CPU count: 8
== encodings: locale=UTF-8 FS=utf-8
== resources: all

(...)

test_atomic_load_store_uint64 (test.test_capi.test_pyatomic.PyAtomicTests.test_atomic_load_store_uint64) ... ok
test_atomic_load_store_uint8 (test.test_capi.test_pyatomic.PyAtomicTests.test_atomic_load_store_uint8) ... ok
test_atomic_load_store_uintptr (test.test_capi.test_pyatomic.PyAtomicTests.test_atomic_load_store_uintptr) ... ok
test_atomic_release_acquire (test.test_capi.test_pyatomic.PyAtomicTests.test_atomic_release_acquire) ... ok
test_run_fileexflags (test.test_capi.test_run.CAPITest.test_run_fileexflags) ...

Fatal Python error: Segmentation fault
Current thread 0x00003fff876747f0 (most recent call first):
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL7-ppc64le.lto-pgo/build/Lib/test/test_capi/test_run.py", line 72 in run
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL7-ppc64le.lto-pgo/build/Lib/test/test_capi/test_run.py", line 78 in test_run_fileexflags
  (...)

Extension modules: _testinternalcapi, _testcapi, _testlimitedcapi, _testmultiphase, _testsinglephase (total: 5)

Linked PRs

Activity

  1. vstinner
    added
    type-bugAn unexpected behavior, bug, or error
    testsTests in the Lib/test dir
    on Apr 30, 2024

  2. vstinner commented on Apr 30, 2024

    @vstinner
    vstinner
    on Apr 30, 2024
    MemberAuthor

    The bug can be reproduced just with these 2 tests:

    $ cat bisect8
test.test_capi.test_misc.SubinterpreterTest.test_configured_settings
test.test_capi.test_run.CAPITest.test_run_fileexflags
$ ./python -m test test_capi -v --matchfile=bisect8

    Sometimes, I get the glibc error message:

    Fatal error: glibc detected an invalid stdio handle

  3. vstinner commented on Apr 30, 2024

    @vstinner
    vstinner
    on Apr 30, 2024
    MemberAuthor

    The crash is more likely if PYC files are removed:

    rm -rf ./Lib/__pycache__ ./Lib/test/test_capi/__pycache__

  4. vstinner commented on Apr 30, 2024

    @vstinner
    vstinner
    on Apr 30, 2024
    MemberAuthor

    I cannot reproduce the issue with gcc -O0 and gcc -Og, only with a release build: gcc -O3.

    This buildbot uses an old GCC version: gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-44).

  5. serhiy-storchaka commented on Apr 30, 2024

    @serhiy-storchaka
    serhiy-storchaka
    on Apr 30, 2024
    Member

    If it is only reproducible with a release build, would the following patch help?

    diff --git a/Modules/_testcapi/run.c b/Modules/_testcapi/run.c
index 4fd98b82d76..b35a12e9424 100644
--- a/Modules/_testcapi/run.c
+++ b/Modules/_testcapi/run.c
@@ -74,6 +74,7 @@ run_fileexflags(PyObject *mod, PyObject *pos_args)
 
     result = PyRun_FileExFlags(fp, filename, start, globals, locals, closeit, pflags);
 
+#ifndef NDEBUG
 #if defined(__linux__) || defined(MS_WINDOWS) || defined(__APPLE__)
     /* The behavior of fileno() after fclose() is undefined, but it is
      * the only practical way to check whether the file was closed.
@@ -85,6 +86,7 @@ run_fileexflags(PyObject *mod, PyObject *pos_args)
         return NULL;
     }
 #endif
+#endif // NDEBUG
     if (!closeit && fileno(fp) < 0) {
         PyErr_SetString(PyExc_AssertionError, "Bad file descriptor after excution");
         Py_XDECREF(result);

    Or maybe just remove this dangerous implementation depending code?

  6. vstinner
    added 3 commits that reference this issue on Apr 30, 2024

    pythongh-118422: Fix run_fileexflags() test

    7910aa0

    pythongh-118422: Fix run_fileexflags() test

    f0ff979

    pythongh-118422: Fix run_fileexflags() test

    4cba722

  7. vstinner commented on Apr 30, 2024

    @vstinner
    vstinner
    on Apr 30, 2024
    MemberAuthor

    Or maybe just remove this dangerous implementation depending code?

    I wrote PR gh-118429 to use fstat() instead of relying on an undefined behavior of fileno() on a closed file.

  8. vstinner
    added a commit that references this issue on Apr 30, 2024

    gh-118422: Fix run_fileexflags() test (#118429)

    e93c39b
  9. vstinner
    closed this as completedon Apr 30, 2024
  10. SonicField
    added a commit that references this issue on May 8, 2024

    pythongh-118422: Fix run_fileexflags() test (python#118429)

    1c20fc5
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    testsTests in the Lib/test dirtype-bugAn unexpected behavior, bug, or error

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions