Skip to content

memoryview rely on _Bool undefined behavior (test_buffer fails on Python built with GCC 11) #86753

Closed
Closed
memoryview rely on _Bool undefined behavior (test_buffer fails on Python built with GCC 11)#86753
Labels
3.10 (EOL)end of life3.9 (EOL)end of lifetestsTests in the Lib/test dir

Description

@vstinner
vstinner
opened on Dec 7, 2020
BPO 42587
Nosy @vstinner

Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2021-01-04.11:58:06.897>
created_at = <Date 2020-12-07.15:00:21.718>
labels = ['tests', '3.9', '3.10']
title = 'memoryview rely on _Bool undefined behavior (test_buffer fails on Python built with GCC 11)'
updated_at = <Date 2021-01-04.11:58:06.896>
user = 'https://github.com/vstinner'

bugs.python.org fields:

activity = <Date 2021-01-04.11:58:06.896>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2021-01-04.11:58:06.897>
closer = 'vstinner'
components = ['Tests']
creation = <Date 2020-12-07.15:00:21.718>
creator = 'vstinner'
dependencies = []
files = []
hgrepos = []
issue_num = 42587
keywords = []
message_count = 10.0
messages = ['382643', '382644', '382645', '382646', '382693', '382701', '382719', '382973', '384322', '384323']
nosy_count = 1.0
nosy_names = ['vstinner']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue42587'
versions = ['Python 3.9', 'Python 3.10']

Activity

  1. vstinner commented on Dec 7, 2020

    @vstinner
    vstinner
    on Dec 7, 2020
    MemberAuthor

    PPC64LE Fedora Rawhide LTO 3.x:
    https://buildbot.python.org/all/#/builders/448/builds/433

    FAIL: test_memoryview_cast (test.test_buffer.TestBufferProtocol)
    FAIL: test_memoryview_cast_1D_ND (test.test_buffer.TestBufferProtocol)
    FAIL: test_memoryview_compare_random_formats (test.test_buffer.TestBufferProtocol)
    FAIL: test_ndarray_format_shape (test.test_buffer.TestBufferProtocol)
    FAIL: test_ndarray_format_strides (test.test_buffer.TestBufferProtocol)
    FAIL: test_ndarray_getbuf (test.test_buffer.TestBufferProtocol)
    FAIL: test_ndarray_index_getitem_single (test.test_buffer.TestBufferProtocol)
    FAIL: test_ndarray_random (test.test_buffer.TestBufferProtocol)
    FAIL: test_ndarray_slice_assign_single (test.test_buffer.TestBufferProtocol)

    Example:

    FAIL: test_ndarray_slice_assign_single (test.test_buffer.TestBufferProtocol)
    ----------------------------------------------------------------------

    Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-fedora-rawhide-ppc64le.lto/build/Lib/test/test_buffer.py", line 1836, in test_ndarray_slice_assign_single
    self.assertEqual(mv, nd)
AssertionError: <memory at 0x7fff80e51dc0> != <ndarray object at 0x7fff80c35030>
  2. vstinner
    added
    3.9 (EOL)end of life
    3.10 (EOL)end of life
    testsTests in the Lib/test dir
    on Dec 7, 2020

  3. vstinner commented on Dec 7, 2020

    @vstinner
    vstinner
    on Dec 7, 2020
    MemberAuthor

    On Python 3.6/x86-64, Python does crash in libregrtest while loading test_code:

    0:01:13 load avg: 2.23 [ 56/406] test_code

    Fatal Python error: Segmentation fault
    Current thread 0x00007f5a6864d740 (most recent call first):
    File "/builddir/build/BUILD/Python-3.6.12/Lib/test/test_code.py", line 288 in <module>
    File "<frozen importlib._bootstrap>", line 219 in _call_with_frames_removed
    File "<frozen importlib._bootstrap_external>", line 678 in exec_module
    File "<frozen importlib._bootstrap>", line 665 in _load_unlocked
    File "<frozen importlib._bootstrap>", line 955 in _find_and_load_unlocked
    File "<frozen importlib._bootstrap>", line 971 in _find_and_load
    File "<frozen importlib._bootstrap>", line 994 in _gcd_import
    File "/builddir/build/BUILD/Python-3.6.12/Lib/importlib/init.py", line 126 in import_module
    (...)

    On Python 3.5/x86-64, test_ctypes.test_callbacks() crashs with a SIGSEGV:

    test_callbacks (ctypes.test.test_as_parameter.AsParamPropertyWrapperTestCase) ...

    Fatal Python error: Segmentation fault
    Current thread 0x00007f5de8322740 (most recent call first):
    File "/builddir/build/BUILD/Python-3.5.10/Lib/ctypes/test/test_as_parameter.py", line 90 in test_callbacks

    The strange part is that the test suite pass on some platforms depending on the Python version.

    It might be a GCC 11 regression.

  4. vstinner commented on Dec 7, 2020

    @vstinner
    vstinner
    on Dec 7, 2020
    MemberAuthor

    Copy of Miro Hrončok's email to Fedora python-devel list:

    https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org/thread/ADKCB6HXGZHJPYX5LSJ7422FITYSOAL3/

    ----
    Hello.

    I've been notified that many of the Python interpreters started to fail with new
    GNU Toolchain (gcc 11, glibc 2.33).

    https://koschei.fedoraproject.org/package/pythonX.Y (replace X.Y with version)

    Python 3.5

    • test_buffer fails on aarch64
    • test_buffer+test_dbm_gnu fails on ppc64le
    • %check segafults on x86_64

    Python 3.6

    • %check segafults on x86_64
    • test_buffer fails on aarch64

    Python 3.7

    • test_buffer fails on aarch64+ppc64le

    Python 3.8

    • seems to build fine

    Python 3.9

    • test_buffer fails on aarch64+ppc64le

    Python 3.10

    • seems to build fine

    PyPys have not yet been collected by Koschei, Python 2.7 seems fine.
    ----

    Python is built with LTO+PGO on Fedora.

  5. vstinner commented on Dec 7, 2020

    @vstinner
    vstinner
    on Dec 7, 2020
    MemberAuthor

    test_buffer also fails with GCC 11 on Python 3.8 built without LTO nor PGO, just -Og (./configure --enable-pydebug).

    PPC64LE Fedora Rawhide Refleaks 3.8:
    https://buildbot.python.org/all/#/builders/398/builds/101

    test.pythoninfo:

    sysconfig[CFLAGS]: -Wno-unused-result -Wsign-compare -g -Og -Wall

    sysconfig[PY_CFLAGS]: -Wno-unused-result -Wsign-compare -g -Og -Wall

    sysconfig[PY_CFLAGS_NODIST]: -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -Werror=implicit-function-declaration -I./Include/internal

    sysconfig[PY_STDMODULE_CFLAGS]: -Wno-unused-result -Wsign-compare -g -Og -Wall -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -Werror=implicit-function-declaration -I./Include/internal -I. -I./Include

    Note: I'm not sure why test.pythoninfo didn't log LDFLAGS? It's supposed to log them.

  6. vstinner commented on Dec 7, 2020

    @vstinner
    vstinner
    on Dec 7, 2020
    MemberAuthor

    FAIL: test_memoryview_cast (test.test_buffer.TestBufferProtocol)

    It might be related to _Bool. See also bpo-39689.

  7. vstinner commented on Dec 8, 2020

    @vstinner
    vstinner
    on Dec 8, 2020
    MemberAuthor

    FAIL: test_memoryview_cast (test.test_buffer.TestBufferProtocol)

    On AArch64, GCC 11 miscompiles unpack_cmp() of Objects/memoryview.c for the '?' case:

    /* boolean */
case '?': CMP_SINGLE(p, q, _Bool); return equal;

    I reported the bug to GCC upstream: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=98190

  8. vstinner commented on Dec 8, 2020

    @vstinner
    vstinner
    on Dec 8, 2020
    MemberAuthor

    See also: Behavior of struct “?” format (native _Bool)
    https://discuss.python.org/t/behavior-of-struct-format-native-bool/3774

  9. vstinner
    changed the title [-]test_buffer fails on Python built with GCC 11[/-] [+]memoryview rely on _Bool undefined behavior (test_buffer fails on Python built with GCC 11)[/+] on Dec 8, 2020
  10. vstinner
    changed the title [-]test_buffer fails on Python built with GCC 11[/-] [+]memoryview rely on _Bool undefined behavior (test_buffer fails on Python built with GCC 11)[/+] on Dec 8, 2020

  11. vstinner commented on Dec 14, 2020

    @vstinner
    vstinner
    on Dec 14, 2020
    MemberAuthor

    I reported the bug to GCC upstream: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=98190

    GCC got a fix:
    https://gcc.gnu.org/git/gitweb.cgi?p=gcc.git;h=3e60ddeb8220ed388819bb3f14e8caa9309fd3c2

  12. vstinner commented on Jan 4, 2021

    @vstinner
    vstinner
    on Jan 4, 2021
    MemberAuthor

    The Fedora package gcc-11.0.0-0.11 includes https://gcc.gnu.org/git/gitweb.cgi?p=gcc.git;h=3e60ddeb8220ed388819bb3f14e8caa9309fd3c2 fix for the https://gcc.gnu.org/bugzilla/show_bug.cgi?id=98190 bug.

  13. vstinner commented on Jan 4, 2021

    @vstinner
    vstinner
    on Jan 4, 2021
    MemberAuthor

    "PPC64LE Fedora Rawhide LTO 3.x" buildbot is back to green. test.pythoninfo now says:

    CC.version: gcc (GCC) 11.0.0 20201223 (Red Hat 11.0.0-0)

  14. vstinner
    closed this as completedon Jan 4, 2021
  15. vstinner
    closed this as completedon Jan 4, 2021
  16. ezio-melotti
    transferred this issue fromon Apr 10, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    3.10 (EOL)end of life3.9 (EOL)end of lifetestsTests in the Lib/test dir

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions