Skip to content

Segmentation fault in test_call.py (c_recurse) on Linux with Clang (LTO+PGO) #145953

Closed as duplicate
Closed as duplicate
Segmentation fault in test_call.py (c_recurse) on Linux with Clang (LTO+PGO)#145953
Labels
interpreter-core(Objects, Python, Grammar, and Parser dirs)type-bugAn unexpected behavior, bug, or error

Description

@ashm-dev
ashm-dev
opened on Mar 14, 2026

Bug report

Bug description:

Running the CPython test suite results in a segmentation fault during test_call. The crash occurs in Lib/test/test_call.py at line 1081 (c_recurse).

Environment:

  • CPython commit: 798070d8ca4 (Mar 14 2026)
  • OS: Linux-6.19.6-arch1-1-x86_64-with-glibc2.43 little-endian
  • Compiler: Clang 22.1.1 (Target: x86_64-pc-linux-gnu, GCC installation: 15.2.1)

Build command:

CC=clang CXX=clang++ CFLAGS='-march=native' AR=llvm-ar NM=llvm-nm RANLIB=llvm-ranlib LDFLAGS='-fuse-ld=lld' ./configure --enable-optimizations --with-lto=full
make
Output / Traceback: 
0:06:59 load avg: 3.73 [117/501] test_call
Fatal Python error: Segmentation fault

Current thread 0x00007f2aa84d5780 [python] (most recent call first):
  File "/home/boss/projects/oss/cpython/main/Lib/test/test_call.py", line 1081 in c_recurse
  File "/home/boss/projects/oss/cpython/main/Lib/test/test_call.py", line 1081 in c_recurse
  File "/home/boss/projects/oss/cpython/main/Lib/test/test_call.py", line 1081 in c_recurse
  File "/home/boss/projects/oss/cpython/main/Lib/test/test_call.py", line 1081 in c_recurse
  File "/home/boss/projects/oss/cpython/main/Lib/test/test_call.py", line 1081 in c_recurse
  File "/home/boss/projects/oss/cpython/main/Lib/test/test_call.py", line 1081 in c_recurse
  File "/home/boss/projects/oss/cpython/main/Lib/test/test_call.py", line 1081 in c_recurse
  File "/home/boss/projects/oss/cpython/main/Lib/test/test_call.py", line 1081 in c_recurse
  File "/home/boss/projects/oss/cpython/main/Lib/test/test_call.py", line 1081 in c_recurse
  File "/home/boss/projects/oss/cpython/main/Lib/test/test_call.py", line 1081 in c_recurse
  File "/home/boss/projects/oss/cpython/main/Lib/test/test_call.py", line 1081 in c_recurse
  File "/home/boss/projects/oss/cpython/main/Lib/test/test_call.py", line 1081 in c_recurse
  File "/home/boss/projects/oss/cpython/main/Lib/test/test_call.py", line 1081 in c_recurse
  File "/home/boss/projects/oss/cpython/main/Lib/test/test_call.py", line 1081 in c_recurse
  File "/home/boss/projects/oss/cpython/main/Lib/test/test_call.py", line 1081 in c_recurse
  File "/home/boss/projects/oss/cpython/main/Lib/test/test_call.py", line 1081 in c_recurse
  File "/home/boss/projects/oss/cpython/main/Lib/test/test_call.py", line 1081 in c_recurse
  File "/home/boss/projects/oss/cpython/main/Lib/test/test_call.py", line 1081 in c_recurse
  File "/home/boss/projects/oss/cpython/main/Lib/test/test_call.py", line 1081 in c_recurse
  File "/home/boss/projects/oss/cpython/main/Lib/test/test_call.py", line 1081 in c_recurse
  File "/home/boss/projects/oss/cpython/main/Lib/test/test_call.py", line 1081 in c_recurse
  File "/home/boss/projects/oss/cpython/main/Lib/test/test_call.py", line 1081 in c_recurse
  File "/home/boss/projects/oss/cpython/main/Lib/test/test_call.py", line 1081 in c_recurse
  File "/home/boss/projects/oss/cpython/main/Lib/test/test_call.py", line 1081 in c_recurse
  File "/home/boss/projects/oss/cpython/main/Lib/test/test_call.py", line 1081 in c_recurse
  File "/home/boss/projects/oss/cpython/main/Lib/test/test_call.py", line 1081 in c_recurse
  File "/home/boss/projects/oss/cpython/main/Lib/test/test_call.py", line 1081 in c_recurse
  File "/home/boss/projects/oss/cpython/main/Lib/test/test_call.py", line 1081 in c_recurse
  File "/home/boss/projects/oss/cpython/main/Lib/test/test_call.py", line 1081 in c_recurse
  File "/home/boss/projects/oss/cpython/main/Lib/test/test_call.py", line 1081 in c_recurse
  File "/home/boss/projects/oss/cpython/main/Lib/test/test_call.py", line 1081 in c_recurse
  File "/home/boss/projects/oss/cpython/main/Lib/test/test_call.py", line 1081 in c_recurse
  File "/home/boss/projects/oss/cpython/main/Lib/test/test_call.py", line 1081 in c_recurse
  File "/home/boss/projects/oss/cpython/main/Lib/test/test_call.py", line 1081 in c_recurse
  File "/home/boss/projects/oss/cpython/main/Lib/test/test_call.py", line 1081 in c_recurse
  File "/home/boss/projects/oss/cpython/main/Lib/test/test_call.py", line 1081 in c_recurse
  File "/home/boss/projects/oss/cpython/main/Lib/test/test_call.py", line 1081 in c_recurse
  File "/home/boss/projects/oss/cpython/main/Lib/test/test_call.py", line 1081 in c_recurse
  File "/home/boss/projects/oss/cpython/main/Lib/test/test_call.py", line 1081 in c_recurse
  File "/home/boss/projects/oss/cpython/main/Lib/test/test_call.py", line 1081 in c_recurse
  File "/home/boss/projects/oss/cpython/main/Lib/test/test_call.py", line 1081 in c_recurse
  File "/home/boss/projects/oss/cpython/main/Lib/test/test_call.py", line 1081 in c_recurse
  File "/home/boss/projects/oss/cpython/main/Lib/test/test_call.py", line 1081 in c_recurse
  File "/home/boss/projects/oss/cpython/main/Lib/test/test_call.py", line 1081 in c_recurse
  File "/home/boss/projects/oss/cpython/main/Lib/test/test_call.py", line 1081 in c_recurse
  File "/home/boss/projects/oss/cpython/main/Lib/test/test_call.py", line 1081 in c_recurse
  File "/home/boss/projects/oss/cpython/main/Lib/test/test_call.py", line 1081 in c_recurse
  File "/home/boss/projects/oss/cpython/main/Lib/test/test_call.py", line 1081 in c_recurse
  File "/home/boss/projects/oss/cpython/main/Lib/test/test_call.py", line 1081 in c_recurse
  File "/home/boss/projects/oss/cpython/main/Lib/test/test_call.py", line 1081 in c_recurse

Current thread's C stack trace (most recent call first):
  Binary file "./python", at _Py_DumpStack+0x3a [0x55753f5efb8a]
  Binary file "./python", at +0x664ab8 [0x55753f5fbab8]
  Binary file "./python", at +0x6649f7 [0x55753f5fb9f7]
  Binary file "/usr/lib/libc.so.6", at +0x3e2d0 [0x7f2aa85162d0]
  Binary file "./python", at _PyEval_EvalFrameDefault+0x14 [0x55753f1bd324]
  Binary file "./python", at _PyFunction_Vectorcall+0x318 [0x55753f1bb408]
  Binary file "./python", at PyObject_Vectorcall+0x3d [0x55753f386dfd]
  Binary file "./python", at _PyEval_EvalFrameDefault+0x25bfa [0x55753f1e2f0a]
  Binary file "./python", at _PyFunction_Vectorcall+0x318 [0x55753f1bb408]
  Binary file "./python", at PyObject_Vectorcall+0x3d [0x55753f386dfd]
  Binary file "./python", at _PyEval_EvalFrameDefault+0x25bfa [0x55753f1e2f0a]
  Binary file "./python", at _PyFunction_Vectorcall+0x318 [0x55753f1bb408]
  Binary file "./python", at PyObject_Vectorcall+0x3d [0x55753f386dfd]
  Binary file "./python", at _PyEval_EvalFrameDefault+0x25bfa [0x55753f1e2f0a]
  Binary file "./python", at _PyFunction_Vectorcall+0x318 [0x55753f1bb408]
  Binary file "./python", at PyObject_Vectorcall+0x3d [0x55753f386dfd]
  Binary file "./python", at _PyEval_EvalFrameDefault+0x25bfa [0x55753f1e2f0a]
  Binary file "./python", at _PyFunction_Vectorcall+0x318 [0x55753f1bb408]
  Binary file "./python", at PyObject_Vectorcall+0x3d [0x55753f386dfd]
  Binary file "./python", at _PyEval_EvalFrameDefault+0x25bfa [0x55753f1e2f0a]
  Binary file "./python", at _PyFunction_Vectorcall+0x318 [0x55753f1bb408]
  Binary file "./python", at PyObject_Vectorcall+0x3d [0x55753f386dfd]
  Binary file "./python", at _PyEval_EvalFrameDefault+0x25bfa [0x55753f1e2f0a]
  Binary file "./python", at _PyFunction_Vectorcall+0x318 [0x55753f1bb408]
  Binary file "./python", at PyObject_Vectorcall+0x3d [0x55753f386dfd]
  Binary file "./python", at _PyEval_EvalFrameDefault+0x25bfa [0x55753f1e2f0a]
  Binary file "./python", at _PyFunction_Vectorcall+0x318 [0x55753f1bb408]
  Binary file "./python", at PyObject_Vectorcall+0x3d [0x55753f386dfd]
  Binary file "./python", at _PyEval_EvalFrameDefault+0x25bfa [0x55753f1e2f0a]
  Binary file "./python", at _PyFunction_Vectorcall+0x318 [0x55753f1bb408]
  Binary file "./python", at PyObject_Vectorcall+0x3d [0x55753f386dfd]
  Binary file "./python", at _PyEval_EvalFrameDefault+0x25bfa [0x55753f1e2f0a]
  <truncated rest of calls>

Extension modules: _testcapi, _testinternalcapi, _testlimitedcapi, _testmultiphase, _testsinglephase, _testbuffer (total: 6)

CPython versions tested on:

3.15, CPython main branch

Operating systems tested on:

Linux

Activity

  1. ashm-dev
    added
    type-bugAn unexpected behavior, bug, or error
    on Mar 14, 2026
  2. picnixz
    added
    interpreter-core(Objects, Python, Grammar, and Parser dirs)
    on Mar 14, 2026

  3. ashm-dev commented on Mar 14, 2026

    @ashm-dev
    ashm-dev
    on Mar 14, 2026
    ContributorAuthor

    Update:

    I have investigated the issue further by isolating the build flags.

    • Building with --with-pydebug works as expected.
    • Building with --enable-optimizations (without LTO) works as expected.
    • Building with LTO=full (without PGO) works as expected.
    • Building with LTO (standard) works as expected.

    The segmentation fault only occurs when both LTO and PGO are enabled simultaneously (specifically with --enable-optimizations --with-lto=full). It appears that this combination triggers an aggressive inlining optimization by the compiler (Clang 22.1.1) that leads to C-stack overflow during the test_call recursion tests.

    UPD: Testing with --with-lto=thin + PGO shows that the segmentation fault persists across all LTO modes when combined with PGO, confirming the issue is specific to the interaction between LTO and PGO optimizations.

  4. Fidget-Spinner commented on Mar 14, 2026

    @Fidget-Spinner
    Fidget-Spinner
    on Mar 14, 2026
    Member

    Can you try if passing --with-tail-call-interp fixes it? Usually if it does that means a giant function that isn't meant to be inlined in the eval loop is being inlined, and we should mark a Py_NO_INLINE on it.

  5. ashm-dev commented on Mar 14, 2026

    @ashm-dev
    ashm-dev
    on Mar 14, 2026
    ContributorAuthor

    @Fidget-Spinner Yes, with the --with-tail-call-interp flag enabled, everything runs without a segfault

  6. ashm-dev commented on Mar 14, 2026

    @ashm-dev
    ashm-dev
    on Mar 14, 2026
    ContributorAuthor

    I tried the --with-tail-call-interp flag as suggested. It indeed resolves the segmentation fault during test_call.

    However, it causes a regression in test_perf_profiler (test_python_calls_appear_in_the_stack_if_perf_activated), which fails with an AssertionError. The profiler no longer finds the expected function names (py::bar) in the output, likely because the tail-call interpreter changes the stack frame structure that the profiler relies on.

    This confirms that the issue is closely tied to how the eval loop manages function calls and stack frames.

  7. Fidget-Spinner commented on Mar 15, 2026

    @Fidget-Spinner
    Fidget-Spinner
    on Mar 15, 2026
    Member

    Alright, now it's just a matter of finding which function is responsible :)

  8. ashm-dev commented on Mar 15, 2026

    @ashm-dev
    ashm-dev
    on Mar 15, 2026
    ContributorAuthor

    @Fidget-Spinner The segfault is reproducible up to and including commit 9a83c02; it is no longer reproducible in subsequent commits.

  9. ashm-dev commented on Mar 15, 2026

    @ashm-dev
    ashm-dev
    on Mar 15, 2026
    ContributorAuthor

    The issue reproduces from 798070d to 9a83c02, inclusive.

  10. Fidget-Spinner commented on Mar 15, 2026

    @Fidget-Spinner
    Fidget-Spinner
    on Mar 15, 2026
    Member

    Looks like it's indeed a compiler problem? The commits listed shouldn't affect the interpreter to cause a stack overflow.

  11. Fidget-Spinner
    closed this as a duplicateof python -m test -v test_call segfault with clang 22.1.3 build #148284 on Apr 10, 2026

  12. Fidget-Spinner commented on Apr 10, 2026

    @Fidget-Spinner
    Fidget-Spinner
    on Apr 10, 2026
    Member

    I think this is fixed now on 3.15 and 3.14. Thanks for reporting.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    interpreter-core(Objects, Python, Grammar, and Parser dirs)type-bugAn unexpected behavior, bug, or error

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions