Repository navigation
Segfault during BOLT profile collection in test_functools (fib) when building with --enable-bolt --enable-optimizations #157062
Copy link
Copy link
Open
Labels
interpreter-core(Objects, Python, Grammar, and Parser dirs)(Objects, Python, Grammar, and Parser dirs)type-crashA hard crash of the interpreter, possibly with a core dumpA hard crash of the interpreter, possibly with a core dump
Description
Activity
- addedtype-crashA hard crash of the interpreter, possibly with a core dumpA hard crash of the interpreter, possibly with a core dump
on Sep 7, 2026 - addedinterpreter-core(Objects, Python, Grammar, and Parser dirs)(Objects, Python, Grammar, and Parser dirs)
on Sep 7, 2026 @Fidget-Spinner @corona10 @markshannon Is BOLT no longer experimental? could it be because we are exhausting the stack somehow (or maybe it's clang 22.1.8 that is the culprit...?)
@ashm-dev does it happen on 3.15?
I never consider that BOLT is non experimental, I need to reevaluate the current status.
@picnixz Yes, the same issue occurs on both 3.15 and 3.14.
Reacted by Bénédikt Tran@ashm-dev pretty easy way to test if this is indeed a clang inlining too much and causing stack overflow: just pass
--with-tail-call-interpto your configure. If now the tests pass, that points to a high stack consumption bug on normal interpreter builds.CC=clang CXX=clang++ LDFLAGS='-fuse-ld=lld' ./configure --enable-bolt --enable-optimizations --with-tail-call-interp
BOLT-INFO: Target architecture: x86_64 BOLT-INFO: BOLT version: ca2b52862de907b4e4dca1d064098a28ab8269be BOLT-INFO: first alloc address is 0x200000 BOLT-INFO: creating new program header table at address 0x800000, offset 0x600000 BOLT-INFO: enabling relocation mode BOLT-INFO: forcing -jump-tables=move for instrumentation BOLT-INFO: enabling lite mode BOLT-INFO: 0 out of 9260 functions in the binary (0.0%) have non-empty execution profile BOLT-INFO: validate-mem-refs updated 2 object references BOLT-INSTRUMENTER: Number of indirect call site descriptors: 2305 BOLT-INSTRUMENTER: Number of indirect call target descriptors: 9216 BOLT-INSTRUMENTER: Number of function descriptors: 9216 BOLT-INSTRUMENTER: Number of branch counters: 106963 BOLT-INSTRUMENTER: Number of ST leaf node counters: 54155 BOLT-INSTRUMENTER: Number of direct call counters: 0 BOLT-INSTRUMENTER: Total number of counters: 161118 BOLT-INSTRUMENTER: Total size of counters: 1288944 bytes (static alloc memory) BOLT-INSTRUMENTER: Total size of string table emitted: 205747 bytes in file BOLT-INSTRUMENTER: Total size of descriptors: 10697116 bytes in file BOLT-INSTRUMENTER: Profile will be saved to file /home/boss/projects/oss/cpython/main/python.bolt BOLT-INFO: 75452 instructions were shortened BOLT-INFO: removed 87 empty blocks BOLT-INFO: UCE removed 615 blocks and 37565 bytes of code BOLT-INFO: padding code to 0x1200000 to accommodate hot text BOLT-INFO: output linked against instrumentation runtime library, lib entry point is 0x140db80 BOLT-INFO: clear procedure is 0x1409310 BOLT-INFO: patched build-id (flipped last bit) BOLT-INFO: setting __bolt_runtime_start to 0x140db80 BOLT-INFO: setting __bolt_runtime_fini to 0x140dbd0 BOLT-INFO: setting __hot_start to 0xa00000 BOLT-INFO: setting __hot_end to 0x11d7378 BOLT-INFO: runtime library finalization was hooked via DT_FINI, set to 0x140dbd0 BOLT-INFO: runtime library initialization was hooked via ELF Header Entry Point, set to 0x140db80 # Run instrumented binaries to collect data. ./python -m test --pgo --timeout= Using random seed: 2596992137 0:00:00 load avg: 2.84 Run 43 tests sequentially in a single process 0:00:00 load avg: 2.84 [ 1/43] test_array 0:00:04 load avg: 2.77 [ 1/43] test_array passed 0:00:04 load avg: 2.77 [ 2/43] test_base64 0:00:06 load avg: 2.77 [ 2/43] test_base64 passed 0:00:06 load avg: 2.77 [ 3/43] test_binascii 0:00:06 load avg: 2.77 [ 3/43] test_binascii passed 0:00:06 load avg: 2.77 [ 4/43] test_binop 0:00:07 load avg: 2.77 [ 4/43] test_binop passed 0:00:07 load avg: 2.77 [ 5/43] test_bisect 0:00:07 load avg: 2.77 [ 5/43] test_bisect passed 0:00:07 load avg: 2.77 [ 6/43] test_bytes 0:00:26 load avg: 2.48 [ 6/43] test_bytes passed 0:00:26 load avg: 2.48 [ 7/43] test_bz2 0:00:27 load avg: 2.48 [ 7/43] test_bz2 passed 0:00:27 load avg: 2.48 [ 8/43] test_cmath 0:00:27 load avg: 2.48 [ 8/43] test_cmath passed 0:00:27 load avg: 2.48 [ 9/43] test_codecs 0:00:30 load avg: 2.37 [ 9/43] test_codecs passed 0:00:30 load avg: 2.37 [10/43] test_collections 0:00:34 load avg: 2.37 [10/43] test_collections passed 0:00:34 load avg: 2.37 [11/43] test_complex 0:00:34 load avg: 2.26 [11/43] test_complex passed 0:00:34 load avg: 2.26 [12/43] test_dataclasses 0:00:36 load avg: 2.26 [12/43] test_dataclasses passed 0:00:36 load avg: 2.26 [13/43] test_datetime 0:00:49 load avg: 2.55 [13/43] test_datetime passed 0:00:49 load avg: 2.55 [14/43] test_decimal 0:00:57 load avg: 2.46 [14/43] test_decimal passed 0:00:57 load avg: 2.46 [15/43] test_difflib 0:01:00 load avg: 2.34 [15/43] test_difflib passed 0:01:00 load avg: 2.34 [16/43] test_float 0:01:00 load avg: 2.34 [16/43] test_float passed 0:01:00 load avg: 2.34 [17/43] test_fstring 0:01:04 load avg: 2.34 [17/43] test_fstring passed 0:01:04 load avg: 2.34 [18/43] test_functools Fatal Python error: Segmentation fault Current thread 0x00007f0e3e0a2780 [python] (most recent call first): File "/home/boss/projects/oss/cpython/main/Lib/test/test_functools.py", line 2251 in fib File "/home/boss/projects/oss/cpython/main/Lib/test/test_functools.py", line 2251 in fib File "/home/boss/projects/oss/cpython/main/Lib/test/test_functools.py", line 2251 in fib File "/home/boss/projects/oss/cpython/main/Lib/test/test_functools.py", line 2251 in fib File "/home/boss/projects/oss/cpython/main/Lib/test/test_functools.py", line 2251 in fib File "/home/boss/projects/oss/cpython/main/Lib/test/test_functools.py", line 2251 in fib File "/home/boss/projects/oss/cpython/main/Lib/test/test_functools.py", line 2251 in fib File "/home/boss/projects/oss/cpython/main/Lib/test/test_functools.py", line 2251 in fib File "/home/boss/projects/oss/cpython/main/Lib/test/test_functools.py", line 2251 in fib File "/home/boss/projects/oss/cpython/main/Lib/test/test_functools.py", line 2251 in fib File "/home/boss/projects/oss/cpython/main/Lib/test/test_functools.py", line 2251 in fib File "/home/boss/projects/oss/cpython/main/Lib/test/test_functools.py", line 2251 in fib File "/home/boss/projects/oss/cpython/main/Lib/test/test_functools.py", line 2251 in fib File "/home/boss/projects/oss/cpython/main/Lib/test/test_functools.py", line 2251 in fib File "/home/boss/projects/oss/cpython/main/Lib/test/test_functools.py", line 2251 in fib File "/home/boss/projects/oss/cpython/main/Lib/test/test_functools.py", line 2251 in fib File "/home/boss/projects/oss/cpython/main/Lib/test/test_functools.py", line 2251 in fib File "/home/boss/projects/oss/cpython/main/Lib/test/test_functools.py", line 2251 in fib File "/home/boss/projects/oss/cpython/main/Lib/test/test_functools.py", line 2251 in fib File "/home/boss/projects/oss/cpython/main/Lib/test/test_functools.py", line 2251 in fib File "/home/boss/projects/oss/cpython/main/Lib/test/test_functools.py", line 2251 in fib File "/home/boss/projects/oss/cpython/main/Lib/test/test_functools.py", line 2251 in fib File "/home/boss/projects/oss/cpython/main/Lib/test/test_functools.py", line 2251 in fib File "/home/boss/projects/oss/cpython/main/Lib/test/test_functools.py", line 2251 in fib File "/home/boss/projects/oss/cpython/main/Lib/test/test_functools.py", line 2251 in fib File "/home/boss/projects/oss/cpython/main/Lib/test/test_functools.py", line 2251 in fib File "/home/boss/projects/oss/cpython/main/Lib/test/test_functools.py", line 2251 in fib File "/home/boss/projects/oss/cpython/main/Lib/test/test_functools.py", line 2251 in fib File "/home/boss/projects/oss/cpython/main/Lib/test/test_functools.py", line 2251 in fib File "/home/boss/projects/oss/cpython/main/Lib/test/test_functools.py", line 2251 in fib File "/home/boss/projects/oss/cpython/main/Lib/test/test_functools.py", line 2251 in fib File "/home/boss/projects/oss/cpython/main/Lib/test/test_functools.py", line 2251 in fib File "/home/boss/projects/oss/cpython/main/Lib/test/test_functools.py", line 2251 in fib File "/home/boss/projects/oss/cpython/main/Lib/test/test_functools.py", line 2251 in fib File "/home/boss/projects/oss/cpython/main/Lib/test/test_functools.py", line 2251 in fib File "/home/boss/projects/oss/cpython/main/Lib/test/test_functools.py", line 2251 in fib File "/home/boss/projects/oss/cpython/main/Lib/test/test_functools.py", line 2251 in fib File "/home/boss/projects/oss/cpython/main/Lib/test/test_functools.py", line 2251 in fib File "/home/boss/projects/oss/cpython/main/Lib/test/test_functools.py", line 2251 in fib File "/home/boss/projects/oss/cpython/main/Lib/test/test_functools.py", line 2251 in fib File "/home/boss/projects/oss/cpython/main/Lib/test/test_functools.py", line 2251 in fib File "/home/boss/projects/oss/cpython/main/Lib/test/test_functools.py", line 2251 in fib File "/home/boss/projects/oss/cpython/main/Lib/test/test_functools.py", line 2251 in fib File "/home/boss/projects/oss/cpython/main/Lib/test/test_functools.py", line 2251 in fib File "/home/boss/projects/oss/cpython/main/Lib/test/test_functools.py", line 2251 in fib File "/home/boss/projects/oss/cpython/main/Lib/test/test_functools.py", line 2251 in fib File "/home/boss/projects/oss/cpython/main/Lib/test/test_functools.py", line 2251 in fib File "/home/boss/projects/oss/cpython/main/Lib/test/test_functools.py", line 2251 in fib File "/home/boss/projects/oss/cpython/main/Lib/test/test_functools.py", line 2251 in fib File "/home/boss/projects/oss/cpython/main/Lib/test/test_functools.py", line 2251 in fib Current thread's C stack trace (most recent call first): Binary file "./python", at _Py_DumpStack+0x3d [0x1118dd1] Binary file "./python" [0x1113791] Binary file "./python" [0x11134ec] Binary file "/usr/lib/libc.so.6", at +0x3e6f0 [0x7f0e3de3e6f0] Binary file "./python" [0x140f2ab] Extension modules: _testcapi, _testinternalcapi (total: 2) make[1]: *** [Makefile:1054: profile-bolt-stamp] Segmentation fault (core dumped) make[1]: Leaving directory '/home/boss/projects/oss/cpython/main' make: *** [Makefile:1081: bolt-opt] Error 2That's really suspicious 🤔
Metadata
Metadata
Assignees
Labels
interpreter-core(Objects, Python, Grammar, and Parser dirs)(Objects, Python, Grammar, and Parser dirs)type-crashA hard crash of the interpreter, possibly with a core dumpA hard crash of the interpreter, possibly with a core dump
Crash report
What happened?
Environment
mainbranchlld)ca2b52862de907b4e4dca1d064098a28ab8269beSteps to Reproduce
CC=clang CXX=clang++ LDFLAGS='-fuse-ld=lld' ./configure --enable-bolt --enable-optimizationsExpected Behavior
Successful build and optimization of Python binary using BOLT.
Actual Behavior
Segmentation fault occurs while running instrumented binary to collect profile data (
./python -m test --pgo --timeout=) duringtest_functools. The crash happens in the recursivefibfunction inside the BOLT runtime instrumentation layer.Crash Log / Backtrace
Details
CPython versions tested on:
3.16, CPython main branch
Operating systems tested on:
Linux
Output from running 'python -VV' on the command line:
Python 3.16.0a0 (heads/main:42de93a62e7, Sep 7 2026, 13:34:10) [Clang 22.1.8 ]