Skip to content

Py_CLEAR/Py_SETREF() regression on the main branch with C++: SEGV with simple numpy program #157649

Description

@lucascolley

Bug report

Bug description:

The program that triggers the crash:

import numpy

for _ in range(200):
    numpy.prod(2, dtype=numpy.intp)

Reproducer at https://github.com/lucascolley/cpython-np-prod-repro. This produces the SEGV on CPython main branch, but passes on the 3.15 branch.

❯ export BUILD_EDITABLE_PYTHON=false

❯ pixi run repro
✨ Pixi task (repro): python repro.py
Python: 3.16.0a0 (heads/master-dirty:d8a1072491, Sep 16 2026, 22:19:03) [Clang 21.1.8 ]
AddressSanitizer:DEADLYSIGNAL
=================================================================
==54023==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000000 (pc 0x00010639c214 bp 0x00016afdf230 sp 0x00016afdf1d0 T0)
==54023==The signal is caused by a WRITE memory access.
==54023==HINT: address points to the zero page.
    #0 0x00010639c214 in _PyGCHead_SET_NEXT pycore_gc.h:151
    #1 0x00010639c214 in _PyObject_GC_UNTRACK pycore_gc.h:264
    #2 0x00010639c214 in type_dealloc typeobject.c:6898
    #3 0x000106338e40 in _Py_Dealloc object.c:3333
    #4 0x00010e880a34 in PyUFunc_GenericReduction+0x119c (_multiarray_umath.cpython-316-darwin.so:arm64+0x704a34)
    #5 0x000106237858 in _PyVectorcall_Call call.c:273
    #6 0x00010652b77c in _PyEval_EvalFrameDefault generated_cases.c.h:2831
    #7 0x0001065153bc in _PyEval_EvalFrame pycore_ceval.h:122
    #8 0x0001065153bc in _PyEval_Vector ceval.c:2176
    #9 0x000106237a78 in _PyObject_VectorcallTstate pycore_call.h:144
    #10 0x000106237a78 in PyObject_Vectorcall call.c:327
    #11 0x00010e54bd1c in dispatcher_vectorcall+0xc70 (_multiarray_umath.cpython-316-darwin.so:arm64+0x3cfd1c)
    #12 0x000106237a78 in _PyObject_VectorcallTstate pycore_call.h:144
    #13 0x000106237a78 in PyObject_Vectorcall call.c:327
    #14 0x000106515938 in _Py_VectorCall_StackRefSteal ceval.c:728
    #15 0x000106529170 in _PyEval_EvalFrameDefault generated_cases.c.h:3686
    #16 0x000106514db0 in _PyEval_EvalFrame pycore_ceval.h:122
    #17 0x000106514db0 in _PyEval_Vector ceval.c:2176
    #18 0x000106514db0 in PyEval_EvalCode ceval.c:681
    #19 0x00010665ba10 in run_mod pythonrun.c:1509
    #20 0x00010665633c in _PyRun_File pythonrun.c:1332
    #21 0x00010665633c in _PyRun_SimpleFile pythonrun.c:544
    #22 0x000106655998 in _PyRun_AnyFile pythonrun.c:92
    #23 0x0001066bd0b0 in pymain_run_file_obj main.c:478
    #24 0x0001066bd0b0 in pymain_run_file main.c:494
    #25 0x0001066bbe68 in pymain_run_python main.c:804
    #26 0x0001066bbe68 in Py_RunMain main.c:891
    #27 0x0001066bc830 in pymain_main main.c:921
    #28 0x0001066bc9e0 in Py_BytesMain main.c:945
    #29 0x000189febdfc in start+0x1b4c (dyld:arm64e+0x1fdfc)

==54023==Register values:
 x[0] = 0x00006190000dde88   x[1] = 0x000000010639c054   x[2] = 0x000000016afde9d0   x[3] = 0x000061600009a870
 x[4] = 0x0000000000000218   x[5] = 0x0000000000000218   x[6] = 0x0000000169fe4000   x[7] = 0x0000000000000001
 x[8] = 0x00006190000dde80   x[9] = 0x0000000000000000  x[10] = 0x0000000000000000  x[11] = 0x00000fffffffffff
x[12] = 0x0000000000000000  x[13] = 0xffffffffffffffff  x[14] = 0x0000000000000000  x[15] = 0x00007fffffffffff
x[16] = 0x0000000000000028  x[17] = 0x0000604000000410  x[18] = 0x0000000000000000  x[19] = 0x00006190000dde90
x[20] = 0x0000604000000438  x[21] = 0x0000000080805980  x[22] = 0x0000008000020000  x[23] = 0x0000000106c42940
x[24] = 0x0000008000020000  x[25] = 0x0000000000000000  x[26] = 0x000000016afdfb20  x[27] = 0x0000000000000000
x[28] = 0x00006110001f42e4     fp = 0x000000016afdf230     lr = 0x000000010639c1d8     sp = 0x000000016afdf1d0
AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: SEGV pycore_gc.h:151 in _PyGCHead_SET_NEXT
==54023==ABORTING

@ngoldbaum perhaps you can reproduce?

[workspace]
channels = ["https://prefix.dev/conda-forge"]
platforms = ["osx-arm64"]
preview = ["pixi-build"]
requires-pixi = ">=0.74.0"

[tasks]
repro = "python repro.py"

[dependencies]
numpy = {
  git = "https://github.com/numpy/numpy",
  package = {
    host-dependencies = {
      python = {
        flags = ["asan"],
        # passing with 3.15 branch
        # branch = "3.15",
        # failing with main
        branch = "main",
        git = "https://github.com/python/cpython",
        subdirectory = "Tools/pixi-packages",
      },
      meson-python = "*",
      cython = "*",
    },
    build = {
      backend.name = "pixi-build-python",
      config.compilers = ["c", "cxx"],
    },
  },
}

CPython versions tested on:

CPython main branch

Operating systems tested on:

macOS

Linked PRs

Activity

  1. lucascolley commented on Sep 16, 2026

    @lucascolley
    ContributorAuthor

    Update: bisected to bba99f2 gh-157267

  2. ngoldbaum commented on Sep 16, 2026

    @ngoldbaum
    Contributor
  3. lucascolley commented on Sep 16, 2026

    @lucascolley
    ContributorAuthor

    runtime packages:

    Details
    cpython-np-prod-repro on 🎋 main via 🧚 v0.81.0
    ❯ pixi list
    Installed for: osx-arm64
    Name             Version     Build                  Size  Kind   Source
    bzip2            1.0.8       h4e30115_10      122.04 KiB  conda  https://prefix.dev/conda-forge
    ca-certificates  2026.7.22   hbd8a1cb_0       128.69 KiB  conda  https://prefix.dev/conda-forge
    icu              78.3        py310h579977c_2   13.42 MiB  conda  https://prefix.dev/conda-forge
    libcompiler-rt   23.1.1      hdb3d66b_0         1.32 MiB  conda  https://prefix.dev/conda-forge
    libcxx           23.1.1      h55c6f16_0       565.00 KiB  conda  https://prefix.dev/conda-forge
    libexpat         2.8.1       hf6b4638_1        67.74 KiB  conda  https://prefix.dev/conda-forge
    libffi           3.7.0       h47dc5ef_1        42.61 KiB  conda  https://prefix.dev/conda-forge
    liblzma          5.8.3       h8088a28_1        89.57 KiB  conda  https://prefix.dev/conda-forge
    libmpdec         4.0.0       h84a0fba_2        71.57 KiB  conda  https://prefix.dev/conda-forge
    libsqlite        3.53.4      hca69786_1       920.66 KiB  conda  https://prefix.dev/conda-forge
    libzlib          1.3.2       h8088a28_3        46.70 KiB  conda  https://prefix.dev/conda-forge
    ncurses          6.6         he64c551_1       785.45 KiB  conda  https://prefix.dev/conda-forge
    numpy            2.6.0.dev0  hea27dcd_0                   conda  git+https://github.com/numpy/numpy#9e5c44c061f04992033e852aadc405b5b3b31564
    openssl          3.6.4       h55eecbc_0         2.97 MiB  conda  https://prefix.dev/conda-forge
    python           3.16        0_asan_cp316                 conda  git+https://github.com/python/cpython?subdirectory=Tools%2Fpixi-packages&rev=bba99f2b3272cbe47af9bee05a294961c0528829#bba99f2b3272cbe47af9bee05a294961c0528829
    python_abi       3.16        0_asan_cp316                 conda  git+https://github.com/python/cpython?subdirectory=Tools%2Fpixi-packages&rev=bba99f2b3272cbe47af9bee05a294961c0528829#bba99f2b3272cbe47af9bee05a294961c0528829
    readline         8.3         h8b90a29_1       307.03 KiB  conda  https://prefix.dev/conda-forge
    zstd             1.5.7       hf451053_7       423.52 KiB  conda  https://prefix.dev/conda-forge
    

    CPython build-time deps info:

    Details
     │ │ Resolved build dependencies(python-3.16-0_asan_cp316):
     │ │ ╭─────────────────────────┬────────────────────────┬─────────────┬─────────────────────┬─────────────┬────────────╮
     │ │ │ Package                 ┆ Spec                   ┆ Version     ┆ Build               ┆ Channel     ┆       Size │
     │ │ ╞═════════════════════════╪════════════════════════╪═════════════╪═════════════════════╪═════════════╪════════════╡
     │ │ │ clang_osx-arm64         ┆ clang_osx-arm64 21.*   ┆ 21.1.8      ┆ h15b5089_34         ┆ conda-forge ┆  20.20 KiB │
     │ │ │ clangxx_osx-arm64       ┆ clangxx_osx-arm64 21.* ┆ 21.1.8      ┆ h15b5089_34         ┆ conda-forge ┆  18.91 KiB │
     │ │ │ llvm-tools              ┆ llvm-tools             ┆ 21.1.8      ┆ hdb3d66b_1          ┆ conda-forge ┆  87.02 KiB │
     │ │ │ make                    ┆ make                   ┆ 4.4.1       ┆ h84a0fba_3          ┆ conda-forge ┆ 268.66 KiB │
     │ │ │ pkg-config              ┆ pkg-config             ┆ 0.29.2      ┆ he0b766d_1013       ┆ conda-forge ┆ 267.48 KiB │
     │ │ │ ca-certificates         ┆                        ┆ 2026.7.22   ┆ hbd8a1cb_0          ┆ conda-forge ┆ 128.69 KiB │
     │ │ │ cctools_impl_osx-arm64  ┆                        ┆ 1030.6.3    ┆ llvm21_1_h2e2604f_6 ┆ conda-forge ┆ 744.41 KiB │
     │ │ │ cctools_osx-arm64       ┆                        ┆ 1030.6.3    ┆ llvm21_1_h8e84c17_6 ┆ conda-forge ┆  22.95 KiB │
     │ │ │ clang-21                ┆                        ┆ 21.1.8      ┆ default_h4399b93_6  ┆ conda-forge ┆ 800.13 KiB │
     │ │ │ clang-scan-deps         ┆                        ┆ 21.1.8      ┆ default_h4399b93_6  ┆ conda-forge ┆  98.90 KiB │
     │ │ │ clang_impl_osx-arm64    ┆                        ┆ 21.1.8      ┆ default_hc11f16d_6  ┆ conda-forge ┆  27.69 KiB │
     │ │ │ clangxx_impl_osx-arm64  ┆                        ┆ 21.1.8      ┆ default_hc11f16d_6  ┆ conda-forge ┆  27.48 KiB │
     │ │ │ compiler-rt             ┆                        ┆ 21.1.8      ┆ hce30654_2          ┆ conda-forge ┆  15.86 KiB │
     │ │ │ compiler-rt21           ┆                        ┆ 21.1.8      ┆ hdb3d66b_2          ┆ conda-forge ┆  97.24 KiB │
     │ │ │ compiler-rt21_osx-arm64 ┆                        ┆ 21.1.8      ┆ hb8825d9_2          ┆ conda-forge ┆   9.75 MiB │
     │ │ │ compiler-rt_osx-arm64   ┆                        ┆ 21.1.8      ┆ hce30654_2          ┆ conda-forge ┆  16.02 KiB │
     │ │ │ icu                     ┆                        ┆ 78.3        ┆ py310h579977c_2     ┆ conda-forge ┆  13.42 MiB │
     │ │ │ ld64_osx-arm64          ┆                        ┆ 956.6       ┆ llvm21_1_h1a10cc4_6 ┆ conda-forge ┆ 961.26 KiB │
     │ │ │ libclang-cpp21.1        ┆                        ┆ 21.1.8      ┆ default_h4399b93_6  ┆ conda-forge ┆  13.07 MiB │
     │ │ │ libclang-cpp23.1        ┆                        ┆ 23.1.1      ┆ default_h79071a4_0  ┆ conda-forge ┆  15.84 MiB │
     │ │ │ libclang13              ┆                        ┆ 23.1.1      ┆ default_hfbe3a31_0  ┆ conda-forge ┆  10.13 MiB │
     │ │ │ libcompiler-rt          ┆                        ┆ 21.1.8      ┆ hdb3d66b_2          ┆ conda-forge ┆   1.27 MiB │
     │ │ │ libcxx                  ┆                        ┆ 23.1.1      ┆ h55c6f16_0          ┆ conda-forge ┆ 565.00 KiB │
     │ │ │ libcxx-devel            ┆                        ┆ 21.1.8      ┆ h6dc3340_3          ┆ conda-forge ┆  21.58 KiB │
     │ │ │ libcxx-headers          ┆                        ┆ 21.1.8      ┆ h707e725_3          ┆ conda-forge ┆   1.05 MiB │
     │ │ │ libiconv                ┆                        ┆ 1.18        ┆ he4c29f2_3          ┆ conda-forge ┆ 733.22 KiB │
     │ │ │ libllvm21               ┆                        ┆ 21.1.8      ┆ h759d1ac_1          ┆ conda-forge ┆  28.26 MiB │
     │ │ │ libllvm23               ┆                        ┆ 23.1.1      ┆ h759d1ac_0          ┆ conda-forge ┆  30.12 MiB │
     │ │ │ liblzma                 ┆                        ┆ 5.8.3       ┆ h8088a28_1          ┆ conda-forge ┆  89.57 KiB │
     │ │ │ libsigtool              ┆                        ┆ 0.1.3       ┆ h98dc951_1          ┆ conda-forge ┆  35.79 KiB │
     │ │ │ libxml2                 ┆                        ┆ 2.15.4      ┆ h550178d_0          ┆ conda-forge ┆  40.41 KiB │
     │ │ │ libxml2-16              ┆                        ┆ 2.15.4      ┆ heb56d2d_0          ┆ conda-forge ┆ 458.30 KiB │
     │ │ │ libzlib                 ┆                        ┆ 1.3.2       ┆ h8088a28_3          ┆ conda-forge ┆  46.70 KiB │
     │ │ │ llvm-tools-21           ┆                        ┆ 21.1.8      ┆ h79441dc_1          ┆ conda-forge ┆  17.68 MiB │
     │ │ │ ncurses                 ┆                        ┆ 6.6         ┆ he64c551_1          ┆ conda-forge ┆ 785.45 KiB │
     │ │ │ openssl                 ┆                        ┆ 3.6.4       ┆ h55eecbc_0          ┆ conda-forge ┆   2.97 MiB │
     │ │ │ sdkroot_env_osx-arm64   ┆                        ┆ 26.0        ┆ ha3f98da_7          ┆ conda-forge ┆   4.85 KiB │
     │ │ │ sigtool-codesign        ┆                        ┆ 0.1.3       ┆ h98dc951_1          ┆ conda-forge ┆ 111.97 KiB │
     │ │ │ tapi                    ┆                        ┆ 1600.0.11.8 ┆ hb561403_3          ┆ conda-forge ┆ 195.70 KiB │
     │ │ │ zstd                    ┆                        ┆ 1.5.7       ┆ hf451053_7          ┆ conda-forge ┆ 423.52 KiB │
     │ │ ╰─────────────────────────┴────────────────────────┴─────────────┴─────────────────────┴─────────────┴────────────╯
     │ │ Resolved host dependencies(python-3.16-0_asan_cp316):
     │ │ ╭─────────────────┬────────────────────────────────────────────────┬───────────┬─────────────────┬─────────────┬────────────╮
     │ │ │ Package         ┆ Spec                                           ┆ Version   ┆ Build           ┆ Channel     ┆       Size │
     │ │ ╞═════════════════╪════════════════════════════════════════════════╪═══════════╪═════════════════╪═════════════╪════════════╡
     │ │ │ bzip2           ┆ bzip2                                          ┆ 1.0.8     ┆ h4e30115_10     ┆ conda-forge ┆ 122.04 KiB │
     │ │ │ expat           ┆ expat                                          ┆ 2.8.1     ┆ hf6b4638_1      ┆ conda-forge ┆ 132.87 KiB │
     │ │ │ libcompiler-rt  ┆ libcompiler-rt                                 ┆ 23.1.1    ┆ hdb3d66b_0      ┆ conda-forge ┆   1.32 MiB │
     │ │ │ libcxx          ┆ libcxx >=21 (RE of [build: clangxx_osx-arm64]) ┆ 23.1.1    ┆ h55c6f16_0      ┆ conda-forge ┆ 565.00 KiB │
     │ │ │ libffi          ┆ libffi                                         ┆ 3.7.0     ┆ h47dc5ef_1      ┆ conda-forge ┆  42.61 KiB │
     │ │ │ liblzma-devel   ┆ liblzma-devel                                  ┆ 5.8.3     ┆ h8088a28_1      ┆ conda-forge ┆ 116.12 KiB │
     │ │ │ libmpdec-devel  ┆ libmpdec-devel                                 ┆ 4.0.0     ┆ hce30654_2      ┆ conda-forge ┆  14.78 KiB │
     │ │ │ ncurses         ┆ ncurses                                        ┆ 6.6       ┆ he64c551_1      ┆ conda-forge ┆ 785.45 KiB │
     │ │ │ openssl         ┆ openssl 3.5.*                                  ┆ 3.5.8     ┆ h55eecbc_0      ┆ conda-forge ┆   2.92 MiB │
     │ │ │ readline        ┆ readline                                       ┆ 8.3       ┆ h8b90a29_1      ┆ conda-forge ┆ 307.03 KiB │
     │ │ │ sqlite          ┆ sqlite                                         ┆ 3.53.4    ┆ ha9f7ffe_1      ┆ conda-forge ┆ 175.69 KiB │
     │ │ │ tk              ┆ tk                                             ┆ 9.0.4     ┆ hbeba79b_1      ┆ conda-forge ┆  16.14 KiB │
     │ │ │ xorg-libx11     ┆ xorg-libx11                                    ┆ 1.8.13    ┆ hf948f5a_1      ┆ conda-forge ┆ 740.99 KiB │
     │ │ │ xorg-xorgproto  ┆ xorg-xorgproto                                 ┆ 2025.1    ┆ h74c22ad_3      ┆ conda-forge ┆ 583.54 KiB │
     │ │ │ zlib            ┆ zlib                                           ┆ 1.3.2     ┆ h8088a28_3      ┆ conda-forge ┆  79.83 KiB │
     │ │ │ zstd            ┆ zstd                                           ┆ 1.5.7     ┆ hf451053_7      ┆ conda-forge ┆ 423.52 KiB │
     │ │ │ ca-certificates ┆                                                ┆ 2026.7.22 ┆ hbd8a1cb_0      ┆ conda-forge ┆ 128.69 KiB │
     │ │ │ icu             ┆                                                ┆ 78.3      ┆ py310h579977c_2 ┆ conda-forge ┆  13.42 MiB │
     │ │ │ libexpat        ┆                                                ┆ 2.8.1     ┆ hf6b4638_1      ┆ conda-forge ┆  67.74 KiB │
     │ │ │ liblzma         ┆                                                ┆ 5.8.3     ┆ h8088a28_1      ┆ conda-forge ┆  89.57 KiB │
     │ │ │ libmpdec        ┆                                                ┆ 4.0.0     ┆ h84a0fba_2      ┆ conda-forge ┆  71.57 KiB │
     │ │ │ libsqlite       ┆                                                ┆ 3.53.4    ┆ hca69786_1      ┆ conda-forge ┆ 920.66 KiB │
     │ │ │ libtcl          ┆                                                ┆ 9.0.4     ┆ hbeba79b_1      ┆ conda-forge ┆   1.13 MiB │
     │ │ │ libtk           ┆                                                ┆ 9.0.4     ┆ hbeba79b_1      ┆ conda-forge ┆   1.32 MiB │
     │ │ │ libtommath      ┆                                                ┆ 1.3.0     ┆ h74c22ad_3      ┆ conda-forge ┆  57.29 KiB │
     │ │ │ libxcb          ┆                                                ┆ 1.17.0    ┆ hbffa61f_2      ┆ conda-forge ┆ 316.63 KiB │
     │ │ │ libzlib         ┆                                                ┆ 1.3.2     ┆ h8088a28_3      ┆ conda-forge ┆  46.70 KiB │
     │ │ │ pthread-stubs   ┆                                                ┆ 0.4       ┆ hb001647_1004   ┆ conda-forge ┆   9.44 KiB │
     │ │ │ xorg-libxau     ┆                                                ┆ 1.0.12    ┆ hb001647_2      ┆ conda-forge ┆  16.92 KiB │
     │ │ │ xorg-libxdmcp   ┆                                                ┆ 1.1.5     ┆ h84a0fba_2      ┆ conda-forge ┆  19.03 KiB │
     │ │ ╰─────────────────┴────────────────────────────────────────────────┴───────────┴─────────────────┴─────────────┴────────────╯
     │ │ Resolved run dependencies(python-3.16-0_asan_cp316):
     │ │ ╭──────────────────────┬───────────────────────────────────────────────╮
     │ │ │ Name                 ┆ Spec                                          │
     │ │ ╞══════════════════════╪═══════════════════════════════════════════════╡
     │ │ │ Run dependencies     ┆                                               │
     │ │ │ bzip2                ┆ >=1.0.8,<2.0a0 (RE of [host: bzip2])          │
     │ │ │ libcompiler-rt       ┆ >=23.1.1 (RE of [host: libcompiler-rt])       │
     │ │ │ libcxx               ┆ >=21 (RE of [build: clangxx_osx-arm64])       │
     │ │ │ libexpat             ┆ >=2.8.1,<3.0a0 (RE of [host: expat])          │
     │ │ │ libffi               ┆ >=3.7.0,<3.8.0a0 (RE of [host: libffi])       │
     │ │ │ liblzma              ┆ >=5.8.3,<6.0a0 (RE of [host: liblzma-devel])  │
     │ │ │ libmpdec             ┆ >=4.0.0,<5.0a0 (RE of [host: libmpdec-devel]) │
     │ │ │ libsqlite            ┆ >=3.53.4,<4.0a0 (RE of [host: sqlite])        │
     │ │ │ libzlib              ┆ >=1.3.2,<2.0a0 (RE of [host: zlib])           │
     │ │ │ ncurses              ┆ >=6.6,<7.0a0 (RE of [host: ncurses])          │
     │ │ │ openssl              ┆ >=3.5.8,<4.0a0 (RE of [host: openssl])        │
     │ │ │ readline             ┆ >=8.3,<9.0a0 (RE of [host: readline])         │
     │ │ │ zstd                 ┆ >=1.5.7,<1.6.0a0 (RE of [host: zstd])         │
     │ │ │                      ┆                                               │
     │ │ │ Run exports (Weak)   ┆                                               │
     │ │ │ python_abi           ┆ 3.16.* *_asan_cp316                           │
     │ │ │                      ┆                                               │
     │ │ │ Run exports (Noarch) ┆                                               │
     │ │ │ python               ┆                                               │
     │ │ ╰──────────────────────┴───────────────────────────────────────────────╯
    
  4. ngoldbaum commented on Sep 17, 2026

    @ngoldbaum
    Contributor

    Here's an AI-generated C++ example that triggers the same issue NumPy is triggering:

    Details
    // Standalone demonstration of the _Py_TYPEOF() C++ regression (gh-157649).
    // Build against CPython main as C++ and run; every line marked FAIL shows
    // Py_SETREF()/Py_XSETREF()/Py_CLEAR() operating on the wrong object.
    #include <Python.h>
    #include <cstdio>
    #include <type_traits>
    
    int main()
    {
        Py_Initialize();
    
        PyObject *slots[1];
    
        // decltype(slots[0]) is "PyObject *&" (a reference), so the temporary that
        // is supposed to hold the *old* value aliases the slot itself.
        printf("_Py_TYPEOF(slots[0]) is a reference type: %s\n",
               std::is_reference<_Py_TYPEOF(slots[0])>::value ? "yes" : "no");
    
        PyObject *a = PyList_New(0);
        PyObject *b = PyList_New(0);
        Py_INCREF(a);  // a: 2
        Py_INCREF(b);  // b: 2
        slots[0] = a;
    
        Py_SETREF(slots[0], b);  // should DECREF a, leave b alone
        printf("after Py_SETREF(slots[0], b): refcnt(a)=%zd (expect 1), refcnt(b)=%zd (expect 2)  %s\n",
               Py_REFCNT(a), Py_REFCNT(b),
               (Py_REFCNT(a) == 1 && Py_REFCNT(b) == 2) ? "OK" : "FAIL");
    
        slots[0] = a;
        Py_INCREF(a);
        Py_INCREF(b);
        Py_XSETREF(slots[0], b);
        printf("after Py_XSETREF(slots[0], b): refcnt(a)=%zd (expect 1), refcnt(b)=%zd (expect 3)  %s\n",
               Py_REFCNT(a), Py_REFCNT(b),
               (Py_REFCNT(a) == 1 && Py_REFCNT(b) == 3) ? "OK" : "FAIL");
    
        // Py_CLEAR(slots[0]) expands to: store NULL into the slot, then
        // Py_DECREF(<alias of the slot>) == Py_DECREF(NULL).
        PyObject *c = PyList_New(0);
        Py_INCREF(c);
        slots[0] = c;
        printf("calling Py_CLEAR(slots[0]) ...\n");
        fflush(stdout);
        Py_CLEAR(slots[0]);
        printf("after Py_CLEAR(slots[0]): refcnt(c)=%zd (expect 1)  %s\n",
               Py_REFCNT(c), Py_REFCNT(c) == 1 ? "OK" : "FAIL");
    
        Py_Finalize();
        return 0;
    }
    

    The decltype of an lvalue expression like slots[0] is a reference type rather than an object type. That means with the new semantics, the "old value" temporary in Py_SETREF/Py_XSETREF/Py_CLEAR ends up as an alias of the slot being overwritten.

  5. StanFromIreland commented on Sep 17, 2026

    @StanFromIreland
    Member

    Update: bisected to bba99f2 #157267

    As I noted there, that's not possible as the commit was not backported to 3.15. It's only on main.

    Sorry, I misread the issue, 3.15 is not failing if I understand correctly.

  6. lucascolley commented on Sep 17, 2026

    @lucascolley
    ContributorAuthor

    any ideas what a fix might look like?

    If anyone wants to test a potential fix, you can edit https://github.com/lucascolley/cpython-np-prod-repro/blob/main/pixi.toml#L20-L21 to point to a local path, or a revision on a git fork

  7. changed the title [-]ASan regression since 3.15: SEGV with simple `numpy` program[/-] [+]ASan regression on the main branch in Py_CLEAR/Py_SETREF: SEGV with simple `numpy` program[/+] on Sep 17, 2026
  8. vstinner commented on Sep 17, 2026

    @vstinner
    Member

    I built and installed Python in /opt/py316 with:

    cd ~/cpython  # https://github.com/python/cpython/ checkout
    ./configure --cache-file=../python-config.cache --with-pydebug --with-system-expat --prefix /opt/py316
    make
    make install
    

    I built @ngoldbaum reproducer with:

    $ cd ~/repro  # with repro.cpp
    g++ repro.cpp -o repro $(PKG_CONFIG_PATH=/opt/py316/lib/pkgconfig/ pkg-config python-3.16-embed --cflags --libs)
    

    The reproducer does crash without ASAN:

    $ ./repro
    $ ./repro 
    _Py_TYPEOF(slots[0]) is a reference type: yes
    after Py_SETREF(slots[0], b): refcnt(a)=2 (expect 1), refcnt(b)=1 (expect 2)  FAIL
    after Py_XSETREF(slots[0], b): refcnt(a)=3 (expect 1), refcnt(b)=1 (expect 3)  FAIL
    calling Py_CLEAR(slots[0]) ...
    Segmentation fault         (core dumped) ./repro
    
  9. vstinner commented on Sep 17, 2026

    @vstinner
    Member

    When running g++ -E, the C++ code:

        PyObject *slots[1];
        Py_CLEAR(slots[0]);

    becomes:

    PyObject *slots[1];
    decltype(&(slots[0])) _tmp_op_ptr = &(slots[0]);
    decltype(slots[0]) _tmp_old_op = (*_tmp_op_ptr);
    if (_tmp_old_op != nullptr) {
        *_tmp_op_ptr = nullptr;
        Py_DECREF((PyObject*)_tmp_old_op);
    }

    I can reproduce the crash with this simple C++ code which doesn't use Python:

    struct PyObject {
        int data;
    };
    typedef struct PyObject PyObject;
    
    void Py_DECREF(PyObject *obj)
    {
        obj->data = 0;
    }
    
    int main()
    {
        PyObject obj = {1};
        PyObject *slots[1] = {&obj};
    
        decltype(&(slots[0])) _tmp_op_ptr = &(slots[0]);
        decltype(slots[0]) _tmp_old_op = (*_tmp_op_ptr);
        if (_tmp_old_op != nullptr) {
            *_tmp_op_ptr = nullptr;
            Py_DECREF((PyObject*)_tmp_old_op);
        }
        return (obj.data == 0);
    }

    The following C++ program doesn't crash, it uses Python 3.15 Py_CLEAR() implementation:

    #include <string.h>
    
    struct PyObject {
        int data;
    };
    typedef struct PyObject PyObject;
    
    void Py_DECREF(PyObject *obj)
    {
        obj->data = 0;
    }
    
    int main()
    {
        PyObject obj = {1};
        PyObject *slots[1] = {&obj};
    
        PyObject **_tmp_op_ptr = &(slots[0]);
        PyObject *_tmp_old_op = (*_tmp_op_ptr);
        if (_tmp_old_op != nullptr) {
            PyObject *_null_ptr = nullptr;
            memcpy(_tmp_op_ptr, &_null_ptr, sizeof(PyObject*));
            Py_DECREF((PyObject*)_tmp_old_op);
        }
        return (obj.data == 0);
    }
  10. lucascolley commented on Sep 17, 2026

    @lucascolley
    ContributorAuthor

    The reproducer does crash without ASAN

    (indeed, this diff reproduces the SIGSEGV (albeit without the memory address traceback))

    diff --git a/pixi.toml b/pixi.toml
    index 25be922..ed8b9cb 100644
    --- a/pixi.toml
    +++ b/pixi.toml
    @@ -15,3 +15,3 @@ numpy = {
           python = {
    -        flags = ["asan"],
    +        flags = ["default"],
             # passing
  11. changed the title [-]ASan regression on the main branch in Py_CLEAR/Py_SETREF: SEGV with simple `numpy` program[/-] [+]Py_CLEAR/Py_SETREF() regression on the main branch with C++: SEGV with simple `numpy` program[/+] on Sep 17, 2026
  12. 21 remaining items

  13. da-woods commented on Sep 24, 2026

    @da-woods
    Contributor

    The template version looks fine though and not too much more verbose than auto. So "use that and avoid an extended debate on which C++ features to allow" has an obvious advantage.

  14. da-woods commented on Sep 24, 2026

    @da-woods
    Contributor

    The template version looks fine

    ... Although would probably have to be outside the extern "C" block that I think the headers are wrapped in in c++.

  15. vstinner commented on Sep 24, 2026

    @vstinner
    Member

    I created PR gh-158070 to (1) no longer use decltype() in _Py_TYPEOF and (2) use auto in Py_CLEAR/Py_SETREF on C++11 and newer. On C++03, Py_CLEAR/Py_SETREF is implemented with typeof() (GCC/clang) or memcpy() (MSVC).

    I wrote "Using decltype() produced invalid code when clearing/setting an array item" in the Changelog. I didn't understand well the decltype() issue, so I'm not sure how to explain it.

    I'm less excited by using template since the last time we added template, we got more issues on C++ (the implementation had to be fixed multiple times, to have to remove template at the end to fix the last bug). Using auto looks simple to me, and it seems to be well supported on C++11 and newer.

    Yes, test_cppext makes sure that it's possible to write a Python extension using C++03. It makes sure that no C++ warning is emitted. If you want a clear statement on C++03 support, please open an issue at https://github.com/capi-workgroup/decisions/issues/.

    My implementation simply uses typeof() or memcpy() on C++03, instead of using C++11 auto. Supporting C++03 is not a big deal, and I suppose that it allows some users to stay on C++03 for various reasons.

    The same way, we try to support C older than C11 in the Python C API. C11 is only needed to build Python, not to use the Python C API. For example, test_cext has a test on C99.

  16. added 7 commits that reference this issue on Sep 25, 2026
  17. vstinner commented on Sep 28, 2026

    @vstinner
    Member

    Thanks for your bug report. The issue should now be fixed in the main branch, and tests have been added to check for non regression.

  18. lucascolley commented on Sep 28, 2026

    @lucascolley
    ContributorAuthor

    thank you @vstinner and all for the extensive work here! Glad to have helped find this :)

  19. vstinner commented on Oct 4, 2026

    @vstinner
    Member

    If there are still C++ experts around, I would like your opinion about gh-158810 issue (Py_ARRAY_LENGTH). I'm not sure how to fix it.

  20. added a commit that references this issue on Oct 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    interpreter-core(Objects, Python, Grammar, and Parser dirs)type-bugAn unexpected behavior, bug, or error

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions