Skip to content

_json: Use After Free in encoder_encode_key_value() - borrowed 'key' used after free #145244

Closed
Closed
_json: Use After Free in encoder_encode_key_value() - borrowed 'key' used after free#145244
Labels
extension-modulesC modules in the Modules dirtype-bugAn unexpected behavior, bug, or error

Description

@raminfp
raminfp
opened on Feb 26, 2026

Bug report

Bug description:

In encoder_encode_key_value(), 'key' is a borrowed reference from
PyDict_Next(). If the 'default' callback (invoked during value encoding)
clears the dict, 'key' becomes a dangling pointer. The subsequent
_PyErr_FormatNote("%R", key) calls PyObject_Repr() on freed memory.

Root Cause

  // Modules/_json.c — encoder_encode_key_value()
  // 'key' is a borrowed ref from PyDict_Next()
  if (encoder_listencode_obj(s, writer, value, ...) < 0) {
      _PyErr_FormatNote("when serializing %T item %R", dct, key);
      //                                                    ^^^
      //                          key may be freed if default() cleared the dict
  }

encoder_listencode_obj() can invoke the user-supplied default callable. If default clears or mutates the dict being iterated, the borrowed key reference becomes dangling. The subsequent %R format triggers PyObject_Repr()on freed memory.

import json


class Evil:
    """Value object whose encoding triggers the vulnerability."""
    pass


class AlsoEvil:
    """Second object that fails to encode, triggering the error path."""
    pass


big_key = "A" * 100  # non-interned key so it can be freed
target_dict = {big_key: Evil()}
del big_key  # only the dict holds refs now


def evil_default(obj):
    if isinstance(obj, Evil):
        target_dict.clear()  # frees both key and value
        return AlsoEvil()
    raise TypeError("not serializable")


json.dumps(target_dict, default=evil_default, check_circular=False)

ASAN Output :

=================================================================
==64296==ERROR: AddressSanitizer: heap-use-after-free on address 0x513000044638 at pc 0x5a84e3c1b8ca bp 0x7ffd761be3f0 sp 0x7ffd761be3e0
READ of size 8 at 0x513000044638 thread T0
    #0 0x5a84e3c1b8c9 in _Py_TYPE_impl ../Include/object.h:313
    #1 0x5a84e3c1b8c9 in unicode_fromformat_arg ../Objects/unicodeobject.c:2986
    #2 0x5a84e3c1bd0a in unicode_from_format ../Objects/unicodeobject.c:3075
    #3 0x5a84e3c1beba in PyUnicode_FromFormatV ../Objects/unicodeobject.c:3109
    #4 0x5a84e3d858e7 in _PyErr_FormatNote ../Python/errors.c:1259
    #5 0x7de06590d67b in encoder_listencode_obj ../Modules/_json.c:1644
    #6 0x7de06590e273 in encoder_encode_key_value ../Modules/_json.c:1721
    #7 0x7de06590e829 in _encoder_iterate_dict_lock_held ../Modules/_json.c:1782
    #8 0x7de06590ee86 in encoder_listencode_dict ../Modules/_json.c:1861
    #9 0x7de06590d42a in encoder_listencode_obj ../Modules/_json.c:1605
    #10 0x7de06590d870 in encoder_call ../Modules/_json.c:1464
    #11 0x5a84e3a6aabe in _PyObject_MakeTpCall ../Objects/call.c:242
    #12 0x5a84e3a6ad66 in _PyObject_VectorcallTstate ../Include/internal/pycore_call.h:134
    #13 0x5a84e3a6adbf in PyObject_Vectorcall ../Objects/call.c:327
    #14 0x5a84e3ce9cf7 in _Py_VectorCallInstrumentation_StackRefSteal ../Python/ceval.c:769
    #15 0x5a84e3cf9cbb in _PyEval_EvalFrameDefault ../Python/generated_cases.c.h:1817
    #16 0x5a84e3d30f30 in _PyEval_EvalFrame ../Include/internal/pycore_ceval.h:118
    #17 0x5a84e3d31296 in _PyEval_Vector ../Python/ceval.c:2132
    #18 0x5a84e3d3154c in PyEval_EvalCode ../Python/ceval.c:680
    #19 0x5a84e3e34d31 in run_eval_code_obj ../Python/pythonrun.c:1366
    #20 0x5a84e3e35077 in run_mod ../Python/pythonrun.c:1469
    #21 0x5a84e3e35fac in pyrun_file ../Python/pythonrun.c:1294
    #22 0x5a84e3e38de2 in _PyRun_SimpleFileObject ../Python/pythonrun.c:518
    #23 0x5a84e3e3908e in _PyRun_AnyFileObject ../Python/pythonrun.c:81
    #24 0x5a84e3e8e6b6 in pymain_run_file_obj ../Modules/main.c:410
    #25 0x5a84e3e8e923 in pymain_run_file ../Modules/main.c:429
    #26 0x5a84e3e90121 in pymain_run_python ../Modules/main.c:691
    #27 0x5a84e3e907b7 in Py_RunMain ../Modules/main.c:772
    #28 0x5a84e3e909a3 in pymain_main ../Modules/main.c:802
    #29 0x5a84e3e90d28 in Py_BytesMain ../Modules/main.c:826
    #30 0x5a84e38f5675 in main ../Programs/python.c:15
    #31 0x7de06662a3b7 in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58
    #32 0x7de06662a47a in __libc_start_main_impl ../csu/libc-start.c:360
    #33 0x5a84e38f55a4 in _start (/home/raminfp/Projects/cpython/build-asan/python+0x2ee5a4) (BuildId: b6692a1e94ed1222fac0cb716d806843accd7c52)

0x513000044638 is located 56 bytes inside of 352-byte region [0x513000044600,0x513000044760)
freed by thread T0 here:
    #0 0x7de066afc4d8 in free ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:52
    #1 0x5a84e3b38f90 in _PyMem_RawFree ../Objects/obmalloc.c:93
    #2 0x5a84e3b3b44a in _PyMem_DebugRawFree ../Objects/obmalloc.c:3092
    #3 0x5a84e3b3b48b in _PyMem_DebugFree ../Objects/obmalloc.c:3237
    #4 0x5a84e3b63854 in PyObject_Free ../Objects/obmalloc.c:1659
    #5 0x5a84e3da44e7 in PyObject_GC_Del ../Python/gc.c:2460
    #6 0x5a84e3b7ee36 in object_dealloc ../Objects/typeobject.c:7288
    #7 0x5a84e3b9d7b8 in subtype_dealloc ../Objects/typeobject.c:2864
    #8 0x5a84e3b2f587 in _Py_Dealloc ../Objects/object.c:3271
    #9 0x5a84e3d99cc6 in Py_DECREF_MORTAL ../Include/internal/pycore_object.h:425
    #10 0x5a84e3d99d7c in PyStackRef_XCLOSE ../Include/internal/pycore_stackref.h:703
    #11 0x5a84e3d9aea5 in _PyFrame_ClearLocals ../Python/frame.c:101
    #12 0x5a84e3d9b09d in _PyFrame_ClearExceptCode ../Python/frame.c:126
    #13 0x5a84e3ce4e0d in clear_thread_frame ../Python/ceval.c:1939
    #14 0x5a84e3ceb21e in _PyEval_FrameClearAndPop ../Python/ceval.c:1976
    #15 0x5a84e3d264b0 in _PyEval_EvalFrameDefault ../Python/generated_cases.c.h:10558
    #16 0x5a84e3d30f30 in _PyEval_EvalFrame ../Include/internal/pycore_ceval.h:118
    #17 0x5a84e3d31296 in _PyEval_Vector ../Python/ceval.c:2132
    #18 0x5a84e3a6a805 in _PyFunction_Vectorcall ../Objects/call.c:413
    #19 0x5a84e3a6accc in _PyObject_VectorcallTstate ../Include/internal/pycore_call.h:136
    #20 0x5a84e3a6ae8c in PyObject_CallOneArg ../Objects/call.c:395
    #21 0x7de06590d4f1 in encoder_listencode_obj ../Modules/_json.c:1628
    #22 0x7de06590e273 in encoder_encode_key_value ../Modules/_json.c:1721
    #23 0x7de06590e829 in _encoder_iterate_dict_lock_held ../Modules/_json.c:1782
    #24 0x7de06590ee86 in encoder_listencode_dict ../Modules/_json.c:1861
    #25 0x7de06590d42a in encoder_listencode_obj ../Modules/_json.c:1605
    #26 0x7de06590d870 in encoder_call ../Modules/_json.c:1464
    #27 0x5a84e3a6aabe in _PyObject_MakeTpCall ../Objects/call.c:242
    #28 0x5a84e3a6ad66 in _PyObject_VectorcallTstate ../Include/internal/pycore_call.h:134
    #29 0x5a84e3a6adbf in PyObject_Vectorcall ../Objects/call.c:327

previously allocated by thread T0 here:
    #0 0x7de066afd9c7 in malloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:69
    #1 0x5a84e3b398a7 in _PyMem_RawMalloc ../Objects/obmalloc.c:65
    #2 0x5a84e3b38c78 in _PyMem_DebugRawAlloc ../Objects/obmalloc.c:3024
    #3 0x5a84e3b38ce0 in _PyMem_DebugRawMalloc ../Objects/obmalloc.c:3057
    #4 0x5a84e3b3a68f in _PyMem_DebugMalloc ../Objects/obmalloc.c:3222
    #5 0x5a84e3b63710 in PyObject_Malloc ../Objects/obmalloc.c:1630
    #6 0x5a84e3b9574d in _PyObject_MallocWithType ../Include/internal/pycore_object_alloc.h:46
    #7 0x5a84e3b9574d in _PyType_AllocNoTrack ../Objects/typeobject.c:2516
    #8 0x5a84e3b958d9 in PyType_GenericAlloc ../Objects/typeobject.c:2547
    #9 0x5a84e3b9a2a4 in object_new ../Objects/typeobject.c:7278
    #10 0x5a84e3b9a6e1 in type_call ../Objects/typeobject.c:2460
    #11 0x5a84e3a6aabe in _PyObject_MakeTpCall ../Objects/call.c:242
    #12 0x5a84e3a6ad66 in _PyObject_VectorcallTstate ../Include/internal/pycore_call.h:134
    #13 0x5a84e3a6adbf in PyObject_Vectorcall ../Objects/call.c:327
    #14 0x5a84e3ce9cf7 in _Py_VectorCallInstrumentation_StackRefSteal ../Python/ceval.c:769
    #15 0x5a84e3cf9cbb in _PyEval_EvalFrameDefault ../Python/generated_cases.c.h:1817
    #16 0x5a84e3d30f30 in _PyEval_EvalFrame ../Include/internal/pycore_ceval.h:118
    #17 0x5a84e3d31296 in _PyEval_Vector ../Python/ceval.c:2132
    #18 0x5a84e3d3154c in PyEval_EvalCode ../Python/ceval.c:680
    #19 0x5a84e3e34d31 in run_eval_code_obj ../Python/pythonrun.c:1366
    #20 0x5a84e3e35077 in run_mod ../Python/pythonrun.c:1469
    #21 0x5a84e3e35fac in pyrun_file ../Python/pythonrun.c:1294
    #22 0x5a84e3e38de2 in _PyRun_SimpleFileObject ../Python/pythonrun.c:518
    #23 0x5a84e3e3908e in _PyRun_AnyFileObject ../Python/pythonrun.c:81
    #24 0x5a84e3e8e6b6 in pymain_run_file_obj ../Modules/main.c:410
    #25 0x5a84e3e8e923 in pymain_run_file ../Modules/main.c:429
    #26 0x5a84e3e90121 in pymain_run_python ../Modules/main.c:691
    #27 0x5a84e3e907b7 in Py_RunMain ../Modules/main.c:772
    #28 0x5a84e3e909a3 in pymain_main ../Modules/main.c:802
    #29 0x5a84e3e90d28 in Py_BytesMain ../Modules/main.c:826
    #30 0x5a84e38f5675 in main ../Programs/python.c:15

SUMMARY: AddressSanitizer: heap-use-after-free ../Include/object.h:313 in _Py_TYPE_impl
Shadow bytes around the buggy address:
  0x513000044380: 00 00 00 00 00 00 00 00 00 00 00 00 fa fa fa fa
  0x513000044400: fa fa fa fa fa fa fa fa 00 00 00 00 00 00 00 00
  0x513000044480: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x513000044500: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x513000044580: 00 00 00 00 fa fa fa fa fa fa fa fa fa fa fa fa
=>0x513000044600: fd fd fd fd fd fd fd[fd]fd fd fd fd fd fd fd fd
  0x513000044680: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x513000044700: fd fd fd fd fd fd fd fd fd fd fd fd fa fa fa fa
  0x513000044780: fa fa fa fa fa fa fa fa 00 00 00 00 00 00 00 00
  0x513000044800: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x513000044880: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==64296==ABORTING

Suggested Fix

Py_INCREF(key) before calling encoder_listencode_obj() and Py_DECREF(key) afterward to ensure the key stays alive through the error path.

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

Linked PRs

Activity

  1. raminfp
    added
    type-bugAn unexpected behavior, bug, or error
    on Feb 26, 2026
  2. cocolato
    added
    extension-modulesC modules in the Modules dir
    on Feb 26, 2026
  3. StanFromIreland
    added this to JSON issueson Feb 26, 2026

  4. nineteendo commented on Feb 26, 2026

    @nineteendo
    nineteendo
    on Feb 26, 2026
    Contributor

    Isn’t this the same issue as #142831?

  5. raminfp commented on Feb 26, 2026

    @raminfp
    raminfp
    on Feb 26, 2026
    ContributorAuthor

    This issue is NOT a duplicate of #142823

    #142831 occurs in _encoder_iterate_mapping_lock_held() when a crafted mapping's items() returns a mutable list. A custom key encoder callback clears that list mid-iteration, so the for loop's PyList_GET_ITEM(items, i) accesses freed list elements on subsequent iterations.

    This issue occurs in the standard dict iteration path (_encoder_iterate_dict_lock_held). Here, key and value are borrowed references from PyDict_Next(). When encoder_encode_key_value() calls encoder_listencode_obj(), which invokes the user-supplied default callback, that callback can clear or mutate the dict.

    This frees the borrowed key, and the subsequent _PyErr_FormatNote("when serializing %T item %R", dct, key) dereferences the dangling pointer.

    But i think the patch #142851 will solve this problem.

  6. gpshead
    added 3 commits that reference this issue on Apr 11, 2026

    gh-145244: Fix use-after-free on borrowed dict key in json encoder (G…

    8a466fa

    Move NEWS entry from Core_and_Builtins to Library

    646b447

    Add comments explaining why Py_INCREF is needed in iteration functions

    a412165

  7. gpshead commented on Apr 11, 2026

    @gpshead
    gpshead
    on Apr 11, 2026
    Member

    fwiw this was somewhat of a duplicate. pr merged. i'm taking care of getting the PRs for both in.

  8. gpshead
    closed this as completedon Apr 11, 2026
  9. github-project-automation
    moved this to Done in JSON issueson Apr 11, 2026
  10. gpshead
    added a commit that references this issue on Apr 12, 2026

    gh-142831: Fix use-after-free in json encoder during re-entrant mutat…

    235fa72
  11. ljfp
    added 2 commits that reference this issue on Apr 25, 2026

    pythongh-145244: Fix use-after-free on borrowed dict key in json enco…

    dd3b025

    pythongh-142831: Fix use-after-free in json encoder during re-entrant…

    1f896f8
  12. serhiy-storchaka
    added a commit that references this issue on Sep 5, 2026

    Merge branch 'main' into master

    0a473de
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    extension-modulesC modules in the Modules dirtype-bugAn unexpected behavior, bug, or error

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions