Creating a subinterpreter when Python is run with -X tracemalloc leads to segfault

Crash report

What happened?

Minimal reproducer:

import _interpreters

_interpreters.create()

python -X tracemalloc subtest.py

This also fails if one submits a task to concurrent.futures.InterpreterPoolExecutor.

Example output when run with -X tracemalloc:

Python/tracemalloc.c:705: _Py_NegativeRefcount: Assertion failed: object has negative ref count
<object at 0x200041899d0 is freed>
Fatal Python error: _PyObject_AssertFailed: _PyObject_AssertFailed
Python runtime state: finalizing (tstate=0x0000559f74721680)

Stack (most recent call first):
  <no Python frame>
Aborted (core dumped)

That points here:

cpython/Python/tracemalloc.c

Lines 701 to 706 in f478331

    
           static void 
        
           tracemalloc_clear_filename(void *value) 
        
           { 
        
               PyObject *filename = (PyObject *)value; 
        
               Py_DECREF(filename); 
        
           }

I presume the issue is that the filename is not populated for the subinterpreter?

I'm on Ubuntu 24.04 on an x86_64 machine if that makes a difference. My latest configure command line is:

./configure --config-cache --with-pydebug --disable-gil CC=clang-20

Note that this also reproduces with the GIL enabled, (but it just outputs an unhelpful "segmentation fault").

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

Output from running 'python -VV' on the command line:

Python 3.15.0a0 experimental free-threading build (heads/main:f478331f989, May 23 2025, 16:17:11) [Clang 20.1.5 (++20250430014901+ebfae55af454-1~~exp1~~20250430014920.111)]

Linked PRs

gh-134604: Fix tracemalloc crash with subinterpreters #134667

Looked into this a bit more. I'm not very familiar with subinterpreters internals or the Python runtime lifecycle, so apologies if the following is wrong.

It seems that tracemalloc holds references to frame information, which is not safe with multiple interpreters that may be destroyed before tracemalloc drops it's references. I was able to make the minimal reproducer pass by moving finalize_subinterpreters() after _PyTraceMalloc_Fini() is called:

cpython/Python/pylifecycle.c

Lines 2128 to 2159 in 2fd09b0

    
               finalize_subinterpreters(); 
        
               /* Print debug stats if any */ 
        
               _PyEval_Fini(); 
        
               /* Flush sys.stdout and sys.stderr (again, in case more was printed) */ 
        
               if (flush_std_files() < 0) { 
        
                   status = -1; 
        
               } 
        
               /* Collect final garbage.  This disposes of cycles created by 
        
                * class definitions, for example. 
        
                * XXX This is disabled because it caused too many problems.  If 
        
                * XXX a __del__ or weakref callback triggers here, Python code has 
        
                * XXX a hard time running, because even the sys module has been 
        
                * XXX cleared out (sys.stdout is gone, sys.excepthook is gone, etc). 
        
                * XXX One symptom is a sequence of information-free messages 
        
                * XXX coming from threads (if a __del__ or callback is invoked, 
        
                * XXX other threads can execute too, and any exception they encounter 
        
                * XXX triggers a comedy of errors as subsystem after subsystem 
        
                * XXX fails to find what it *expects* to find in sys to help report 
        
                * XXX the exception and consequent unexpected failures).  I've also 
        
                * XXX seen segfaults then, after adding print statements to the 
        
                * XXX Python code getting called. 
        
                */ 
        
           #if 0 
        
               _PyGC_CollectIfEnabled(); 
        
           #endif 
        
               /* Disable tracemalloc after all Python objects have been destroyed, 
        
                  so it is possible to use tracemalloc in objects destructor. */ 
        
               _PyTraceMalloc_Fini();

This works because frame(s) created by the _interpreters.create() call then live longer than tracemalloc's references to them.

However, that won't work in the general case. Since subinterpreters can be created and destroyed at will, tracemalloc could always potentially hang on to dead references. For example, the following leads to a segfault even after re-ordering finalization steps:

import _interpreters
import tracemalloc
tracemalloc.start()

interpid = _interpreters.create()
_interpreters.destroy(interpid)

I think a potential solution might be to copy the filename from the Python frame to the tracemalloc frame? That way the information is not lost when a subinterpreter is destroyed. If that does sound like a good idea, I can go ahead and work on a PR to do that.

Hm, copying is tricky because making a new unicode object can deadlock the interpreter, I guess we could store the filename as a char * to avoid that?

See also #126315, seems like tracemalloc needs more work to be safe with multiple threads/interpreters.

Navigation Menu

Repository navigation

Creating a subinterpreter when Python is run with -X tracemalloc leads to segfault #134604

Crash report

What happened?

CPython versions tested on:

Operating systems tested on:

Output from running 'python -VV' on the command line:

Linked PRs

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

	static void
	tracemalloc_clear_filename(void *value)
	{
	PyObject filename = (PyObject )value;
	Py_DECREF(filename);
	}

	finalize_subinterpreters();

	/* Print debug stats if any */
	_PyEval_Fini();

	/* Flush sys.stdout and sys.stderr (again, in case more was printed) */
	if (flush_std_files() < 0) {
	status = -1;
	}

	/* Collect final garbage. This disposes of cycles created by
	* class definitions, for example.
	* XXX This is disabled because it caused too many problems. If
	* XXX a __del__ or weakref callback triggers here, Python code has
	* XXX a hard time running, because even the sys module has been
	* XXX cleared out (sys.stdout is gone, sys.excepthook is gone, etc).
	* XXX One symptom is a sequence of information-free messages
	* XXX coming from threads (if a __del__ or callback is invoked,
	* XXX other threads can execute too, and any exception they encounter
	* XXX triggers a comedy of errors as subsystem after subsystem
	* XXX fails to find what it expects to find in sys to help report
	* XXX the exception and consequent unexpected failures). I've also
	* XXX seen segfaults then, after adding print statements to the
	* XXX Python code getting called.
	*/
	#if 0
	_PyGC_CollectIfEnabled();
	#endif

	/* Disable tracemalloc after all Python objects have been destroyed,
	so it is possible to use tracemalloc in objects destructor. */
	_PyTraceMalloc_Fini();

Creating a subinterpreter when Python is run with -X tracemalloc leads to segfault #134604

Description

Crash report

What happened?

CPython versions tested on:

Operating systems tested on:

Output from running 'python -VV' on the command line:

Linked PRs

Activity

emmatyping commented on May 25, 2025

emmatyping commented on May 25, 2025

emmatyping commented on May 26, 2025

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions