Crash when inspecting frame of generator

@markshannon

Crash report

This was discovered in an asyncio program when interrupted with CTRL - C.
Minimal Reproducer:

import gc
import inspect

gc.set_threshold(1, 0, 0)
f = []

def cb(*args):
    f.append(inspect.stack())

gc.callbacks.append(cb)

def gen():
    yield 1

g = gen()
g.__next__()

Error messages

python: Python/ceval.c:5436: _PyEval_EvalFrameDefault: Assertion `frame->frame_obj == NULL' failed.
Aborted (core dumped)

Your environment

CPython versions tested on: Python 3.12.0a0 51fd4de, Python 3.11.0b3+ 41e4b42
Operating system and architecture: Linux 5.4.0-1074-azure

cc @markshannon @pablogsal

This is tricky.

The fundamental problem, IMO, is that almost any memory allocation can run arbitrary code. This makes it really difficult to reason about the state of the VM.

In the short term, we should handle the case where frame->frame_obj != NULL when creating a generator, but there are bound other issues, as the generator and frame object are may not be an introspectable state.

In the longer term, I'd like to only run the GC, and/or finalizers, when the VM is a known state; as we do with interrupts.

I'm concerned that this shows up from an interrupt. It shouldn't.

The fundamental problem, IMO, is that almost any memory allocation can run arbitrary code. This makes it really difficult to reason about the state of the VM.

I agree, not only the state of VM but also any other object state can be mutated which in some can lead to memory leak or worse crash.

I'm concerned that this shows up from an interrupt. It shouldn't.

It was really hard to reproduce the error, my current theory is that if an interrupt happens while executing __del__ of an object and the interpreter creates a traceback to print, it can create a frame object for a generator and it causes a hard crash while the exception is propagated to the generator.

I was correct about my theory, the following code sample demonstrates how KeyboardInterrupt can lead to crash:

import gc
gc.set_threshold(1, 0, 0)

class Sneaky:
    def __del__(self):
        raise KeyboardInterrupt


def gen():
    yield 1

sneaky = Sneaky()
sneaky._s = Sneaky()
sneaky._s._s = sneaky
del sneaky

g = gen()

Crash:

Exception ignored in: <function Sneaky.__del__ at 0x7fbfa49220a0>
Traceback (most recent call last):
  File "/workspaces/cpython/main.py", line 6, in __del__
    raise KeyboardInterrupt
    ^^^^^^^^^^^^^^^^^^^^^^^
KeyboardInterrupt: 
Exception ignored in: <function Sneaky.__del__ at 0x7fbfa49220a0>
Traceback (most recent call last):
  File "/workspaces/cpython/main.py", line 6, in __del__
    raise KeyboardInterrupt
    ^^^^^^^^^^^^^^^^^^^^^^^
KeyboardInterrupt: 
python: Python/ceval.c:5477: _PyEval_EvalFrameDefault: Assertion `frame->frame_obj == NULL' failed.

@kumaraditya303

Thanks @kumaraditya303 that's really helpful.

Turns out that the fix isn't that hard.

The problem isn't really the GC behavior* but that partially constructed frames are visible to Python and C extensions.
We fix this by not creating frame objects for partially constructed frames.

*The behavior of the GC and finalization is a problem, but not the problem here.

GH-94262: Don't create frame objects for frames that aren't yet compl…

pythonGH-94262: Don't create frame objects for frames that aren't yet…

[3.11] GH-94262: Don't create frame objects for frames that aren't ye…

@markshannon

Thanks, @markshannon and @kumaraditya303 for the work! 🎉

Navigation Menu

Repository navigation

Crash when inspecting frame of generator #94262

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Crash when inspecting frame of generator #94262

Description

Activity

markshannon commented on Jun 27, 2022

kumaraditya303 commented on Jun 27, 2022

kumaraditya303 commented on Jun 27, 2022

markshannon commented on Jun 27, 2022

markshannon commented on Jun 28, 2022

pablogsal commented on Jul 4, 2022

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions