Repository navigation
Segfault when raising MemoryError #85820
Description
Activity
hoefling commented
on Aug 28, 2020 hoeflingmannequinMannequinAuthorFirst of all, I guess this is a somewhat obscure error that is unlikely to occur in a usual context, nevertheless IMO worth reporting. We observed this when unit-testing custom exception reporting mechanism, raising different exceptions in different contexts and then analyzing whether they are processed correctly.
This is a somewhat dull example I managed to extract from our tests:
from pathlib import Path from unittest.mock import patch class TestException(MemoryError): pass class report_ctx: def __enter__(self): return self def __exit__(self, exc_type, exc_value, tb): report(exc_value) class raises: def __init__(self, ex): self.ex = ex def __enter__(self): return self def __exit__(self, exc_type, exc_value, tb): return issubclass(exc_type, self.ex) def report(ex): pass def error(): raise MemoryError modname = Path(__file__).stem for _ in range(10): with patch(f"{modname}.report"): with raises(MemoryError), report_ctx(): raise MemoryError with raises(TestException): raise TestException with raises(MemoryError): error()
that yields:
Fatal Python error: Segmentation fault
Current thread 0x00007fcf0833b740 (most recent call first):
File "/home/oleg.hoefling/projects/private/python-memoryerror-segfault/main.py", line 38 in <module>
File "<frozen importlib._bootstrap>", line 228 in _call_with_frames_removed
File "<frozen importlib._bootstrap_external>", line 790 in exec_module
File "<frozen importlib._bootstrap>", line 680 in _load_unlocked
File "<frozen importlib._bootstrap>", line 986 in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 1007 in _find_and_load
File "/usr/lib64/python3.9/unittest/mock.py", line 1236 in _importer
File "/usr/lib64/python3.9/unittest/mock.py", line 1564 in <lambda>
File "/usr/lib64/python3.9/unittest/mock.py", line 1389 in __enter__
File "/home/oleg.hoefling/projects/private/python-memoryerror-segfault/main.py", line 36 in <module>hoefling commented
on Aug 28, 2020 hoeflingmannequinMannequinAuthorIf this is of any help, I've set up an example repository containing the snippet: https://github.com/hoefling/bpo-issue-41654
Here are the results of running the snippet in Travis with Python 3.{5-10} and Pypy 3.6: https://travis-ci.com/github/hoefling/bpo-issue-41654
Aha, interesting bug report. I attached a simplified reproducer.
Output with a Python built in debug mode:
---
Objects/frameobject.c:590: _Py_NegativeRefcount: Assertion failed: object has negative ref count
<object at 0x25e8570 is freed>
Fatal Python error: _PyObject_AssertFailed: _PyObject_AssertFailed
Python runtime state: initializedCurrent thread 0x00007efd7a2d0740 (most recent call first):
File "/home/vstinner/bug.py", line 29 in <module>
Abandon (core dumped)
---Hm, Looks like we need check the double free risk of
Py_DECREF().The problem is that the deallocator of MemoryError class is not taking into account subclasses for the freelist management, but the allocator (tp_new) is.
Thanks for the fix Pablo! Thanks for the funny bug report Oleg Hoefling! I didn't expect that anyone would ever subclass MemoryError.
"AMD64 Arch Linux Asan 3.8" buildbot logs a compiler warning:
https://buildbot.python.org/all/#builders/580/builds/4Objects/exceptions.c: In function ‘MemoryError_dealloc’:
Objects/exceptions.c:2298:23: warning: comparison of distinct pointer types lacks a cast
2298 | if (Py_TYPE(self) != PyExc_MemoryError) {
| ^~
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields: