Repository navigation
Crash in DistributionFinder.context because it is using vars(self).update(kwargs) under memory preasure #152083
Description
Activity
Config:
make clean && ./configure --with-pydebug --disable-gil && make -j$(sysctl -n hw.ncpu)
Simplier repro:import faulthandler, importlib.metadata faulthandler.enable() from _testcapi import set_nomemory import abc class DistributionFinder: class Context: name = None def __init__(self, **kwargs): vars(self).update(kwargs) @abc.abstractmethod def find_distributions(self, context=Context()): pass for start in range(40): set_nomemory(start) try: DistributionFinder.Context(name='f') except BaseException: pass print("done, no crash")
Traceback:
done, no crash Assertion failed: (_Py_IsOwnedByCurrentThread((PyObject *)mp) || IS_DICT_SHARED(mp)), function set_keys, file dictobject.c, line 206. Fatal Python error: Aborted <Cannot show all threads while the GIL is disabled> Stack (most recent call first): Current thread's C stack trace (most recent call first): Binary file "/Users/sobolev/Desktop/cpython/python.exe", at _Py_DumpStack+0x44 [0x1007b2234] Binary file "/Users/sobolev/Desktop/cpython/python.exe", at faulthandler_dump_c_stack+0x68 [0x1007e19e0] Binary file "/Users/sobolev/Desktop/cpython/python.exe", at faulthandler_fatal_error+0x1ac [0x1007e177c] Binary file "/usr/lib/system/libsystem_platform.dylib", at _sigtramp+0x38 [0x1a00d16a4] Binary file "/usr/lib/system/libsystem_pthread.dylib", at pthread_kill+0x128 [0x1a009788c] Binary file "/usr/lib/system/libsystem_c.dylib", at abort+0x7c [0x19ffa0a3c] Binary file "/usr/lib/system/libsystem_c.dylib", at err+0x0 [0x19ff9fc70] Binary file "/Users/sobolev/Desktop/cpython/python.exe", at set_keys+0x70 [0x1004d5d3c] Binary file "/Users/sobolev/Desktop/cpython/python.exe", at PyObject_ClearManagedDict+0x144 [0x1004d58ac] Binary file "/Users/sobolev/Desktop/cpython/python.exe", at subtype_clear+0xd8 [0x1005630ec] Binary file "/Users/sobolev/Desktop/cpython/python.exe", at delete_garbage+0x1c8 [0x10070ab20] Binary file "/Users/sobolev/Desktop/cpython/python.exe", at gc_collect_internal+0x2ec [0x100709474] Binary file "/Users/sobolev/Desktop/cpython/python.exe", at gc_collect_main+0x248 [0x100706708] Binary file "/Users/sobolev/Desktop/cpython/python.exe", at _PyGC_CollectNoFail+0x20 [0x100706984] Binary file "/Users/sobolev/Desktop/cpython/python.exe", at finalize_modules+0xc4 [0x1007658b0] Binary file "/Users/sobolev/Desktop/cpython/python.exe", at _Py_Finalize+0x1c4 [0x100764bac] Binary file "/Users/sobolev/Desktop/cpython/python.exe", at Py_FinalizeEx+0x14 [0x1007649e0] Binary file "/Users/sobolev/Desktop/cpython/python.exe", at Py_RunMain+0x28 [0x1007cb5b8] Binary file "/Users/sobolev/Desktop/cpython/python.exe", at pymain_main+0x64 [0x1007cba84] Binary file "/Users/sobolev/Desktop/cpython/python.exe", at Py_BytesMain+0x38 [0x1007cbad4] Binary file "/Users/sobolev/Desktop/cpython/python.exe", at main+0x24 [0x1003588fc] Binary file "/usr/lib/dyld", at start+0x17bc [0x19fcf6b98] [1] 14739 abort ./python.exe ex.pyThe crash seems different, but it might have the similar cause.
Hi @sobolevn ! I think the proper fix somewhere deep in the c-side, i can work on this issue if you dont mind =)
@deadlovelll please go ahead :)
Reacted by Timofei Ivankov@deadlovelll please go ahead :)
Hello! Just pushed the fix, would appreciate a review =)
Crash report
Original report: https://gist.github.com/devdanzin/bf9fec4554c58c2a279b05b7ff8e6d9b (it produces the correct repro, but everything else is not correct, since it is AI generated and can be fuzzy).
Here's the real picture.
This class raises an error:
cpython/Lib/importlib/metadata/__init__.py
Lines 742 to 750 in 1785f4b
Repro:
Error:
I tested this hack, no crash:
But, this is clearly not a proper fix. Because for some reason the exception is cleared somewhere.
After:
Linked PRs