Repository navigation
Singleton MemoryError can hold traceback data and locals indefinitely #49687
Description
| BPO | 5437 |
|---|---|
| Nosy | @amauryfa, @pitrou, @avassalotti |
| Files |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = None
closed_at = <Date 2010-10-28.23:07:25.847>
created_at = <Date 2009-03-08.01:19:49.933>
labels = ['performance']
title = 'Singleton MemoryError can hold traceback data and locals indefinitely'
updated_at = <Date 2010-10-28.23:07:25.846>
user = 'https://github.com/pitrou'bugs.python.org fields:
activity = <Date 2010-10-28.23:07:25.846>
actor = 'pitrou'
assignee = 'none'
closed = True
closed_date = <Date 2010-10-28.23:07:25.847>
closer = 'pitrou'
components = []
creation = <Date 2009-03-08.01:19:49.933>
creator = 'pitrou'
dependencies = []
files = ['13261', '15230', '19403']
hgrepos = []
issue_num = 5437
keywords = ['patch']
message_count = 10.0
messages = ['83292', '83293', '83296', '89181', '94684', '110325', '112001', '119806', '119822', '119823']
nosy_count = 4.0
nosy_names = ['amaury.forgeotdarc', 'pitrou', 'alexandre.vassalotti', 'BreamoreBoy']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'resource usage'
url = 'https://bugs.python.org/issue5437'
versions = ['Python 3.1', 'Python 3.2']
Activity
pitrou commentedon Mar 8, 2009
The PyExc_MemoryErrorInst object is persistent and its members never get
cleared. This means any local variable which gets caught in the
traceback isn't deallocated until the next MemoryError (!). Sample
script demonstrates this.
(this doesn't seem to affect 2.x because the traceback isn't attached to
the exception instance)
pitrou commentedon Mar 8, 2009
The following patch fixes the case when the exception is discarded in
Python, but not when e.g. PyErr_Clear() is used from C code.
pitrou commentedon Mar 8, 2009
A proper fix would probably be to maintain a bunch of preallocated
instances in a freelist rather than relying on an explicit singleton. It
would enforce proper dereferencing and garbage collection semantics.
avassalotti commentedon Jun 10, 2009
Honestly, I don't think it is a big issue. MemoryErrors are rare and
typically cause the interpreter to shutdown.
By the way, do you think the static PyExc_RecursionErrorInst object is
affected by this bug?
pitrou commentedon Oct 29, 2009
Here is a patch preallocating a freelist of MemoryErrors.
To Alexandre: yes, PyExc_RecursionErrorInst probably has the same problem.
BreamoreBoy commentedon Jul 14, 2010
Would someone please check out thie patch, it's mostly changes to C code that I'm not qualified to comment on.
BreamoreBoy commentedon Jul 29, 2010
Changing priority as nobody appears to have lost any sleep over this issue.
pitrou commentedon Oct 28, 2010
Updated patch against py3k.
pitrou commentedon Oct 28, 2010
I can't manage to reproduce the issue with PyExc_RecursionErrorInst. It seems this instance is only used in a very select condition, that is, when the recursion limit is hit when trying to "normalize" an exception. Therefore, I will add a test without "fixing" anything.
pitrou commentedon Oct 28, 2010
Patch with improved tests committed in r85896 (3.2) and r85898 (3.1).