Repository navigation
Support for weak references in frames #102960
Description
It's occasionally useful to be able to link information to active frames. Two recent examples:
- https://stackoverflow.com/questions/72270154/weakref-weakkeydictionary-to-frame-frametype-objects
- gh-102864: Remove access to f_locals in pdb #102865
Currently, since frames don't support being weakly referenced, custom lifecycle management is required in this case. That's unfortunate because errors in cleaning out the references will lead to memory leaks of dead frames and their respective locals.
The straightforward solution is to add weakref support to frame objects. The performance hit is limited to setting NULL on the f_weakreflist field on frame object creation, and later a NULL check (for whether there are any weakrefs) at the frame object's deallocation. This is what the related PR does.
I ran the PR and the base main branch at the time against pyperformance 1.0.6. The raw results are in benchmarks.zip but the gist is that there are no significant differences between the two branches in any of the benchmarks executed.
Activity
markshannon commentedon Mar 23, 2023
Do you have a link to the results in a legible form?
ambv commentedon Mar 23, 2023
I'll get that for you tomorrow. In the mean time you can:
ambv commentedon Mar 24, 2023
The only "Significant" one is
tornado_httpand this one claims it's in favor of the weakreflist introduction so clearly noise.glyph commentedon Mar 29, 2023
There's some Deferred debugging stuff that has caused significant angst with hard references to frame objects. Weakrefs here would be hugely useful. I'm not sure exactly how we'd use them (the cycles and attendant leaks were such a problem that we have mostly given up on this sort of thing and I'd need to load the whole feature back into my head) but I definitely want this if it is a possibility!
tacaswell commentedon Mar 30, 2023
We have an open issue on Matplotlib due to frame related reference cycles matplotlib/matplotlib#25406
markshannon commentedon Mar 30, 2023
This is not the way to fix memory leaks.
If you really must have cycles involving frames, then you can break the cycle like this:
class SupportsWeakRefs: passframe -> weakref() -> SupportsWeakRefs() -> frame.Although I have to wonder why
plt.savefigneeds to do strange things to the stack.tacaswell commentedon Mar 30, 2023
plt.savefigis how the use found it but the cycles are from calls much deeper in the code (we were caching anExceptionobject 🐑 , and it looks like some versions of pillow and pyparsing also are doing something with cached exceptions). I do not think there is a compelling reason we (Matplotlib) needs this, but we were bit by frame cycles someplace.gaogaotiantian commentedon Jan 20, 2025
@ambv and @markshannon , do we have plans to move forward on this?
pdbkind of needs it.pdb(andbdb) often keeps some data for a specific frame, and it should last during the frame life cycle. It's a bit cumbersome to properly release the reference to the frames - either we hold it for unnecessarily long (and possibly forever), or we need a lot of extra code to release the reference at different places.gaogaotiantian commentedon Feb 16, 2025
ping here again for whether we can have this - doesn't seem like there are a lot of downsides. The perf looks the same and the code change is trivial.
pdb and bdb can use this to keep frame specific data and it can be really helpful.
markshannon commentedon Jun 30, 2026
Anyone want to implement this?
@gaogaotiantian?
@ambv?
ambv commentedon Jun 30, 2026
I will implement this 👍🏻
gh-102960: Make frames weak-referenceable (#152838)