Repository navigation
Document the circumstances where the locals() dict get updated #61746
Description
Activity
techtonik commented
on Mar 25, 2013 techtonikmannequinMannequinAuthor-locals() returns object that does't walk like a duck.
+locals() returns object that does't work like a dict.Much of the confusion with locals() comes from the fact that returned object is labelled a dict, but this heisendict changes its behavior when you're trying to introspect it. So, to give users a hint that this object is different, I propose to rename it to a different type and supply an appropriate help() for it.
The documentation seems pretty clear on the nature of the dict returned by locals(). [1] What documentation are you referring to?
[1] http://docs.python.org/3.4/library/functions.html#locals
Sorry, didn't mean to change the attributed version. However the 2.7 documentation says the same thing.
techtonik commented
on Mar 26, 2013 techtonikmannequinMannequinAuthor-
The documentation doesn't say that the content of this dict may change sporadically even if you don't call locals()
-
I am referring to inline documentation:
>>> h = locals() >>> type(h) <class 'dict'> >>> help(h)-
So the problem you are having is that inside functions the dict returned by locals() does not get updated when the "current local symbol table" changes? Keep in mind that for modules and classes (the other two execution blocks), the dict returned by locals() does get updated.
This difference reflects that fact that, in CPython at least, the local execution namespace of a function call's stack frame is not stored in a dictionary. When you call locals() in a function it is only making a copy of the frame's fast locals. The fast locals never gets directly exposed except through normal name lookup/binding/deletion. While technically possible, changing this simply isn't worth it.
The documentation already makes it clear that the dict returned by locals() represents the "current local symbol table" and that you should not expect changes to that dict to be reflected in the actual locals. Are you recommending that it be more clear that the dict may be no more than a snapshot of the locals at the time locals() is called?
techtonik commented
on Mar 26, 2013 techtonikmannequinMannequinAuthorUnder the trace function, the dict is always updated, and that changed a workflow in the program I was debugging leading to heisenbug. But that's a different story. I'd like to concentrate on the reasons to rename locals() result type from 'dict' to 'livedict' (or other internal type with well-defined behavior).
dict() returned by locals() is sporadically updated. The problem is not with when it is updated, or how it is updated. The problem is that the fact that it is updated is not indicated in any way. And the problem that comes from this problem is that people don't realize that this dict may change its contents. As a result people become confused with Python.
The documentation is also ambiguous. Those people who don't know about self-updating behavior (I was) read "represents" like a "copy of symbol table", not like a "internally updated view of symbol table".
From the OOP point of view, the object doesn't behave like a normal dict, and it will be less confusing if it is renamed.
offtopic: "local execution namespace of a function call's stack frame" sounds cryptic - I am afraid that without visualization of local/global/namespace/scope/frame I am unable to understand that right now.
offtopic2: I also don't get "you should not expect changes to that dict to be reflected in the actual locals". I reads as if in some cases modifying the dict will change the locals. I thought that it is impossible as it is just a copy.
Thanks for the details, Anatoly. I am surprised by the behavior you've described. It may be that functions don't use fast locals when tracing is turned on. I'll have to check. If that's the case, I think the documentation for locals() should be improved. Either way, I agree that inconsistent (and undocumented) behavior is a headache. However, changing the type of the object returned by locals() is going to take a lot more justification. A doc update should be sufficient and a much easier sell.
Here are some further questions:
- the dynamic update you described happens in function bodies (i.e. stack frames)?
- does that behavior happen when you are not using a tracing function?
Also, if you have a minute, throw up a patch that reproduces the behavior you're talking about. That will help get this resolved faster.
Keep in mind that locals() always returns a normal dict. Just like any other dict, other code that has a reference to that "locals" dict can interact with it, which is what you have described. In this case, the interpreter is doing so. The only case where I find that surprising is where a stack frame is using fast locals, which is what function calls normally do.
- changed the title
[-]rename type returned by locals() to livedict[/-][+]Document the circumstances where the locals() dict gets updated[/+]on Mar 27, 2013 techtonik commented
on Mar 27, 2013 techtonikmannequinMannequinAuthorRaymond, could you please get the title back? You unintentionally hijacked the issue. The "Document the circumstances where the locals() dict gets updated" is not a describing title for this issue - it is one of the possible action items. I'd prefer to see a separate issue to document the stuff that "blocks" this one. It is quite evident that this action is required, but IMHO there is not enough evidence to conclude that it will be sufficient to close this ticket.
Eric, I need more time to answer your questions.
The previous title, "rename type returned by locals() to livedict" did not describe the reality, since locals() returns a regular dict.
[Would you call x.__dict__ a livedict?]So either this issue should be closed as invalid, because it's based on incorrect understandings of Python internals; or we could improve documentation about locals() in order to remove surprising behavior and (real) confusion in programmers' minds.
24 remaining items
Hmm. Perhaps the last sentence could be "... because changes to the local dict propagating to the local namespace cannot be relied upon to either happen or not happen". That would make it less redundant, since it would essentially be referencing the previous statement in the specific case of the consequences of modification.
The original included the caution against modifying it, and I think it is valid because of the inconsistent behavior. Perhaps it could be weakened to "it is not a good idea to modify"?
- changed the title
[-]Document the circumstances where the locals() dict gets updated[/-][+]Document the circumstances where the locals() dict get updated[/+]on Mar 31, 2013 techtonik commented
on Mar 31, 2013 techtonikmannequinMannequinAuthor... cannot be relied upon to either happen or not happen...
IMHO this phrase is from Advanced English course.
The original included the caution against modifying it, and I think it is
valid because of the inconsistent behavior.
Perhaps it could be weakened to "it is not a good idea to modify"?Ambiguity already adds fear and uncertainty. Just explaining the
consequences would be good style for the technical doc.Combined the second and last lines, discarded duplication.
Your formulation is more concise, thank you.
I suggest dropping the word 'additionally'. Also, "how it does" would be better phrased as "how it changes", I think. (It really should be "whether and how it changes", but in deference to Anatoly's 'advanced English' comment I'm willing to let that imprecision slide).
What about instead of
'''
Whether changes to one are reflected in the other after the call returns is undefined; additionally, the dictionary may change unpredictably after the call, and how it does is implementation-specific.
'''substitue this wording:
'''
Whether changes to one are reflected in the other after the call returns, and when such updates occur, is undefined and implementation-specific.
'''The old wording seems under-specified. It would allow a function call, garbage collection, etc, to clobber the dictionary, say overwriting with another function’s locals(), before you get a chance to work with the dictionary.
Yeah, the question of thread-safety in regards to what we are talking about here also occurred to me. That is, the wording makes one wonder if locals is thread safe or not. I don't see your suggested wording as making it clearer, though.
The problem is that it *is* underspecified.
So I think the correct description of the current under-specification is that locals() returns a copy of the current locals namespace. If you modify the thing returned by locals it may or may not update the local namespace. If you modify the local namespace (by doing x = y or its equivalents) the change may or may not be reflected in the object that was returned by locals().
Now, how do we boil that down? Or do we just say it more or less that way?
Here is another attempt with different words:
'''
.. note::
The dictionary returned by :func:`locals` is an accurate snapshot of the local namespace at the time it is called. If the namespace changes after the call, the dictionary may become out of date, but it may also automatically update at any time. The contents of the dictionary should not be modified by the user; it is undefined whether such changes affect the namespace or not.
'''I mostly like Martin's suggested wording, but would also note that I filed bpo-17960 to tighten up the requirements for when we expect assigning to locals() to work.
To save folks reading the whole referenced email, I think it would be worth defining that modifying the namespace returned by locals() will affect the runtime namespace at module and class scope and when using exec, but will have no effect at function scope (as locals() returns a copy of the namespace in that case).
I quickly scanned through the email thread from bpo-17960. I guess it makes sense to specify that locals() can be used to directly get a class’s namespace. Probably doesn’t hurt to say locals() is equivalent to globals() at module level, although this seems like a fairly redundant feature.
Here is locals_doc.03.patch, which uses my wording for function namespaces, and also adds more for class and global namespaces, as suggested by Nick.
Some minor tweaks to my earlier patch:
- list comprehension → comprehension
- time it is called → time of the call
PEP 667 has resolved this issue for Python 3.13b1
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields: