Repository navigation
__debug__ is not optimized out at compile time for anything but if: and while: blocks #71356
Description
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = 'https://github.com/serhiy-storchaka'
closed_at = <Date 2017-12-15.11:26:53.815>
created_at = <Date 2016-05-31.18:50:58.224>
labels = ['interpreter-core', 'type-bug', '3.7']
title = '__debug__ is not optimized out at compile time for anything but `if:` and `while:` blocks'
updated_at = <Date 2017-12-18.13:11:58.271>
user = 'https://github.com/MojoVampire'bugs.python.org fields:
activity = <Date 2017-12-18.13:11:58.271>
actor = 'serhiy.storchaka'
assignee = 'serhiy.storchaka'
closed = True
closed_date = <Date 2017-12-15.11:26:53.815>
closer = 'serhiy.storchaka'
components = ['Interpreter Core']
creation = <Date 2016-05-31.18:50:58.224>
creator = 'josh.r'
dependencies = []
files = []
hgrepos = []
issue_num = 27169
keywords = ['patch']
message_count = 7.0
messages = ['266769', '266771', '266780', '308378', '308383', '308550', '308552']
nosy_count = 6.0
nosy_names = ['arigo', 'nascheme', 'SilentGhost', 'serhiy.storchaka', 'eryksun', 'josh.r']
pr_nums = ['4880', '4882', '4916', '4918']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue27169'
versions = ['Python 3.6', 'Python 3.7']
Activity
MojoVampire commentedon May 31, 2016
Issue bpo-22091 points out a quirk in the compile function and use of the debug "constant" causing inconsistent behavior when the optimize level of the compile call differs from that of the main interpreter; debug in an
iforwhilestatement is compiled out, but all other uses load it dynamically (at runtime), so in a mixed environment (interpreter at optimization=0, compile at optimize=2), you get non-obvious behavior.This behavior appears to be a consequence of debug being handled by a special case for the
ifandwhilestatements in compile.c that statements of that form to be compiled out, but not for similar constructs, e.g.a if __debug__ else band__debug__ or aare always evaluated at runtime, whether or notcompileis involved. Theexpr_constantfunction here https://hg.python.org/cpython/file/fd0ac7ba091e/Python/compile.c#l3542 is responsible for this (and only called in the same file, forifandwhilestatements).I'm not sure I understand the peephole optimizer, but if it can operate recursively (that is, an initial replacement of A->B where B could be optimized from B->C is optimized in a subsequent pass, turning all uses of A to C eventually), it seems like the "correct" solution would be to piggyback on optimizations for True and False, by having the peephole optimizer replace LOAD_NAME/LOAD_GLOBAL for debug with an appropriate LOAD_CONST, True or False, based on the compile environment. This would fix this bug (making debug evaluate in the
compilecall, so the environment when the compiled code is executed doesn't matter), and it would optimize all the other cases that the current special cases forifandwhiledon't cover by letting the recursive pass optimize them out the same way uses of literal True and False is optimized, so uses of ternary expressions, non-if/whileboolean operations, and all other operations using debug are optimized using a constant (and possibly optimized out of existence) without needing to independently maintain separate optimizations all over the codebase for debug. Might also allow the removal of the explicit special case for debug in compile.c forifandwhile, simplifying that code a bit.This would fix the problem from bpo-22091 and also make __debug__ reliably useful for "optimization", since all uses of it would "compile out" to optimized runtime code, where right now only two cases do so.
Does this seem reasonable?
Note: I added the nosy list from bpo-22091 here, since that bug is really just a special case of this one for
compileonly.serhiy-storchaka commentedon May 31, 2016
I think that the complete solution of this problem is making __debug__ a named constant like None, True, False.
This needs changing the grammar of Python.
MojoVampire commentedon May 31, 2016
That would also work. The argument I'd give in favor of performing a pass that replaces it with a literal True or False is that you don't have update as many places, don't have to worry about missing a place, and you don't have to decide if __debug__ is a reference to True or False, or a new object entirely.
It's just too easy to miss a case where __debug__ should be special and not notice (because optimizations aren't heavily tested for specific byte code outputs or anything), where a missed optimization for the True or False constant is much less likely to go unnoticed.
serhiy-storchaka commentedon Dec 15, 2017
New changeset 3325a67 by Serhiy Storchaka in branch 'master':
bpo-27169: The __debug__ constant is now optimized out at compile time. (bpo-4880)
3325a67
serhiy-storchaka commentedon Dec 15, 2017
New changeset b82da9e by Serhiy Storchaka in branch '3.6':
[3.6] bpo-27169: The __debug__ constant is now optimized out at compile time. (GH-4880) (bpo-4882)
b82da9e
serhiy-storchaka commentedon Dec 18, 2017
New changeset bd6ec4d by Serhiy Storchaka in branch 'master':
bpo-32365: Fix a reference leak when compile __debug__. (bpo-4916)
bd6ec4d
serhiy-storchaka commentedon Dec 18, 2017
New changeset 5659743 by Serhiy Storchaka (Miss Islington (bot)) in branch '3.6':
bpo-32365: Fix a reference leak when compile __debug__. (GH-4916) (bpo-4918)
5659743