Skip to content

gh-105481: Generate the opcode lists in dis from data extracted from bytecodes.c #106758

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
iritkatriel merged 30 commits into from
Jul 18, 2023
Merged

gh-105481: Generate the opcode lists in dis from data extracted from bytecodes.c #106758

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
36a7d56
gh-105481: add haslocal to _opcode.py. Generate most oplists in opcod…
iritkatriel Jul 14, 2023
75019ca
added HAS_FREE flag. Removed HAS_FREE from the HAS_LOCAL list
iritkatriel Jul 14, 2023
494680a
add hasjump, soft deprecate hasjrel and hasjabs. Fix build bug
iritkatriel Jul 15, 2023
24786fb
generate hasexc from the C macros instead of defining it again
iritkatriel Jul 15, 2023
563b431
typo
iritkatriel Jul 15, 2023
227756f
Merge remote-tracking branch 'upstream/main' into opcode_py
iritkatriel Jul 17, 2023
1b13b96
ascii quotes
iritkatriel Jul 17, 2023
0f5ae32
make clinic
iritkatriel Jul 17, 2023
289fad1
remove oplists
iritkatriel Jul 17, 2023
598ba2a
import less from opcode in build script. _opcode.has_* don't exist pr…
iritkatriel Jul 17, 2023
d2d355e
hascompare in old versions as well
iritkatriel Jul 17, 2023
27eb2cc
remove redundant init
iritkatriel Jul 17, 2023
998024a
📜🤖 Added by blurb_it.
blurb-it[bot] Jul 17, 2023
ffa4ee3
Merge branch 'main' into opcode_py
iritkatriel Jul 17, 2023
ce76a60
remove unnecessary comment
iritkatriel Jul 17, 2023
4194a12
make the tests more precise
iritkatriel Jul 17, 2023
6b8c46b
sort the lists
iritkatriel Jul 17, 2023
e15f7d5
address some of the code review comments
iritkatriel Jul 18, 2023
21fe0a4
remove hard-coded lists from the tests
iritkatriel Jul 18, 2023
75795db
remove opcode metadata file from generated files list so diff are vis…
iritkatriel Jul 18, 2023
1a0a2b2
import _opcode directly into dis, rather than via opcode
iritkatriel Jul 18, 2023
f530469
add stack effect to dis.__all__
iritkatriel Jul 18, 2023
71ffe34
Merge branch 'main' into opcode_py
iritkatriel Jul 18, 2023
d525c53
import _specializations, _specialized_instructions directly from _opc…
iritkatriel Jul 18, 2023
5e1c4a4
update Tools/scripts/summarize_stats.py to use _opcode_metadata inste…
iritkatriel Jul 18, 2023
6077263
Revert "update Tools/scripts/summarize_stats.py to use _opcode_metada…
iritkatriel Jul 18, 2023
98ba11f
Revert "import _specializations, _specialized_instructions directly f…
iritkatriel Jul 18, 2023
cc8e5e1
Revert "add stack effect to dis.__all__"
iritkatriel Jul 18, 2023
31dbfec
Revert "import _opcode directly into dis, rather than via opcode"
iritkatriel Jul 18, 2023
1abf6ca
Merge branch 'main' into opcode_py
iritkatriel Jul 18, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Revert "import _opcode directly into dis, rather than via opcode" 
This reverts commit 1a0a2b2.
  • Loading branch information
@iritkatriel
iritkatriel committed Jul 18, 2023
commit 31dbfeced967ca8c171d170eab0e211cbf7df68d
18 changes: 1 addition & 17 deletions Lib/dis.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,6 @@
import types
import collections
import io
import _opcode
from _opcode import stack_effect

from opcode import *
from opcode import (
Expand All @@ -19,28 +17,14 @@
_specialized_instructions,
)

__all__ = ["hasarg", "hasconst", "hasname", "hasjump", "hasjrel",
"hasjabs", "hasfree", "haslocal", "hasexc", "hascompare",
"code_info", "dis", "disassemble", "distb", "disco",
__all__ = ["code_info", "dis", "disassemble", "distb", "disco",
"findlinestarts", "findlabels", "show_code",
"get_instructions", "Instruction", "Bytecode"] + _opcodes_all
del _opcodes_all

_have_code = (types.MethodType, types.FunctionType, types.CodeType,
classmethod, staticmethod, type)

# These lists are documented as part of the dis module's API
hasarg = [op for op in opmap.values() if _opcode.has_arg(op)]
hasconst = [op for op in opmap.values() if _opcode.has_const(op)]
hasname = [op for op in opmap.values() if _opcode.has_name(op)]
hasjump = [op for op in opmap.values() if _opcode.has_jump(op)]
hasjrel = hasjump # for backward compatibility
hasjabs = []
hasfree = [op for op in opmap.values() if _opcode.has_free(op)]
haslocal = [op for op in opmap.values() if _opcode.has_local(op)]
hasexc = [op for op in opmap.values() if _opcode.has_exc(op)]
hascompare = [opmap["COMPARE_OP"]]

CONVERT_VALUE = opmap['CONVERT_VALUE']

SET_FUNCTION_ATTRIBUTE = opmap['SET_FUNCTION_ATTRIBUTE']
Expand Down
25 changes: 24 additions & 1 deletion Lib/opcode.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,12 @@
"""


__all__ = ["cmp_op", "opname", "opmap", "HAVE_ARGUMENT", "EXTENDED_ARG"]
# Note that __all__ is further extended below
__all__ = ["cmp_op", "opname", "opmap", "stack_effect", "hascompare",
"HAVE_ARGUMENT", "EXTENDED_ARG"]
Comment thread
iritkatriel marked this conversation as resolved.

import _opcode
from _opcode import stack_effect

import sys
# The build uses older versions of Python which do not have _opcode_metadata
Expand Down Expand Up @@ -236,6 +241,24 @@ def pseudo_op(name, op, real_ops):
for op, i in opmap.items():
opname[i] = op

# The build uses older versions of Python which do not have _opcode.has_* functions

@gvanrossum gvanrossum Jul 17, 2023

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is quite mysterious. IIRC Brandt (?) wrote some hack that reads opcode.py and then execs it? But I can't find it.

@iritkatriel iritkatriel Jul 18, 2023

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, in Tools/build/generate_opcode_h.py. But when it execs opcode.py that tries to import _opcode and call its has_arg etc to construct the oplists (which are not used during the build).

Ideally we should just get rid of generate_opcode_h.py if we can.

@iritkatriel iritkatriel Jul 18, 2023

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we don't need to import _opcode into opcode at all. We can just import it directly into dis (which is not used by any build scripts). I'll try that.

if sys.version_info[:2] >= (3, 13):
# These lists are documented as part of the dis module's API
hasarg = [op for op in opmap.values() if _opcode.has_arg(op)]
hasconst = [op for op in opmap.values() if _opcode.has_const(op)]
hasname = [op for op in opmap.values() if _opcode.has_name(op)]
hasjump = [op for op in opmap.values() if _opcode.has_jump(op)]
hasjrel = hasjump # for backward compatibility
hasjabs = []
Comment thread
iritkatriel marked this conversation as resolved.
hasfree = [op for op in opmap.values() if _opcode.has_free(op)]
haslocal = [op for op in opmap.values() if _opcode.has_local(op)]
hasexc = [op for op in opmap.values() if _opcode.has_exc(op)]

__all__.extend(["hasarg", "hasconst", "hasname", "hasjump", "hasjrel",
"hasjabs", "hasfree", "haslocal", "hasexc"])
Comment thread
iritkatriel marked this conversation as resolved.

hascompare = [opmap["COMPARE_OP"]]

_nb_ops = [
("NB_ADD", "+"),
("NB_AND", "&"),
Expand Down