compiler package and SET_LINENO

@mwhudson

BPO	597919
Nosy	@mwhudson, @freddrake, @akuchling
Files	compiler-fixes.diff: first patch compiler-fixes-2.diff: second patch -- working this time! compiler-fixes-3.diff: third patch -- less C

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2002-12-31.18:19:06.000>
created_at = <Date 2002-08-20.19:51:04.000>
labels = ['library']
title = 'compiler package and SET_LINENO'
updated_at = <Date 2002-12-31.18:19:06.000>
user = 'https://github.com/freddrake'

bugs.python.org fields:

activity = <Date 2002-12-31.18:19:06.000>
actor = 'jhylton'
assignee = 'jhylton'
closed = True
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2002-08-20.19:51:04.000>
creator = 'fdrake'
dependencies = []
files = ['594', '595', '596']
hgrepos = []
issue_num = 597919
keywords = []
message_count = 16.0
messages = ['12065', '12066', '12067', '12068', '12069', '12070', '12071', '12072', '12073', '12074', '12075', '12076', '12077', '12078', '12079', '12080']
nosy_count = 5.0
nosy_names = ['mwh', 'jhylton', 'fdrake', 'akuchling', 'bnoon']
pr_nums = []
priority = 'high'
resolution = 'accepted'
stage = None
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue597919'
versions = ['Python 2.3']

The compiler package should not issue SET_LINENO
instructions any more. Running the Zope 2.7 test suite
produces this traceback:

Traceback (most recent call last):
  File "test.py", line 433, in ?
    process_args()
  File "test.py", line 397, in process_args
    bad = main(module_filter, test_filter)
  File "test.py", line 308, in main
    runner(files, test_filter, debug)
  File "test.py", line 275, in runner
    s = get_suite(file)
  File "test.py", line 230, in get_suite
    mod = package_import(modname)
  File "test.py", line 211, in package_import
    mod = __import__(modname)
  File
"/home/fdrake/projects/Zope/Zope2.7/lib/python/RestrictedPython/tests/testRestrictions.py",
line 64, in ?
    create_rmodule()
  File
"/home/fdrake/projects/Zope/Zope2.7/lib/python/RestrictedPython/tests/testRestrictions.py",
line 55, in create_rmodule
    code = compile_restricted(source, fn, 'exec')
  File
"/home/fdrake/projects/Zope/Zope2.7/lib/python/RestrictedPython/RCompile.py",
line 147, in compile_restricted
    gen.compile()
  File
"/usr/local/lib/python2.3/compiler/pycodegen.py", line
123, in compile
    self.code = gen.getCode()
  File
"/usr/local/lib/python2.3/compiler/pycodegen.py", line
252, in getCode
    return self.graph.getCode()
  File "/usr/local/lib/python2.3/compiler/pyassem.py",
line 378, in getCode
    self.convertArgs()
  File "/usr/local/lib/python2.3/compiler/pyassem.py",
line 484, in convertArgs
    self.insts[i] = opname, conv(self, oparg)
  File "/usr/local/lib/python2.3/compiler/pyassem.py",
line 520, in _convert_LOAD_CONST
    arg = arg.getCode()
  File
"/usr/local/lib/python2.3/compiler/pycodegen.py", line
252, in getCode
    return self.graph.getCode()
  File "/usr/local/lib/python2.3/compiler/pyassem.py",
line 380, in getCode
    self.makeByteCode()
  File "/usr/local/lib/python2.3/compiler/pyassem.py",
line 585, in makeByteCode   
lnotab.addCode(self.opnum[opname], lo, hi)
KeyError: SET_LINENO

Logged In: YES
user_id=6656

This is my fault, obviously.

I'm prepared to do some legwork to fix this problem, but don't really understand the code well enough to judge the best approach.

Logged In: YES
user_id=31392

A grep for SET_LINENO in the compiler package source will be
somewhat helpful. There are very few places that actually
emit the instructions. I thought they could just be
removed, but the lnotab is generated *from* the SET_LINENO
instructions. So we need to come up with a different way to
generate the lnotab.

Logged In: YES
user_id=6656

Believe it or not, I'd actually got that far myself <wink>.

If it'd been that easy, the changes would have been part of
my original SET_LINENO removal patch.

I guess the first question to answer is where does the
lnotab live during compilation? In the CodeGenerator? Then
logic has to move from PyFlowGraph.makeByteCode to
CodeGenerator.set_lineno.

Logged In: YES
user_id=6656

I'm about to go away for a week, so if you want my help
you'll probably have to wait until I get back.

Logged In: YES
user_id=6656

Hmm. I think I can fix the SET_LINENO breakage, but the
compiler package is shafted in various other ways, notably
by PEP-263.

Fixing this seems to require a bewildering array of patches
-- to symbol.py, to parsermodule, to token.py...

I also found a couple of bugs, which I'll fix. Some of
these should probably go into 2.2.2, too.

Logged In: YES
user_id=6656

Here's a patch.

Remarkably, it's just a one line patch to work around the
SET_LINENO breakage.

Other changes:

open the file for compiling in universal newlines mode.
always write the mtime to a .pyc little-endian-wise
(2.2.2 candidate)
it's YIELD_VALUE, not YIELD_STMT (2.2.2 candidate)
bodge the parsermodule to return the encoding along with the
encoding_decl non-terminal (this is pretty icky!)
let transformer cope with new 'testlist1' non-terminal
have transformer pick up the encoding declaration

Still to do is using the encoding declaration to decode
string literals. I think there should be an accessible from
Python function to do this -- I really don't want to have to
translate compile.c:parsestr into Python!

Logged In: YES
user_id=6656

Here's a patch that lets me compile the standard library and
test suite with the compiler package and then run the test
suite successfully.

It does... oh, lots and lots of stuff. Is anyone listening?

Logged In: YES
user_id=31392

I'm listening, but it's going in one ear and out the other.
I hope to have time for this later in the week.

Logged In: YES
user_id=6656

Ah good. Will wait for your comments.

Logged In: YES
user_id=298547

I just used the patch to fix problems with Quixote. Seems
to have done the trick.

Logged In: YES
user_id=11375

I'd like to add my vote for bumping up the priority of this patch; two people have run into this so far.

Logged In: YES
user_id=6656

Raising priority: something ABSOLUTELY HAS to be done about
this before 2.3a1.

Logged In: YES
user_id=6656

Here's the patch I posted a link to python-dev yesterday.

Assigning to Guido; he might actually do something about it :)

Logged In: YES
user_id=31392

Looks like a good patch. I'm running
Tools/compiler/regrtest.py just in case. I'll check it in
assuming that the tests pass.

Logged In: YES
user_id=31392

Committed for the 2.3a1 release.

Navigation Menu

Repository navigation

compiler package and SET_LINENO #37064

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

compiler package and SET_LINENO #37064

Description

Activity

freddrake commented on Aug 20, 2002

mwhudson commented on Aug 21, 2002

jhylton commented on Aug 22, 2002

mwhudson commented on Aug 23, 2002

mwhudson commented on Aug 31, 2002

mwhudson commented on Oct 3, 2002

mwhudson commented on Oct 3, 2002

mwhudson commented on Oct 9, 2002

jhylton commented on Oct 9, 2002

mwhudson commented on Oct 10, 2002

bnoon commented on Nov 1, 2002

akuchling commented on Nov 6, 2002

mwhudson commented on Nov 26, 2002

mwhudson commented on Dec 31, 2002

jhylton commented on Dec 31, 2002

jhylton commented on Dec 31, 2002

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions