code.co_names leaks a __doc__ entry

when it shouldn't.

I was able to repro this in cases where a nested function is defined, i.e:

def foo():
    def inner():
        pass

In RustPython:

>>>>> foo.__code__.co_names
('__doc__',)

while in Python:

>>> foo.__code__.co_names
()

and also when compiling a module from a source string:

stmts = """
import blah
 
def foo():
    pass
"""
code = compile(stmts, "<test>", "exec")

In RustPython:

>>>>> code.co_names
('blah', '__doc__', 'foo')

while in Python:

>>> code.co_names
('blah', 'foo')

not entirely sure how easy the fix might be.

@DimitrisJim

@DimitrisJim how difficult do you think this would be?

Ah, still not sure, sorry. There's some subtlety here since it only occurs with nested functions (maybe with anything nested that can take a docstring?).

You're of course welcome to look into it, especially if you'd like to see more of the compiler, and if stuck we'll figure something out :)

Sure! I'd love to get onto the compiler side. Do you have a recommendation for where I should look?

The reason why this happens is that cpython forwards .__doc__ to just the first element of co_consts, which is either the docstring or None if there isn't one. We manually set .__doc__ = ... on the function object, but should probably switch to the other way of doing it.

What Noa said, but in bytecode form, for a given function:

cmp = """
def foo():
    ''' doc '''
    pass
"""

What RustPython generates:

dis(cmp)
  4           0 LoadConst            (<code object foo at ??? file "<dis>", line 2>)
              1 LoadConst            ("foo")
              2 MakeFunction         ((empty))
              3 Duplicate
              4 LoadConst            (" doc ")
              5 Rotate2
              6 StoreAttr            (0, __doc__)
              7 StoreLocal           (1, foo)
              8 LoadConst            (None)
              9 ReturnValue

Disassembly of <code object foo at ??? file "<dis>", line 2>
  4           0 LoadConst            (None)
              1 ReturnValue

While for CPython:

dis(cmp)
  2           0 LOAD_CONST               0 (<code object foo at 0x7fb1db4e03a0, file "<dis>", line 2>)
              2 LOAD_CONST               1 ('foo')
              4 MAKE_FUNCTION            0
              6 STORE_NAME               0 (foo)
              8 LOAD_CONST               2 (None)
             10 RETURN_VALUE

Disassembly of <code object foo at 0x7fb1db4e03a0, file "<dis>", line 2>:
  4           0 LOAD_CONST               1 (None)
              2 RETURN_VALUE

notice the missing instructions for __doc__.

The relevant file that handles this is compiler/codegen/src/compile.rs. It needs some tweaking to stuff __doc__ in the consts if it exists instead of generating the load and store for the __doc__.

You could start for functions (compile_function) and then the process should be similar to the other cases.

Add regression test for RustPython#4505

Merge pull request #6417 from ShaharNaveh/regr-test-doc-leak

Navigation Menu

Repository navigation

`code.co_names` leaks a `doc` entry #4505

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

code.co_names leaks a __doc__ entry #4505

Description

Activity

evanrittenhouse commented on Mar 4, 2023

DimitrisJim commented on Mar 7, 2023

evanrittenhouse commented on Mar 8, 2023

coolreader18 commented on Mar 9, 2023

DimitrisJim commented on Mar 9, 2023

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

`code.co_names` leaks a `doc` entry #4505