member field + lambda results unexpected UnboundLocalError

Code:

class X:
    v = 10
    f = lambda x=v: x

Expected:
silence

Actual:

Traceback (most recent call last):
  File "test_lambda_scope.py", line 3, in <module>
    f = lambda x=v: x
  File "test_lambda_scope.py", line 3, in X
    f = lambda x=v: x
UnboundLocalError: local variable 'v' referenced before assignment

Since I am interested in how the RustPython compiler works, I want to try this! Is there any starting point in the codebase I could start poking around that might be relevant to this issue??

To be honest, I am not understand this problem very well.

The first step will be checking if it is a bytecode problem or not. dis module will be helpful.
The second step depends on the result of first step, but I would dig around LoadClassDeref generation and execution.

Thanks! I will start from the points you've mentioned 👍

@jirheee

@jirheee since it's been a while, do you need any help?

I'm looking into this issue using dis module to the example code.

RustPython's bytecodes of the example code are as follows.

  3           0 LoadGlobal           (0, __name__)
              1 StoreLocal           (1, __module__)
              2 LoadConst            ("X")
              3 StoreLocal           (2, __qualname__)
              4 LoadConst            (None)
              5 StoreLocal           (3, __doc__)
  4           6 LoadConst            (999)
              7 StoreLocal           (4, v)
  5           8 LoadFast             (0, v)
              9 BuildTuple           (1, false)
             10 LoadConst            (<code object <lambda> at ??? file "a.py", line 5>)
             11 LoadConst            ("<lambda>")
             12 MakeFunction         (DEFAULTS)
             13 StoreLocal           (5, w)
             14 LoadConst            (None)
             15 ReturnValue

But cpython's bytecodes are as follow.

  3           0 LOAD_NAME                0 (__name__)
              2 STORE_NAME               1 (__module__)
              4 LOAD_CONST               0 ('X')
              6 STORE_NAME               2 (__qualname__)
  4           8 LOAD_CONST               1 (999)
             10 STORE_NAME               3 (v)
  5          12 LOAD_NAME                3 (v)
             14 BUILD_TUPLE              1
             16 LOAD_CONST               2 (<code object <lambda> at 0x100c49bb0, file "a.py", line 5>)
             18 LOAD_CONST               3 ('X.<lambda>')
             20 MAKE_FUNCTION            1 (defaults)
             22 STORE_NAME               4 (w)
             24 LOAD_CONST               4 (None)
             26 RETURN_VALUE

I don't know about these bytecodes very much, but it seems the UnboundLocalError occurs because of the below bytecodes.

  4           6 LoadConst            (999)
              7 StoreLocal           (4, v)
  5           8 LoadFast             (0, v)

The v variable is stored through StoreLocal, but loaded through LoadFast. (and also the index mismatch too.)
While in cpython, the v variable is stored and loaded through LOAD_NAME, STORE_NAME.

I think it makes the LOAD_FAST fail by UnboundLocalError.

Ref: Maybe the below function might be related
https://github.com/RustPython/RustPython/blob/main/compiler/codegen/src/compile.rs#L461-L534

In the example code, compile_name with v is called twice when v is stored, and loaded.

The problem here is when v is stored, it is not function scope, so its op_typ is resolved as Local type,
but when the v is accessed, because it is in function scope, its op_typ is resolved as a Fast type.

(Related: https://github.com/RustPython/RustPython/blob/main/compiler/codegen/src/compile.rs#L479)

I think maybe using LoadNameAny like using STORE_NAME and LOAD_NAME in cpython might be a method to fix this.

For example, below code does not occur this UnboundLocalError

class X:
    v = 999
    def w(x = v):
        x

The above code's bytecodes are as follows and use LoadNameAny

  3           0 LoadGlobal           (0, __name__)
              1 StoreLocal           (1, __module__)
              2 LoadConst            ("X")
              3 StoreLocal           (2, __qualname__)
              4 LoadConst            (None)
              5 StoreLocal           (3, __doc__)

  4           6 LoadConst            (999)
              7 StoreLocal           (4, v)

  5           8 LoadNameAny          (4, v)
              9 BuildTuple           (1, false)

  6          10 LoadConst            (<code object w at ??? file "a.py", line 5>)
             11 LoadConst            ("X.w")
             12 MakeFunction         (DEFAULTS)
             13 Duplicate
             14 LoadConst            (None)
             15 Rotate2
             16 StoreAttr            (__doc__)
             17 StoreLocal           (5, w)
             18 LoadConst            (None)
             19 ReturnValue

So, I think maybe the problem here is the default argument v's scope is evaluated as in the function's scope in the only lambda expression, which is not correct.

>>>>> f = lambda x=v:x
name: v
self.ctx.in_func(): true

>>>>> def f(x=v): x
name: v
self.ctx.in_func(): false

Fix RustPython#3934

Navigation Menu

Repository navigation

member field + lambda results unexpected UnboundLocalError #3934

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

member field + lambda results unexpected UnboundLocalError #3934

Description

Activity

jirheee commented on Jul 31, 2022

youknowone commented on Aug 1, 2022

jirheee commented on Aug 1, 2022

youknowone commented on Sep 21, 2022

jopemachine commented on Oct 15, 2022

jopemachine commented on Oct 15, 2022

jopemachine commented on Oct 16, 2022

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions