Skip to content

Commit dd2cc4d

Browse files
youknowoneyouknowone
authored
Align compiler futures, annotations, and symtable with CPython (#8550)
* Align compiler futures and annotations with CPython Handle Barry parsing consistently across AST, type-comment, REPL, and WASM paths. Preserve deferred annotation source and scope metadata, and make _symtable conversion cached and linear-time. Assisted-by: Codex:gpt-5.4 * Fix f-string debug ranges and update Barry/annotate test expectations Compute the debug f-string source range from the raw leading and trailing text instead of the comment-stripped text, so a comment inside a multi-line replacement field no longer shifts the emitted LOAD_CONST location. Restrict the obsolete-operator location shift to `<>` by also requiring a `>` at the reported offset; without it any `ExpectedExpression` that followed a `<` (`2 <;`) was reported one column to the left. Add `CodeFlags::FUTURE_MASK` and use it in builtins, the shell, and the WASM VM instead of repeating the eight-flag union, and collapse the duplicated Barry diagnostic sequences into `BarrySource::diagnostic`. Update the `__annotate__` disassembly snapshot to LOAD_FAST_BORROW and replace the RustPython-only `barry_as_FLUFL` co_flags assertion in extra_tests with the CPython-matching one. Assisted-by: Claude Code:claude-opus-5 * Restore the annotated function in __annotate__ qualnames Signature annotation scopes are compiled in the scope enclosing the annotated function, so the function name has to be folded back into the qualname: `f.__annotate__`, `C.m.__annotate__`, `outer.<locals>.inner.__annotate__`. test_type_annotations test_annotate_qualname covers this. test_pyrepl test_future_barry_as_flufl now passes, so drop its expectedFailure marker. Assisted-by: Claude Code:claude-opus-5 * Pass safepoints in the attached-thread interpreter tests `main_and_subinterpreter_run_sections_overlap` parked its workers on a condvar while attached, and `busy_main_interpreter_does_not_block_subinterpreter` looped over protocol calls that never reach a safepoint. An attached thread that never reaches a safepoint cannot be suspended, so a concurrent process-wide stop-the-world (from the gc test running in parallel) never completes and the already-stopped sibling interpreter stays stopped; the second worker then waits in `wait_while_suspended` until the 30s deadline. Both loops now call `vm.check_signals()` each iteration, and the condvar wait uses `wait_timeout` so the lock is released between safepoints. Reproduced on Linux (4 CPUs): the two tests with `--test-threads=2` failed 2/40 before and 0/40 after; the full `rustpython-vm` lib binary failed 1/30 before and 0/30 after. Assisted-by: Claude Code:claude-opus-5 * Report the obsolete `<>` operator from one place `<>` was located twice: `BarrySource` recorded the token ranges it rewrote to `!=`, and `cpython_parse_diagnostic_override` separately peeked at the bytes around an `ExpectedExpression` location. Both produced "invalid syntax" spanning the whole operator. `prepare_barry_as_flufl_source` now records the operator ranges in the non-Barry case too -- by plain text search, since nothing is rewritten there -- so `BarrySource::invalid_legacy_operator` covers both, and the byte-peeking helper is gone. It also defers to an unclosed bracket earlier in the source, which the diagnostic chain it used to sit in did for it. A `<>` that starts a statement now spans both characters: the parser stops at the `<` rather than one character in, which the byte peek did not handle. Assisted-by: Claude Code:claude-opus-5 * Drop a py object's payload before its class `PyInner` declares `typ` before `payload`, so `drop_in_place` dropped the class first and `PyAtomicRef::drop` left it null. A `PyWeak` payload is still linked into the weakref list of the object it points at until its own `Drop` unlinks it under the stripe lock, which happens after that. `WeakRefList::add` walks the same list looking for a proxy to reuse and reads `node.class()` off every node it passes, so it could dereference the null class of a node on its way out. `PyInner::dealloc` now runs the two destructors in the other order at all three of its exit paths, with a compile-time check that no other field has one. Reproduced by the new test in about 0.1s, as the panic CI hit in the c-api `weakrefobject::tests::new_proxy` run: panicked at crates/vm/src/object/ext.rs:312: unsafe precondition(s) violated: hint::unreachable_unchecked must never be reached Assisted-by: Claude Code:claude-opus-5
1 parent 91a725f commit dd2cc4d

23 files changed

Lines changed: 1201 additions & 249 deletions

‎Lib/test/test_flufl.py‎

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,6 @@
44

55
class FLUFLTests(unittest.TestCase):
66

7-
@unittest.expectedFailure # TODO: RUSTPYTHON
87
def test_barry_as_bdfl(self):
98
code = "from __future__ import barry_as_FLUFL\n2 {0} 3"
109
compile(code.format('<>'), '<BDFL test>', 'exec',
@@ -35,7 +34,6 @@ def test_guido_as_bdfl(self):
3534
# parser reports the start of the token
3635
self.assertEqual(cm.exception.offset, 3)
3736

38-
@unittest.expectedFailure # TODO: RUSTPYTHON
3937
def test_barry_as_bdfl_look_ma_with_no_compiler_flags(self):
4038
# Check that the future import is handled by the parser
4139
# even if the compiler flags are not passed.

‎Lib/test/test_future_stmt/test_future.py‎

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -111,7 +111,6 @@ def test_future_import_not_on_top(self):
111111
"""
112112
self.assertSyntaxError(code, lineno=3)
113113

114-
@unittest.expectedFailure # TODO: RUSTPYTHON; AssertionError: SyntaxError not raised
115114
def test_future_import_with_extra_string(self):
116115
code = """
117116
'''Docstring'''
@@ -260,7 +259,6 @@ def _exec_future(self, code):
260259
)
261260
return scope
262261

263-
@unittest.expectedFailure # TODO: RUSTPYTHON; AssertionError: "t'{a + b}'" != "t'{a + b}'"
264262
def test_annotations(self):
265263
eq = self.assertAnnotationEqual
266264
eq('...')

‎Lib/test/test_pyrepl/test_interact.py‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -166,7 +166,6 @@ def g(x: int): ...
166166
self.assertFalse(result)
167167
self.assertEqual(f.getvalue(), "{'x': 'int'}\n")
168168

169-
@unittest.expectedFailure # TODO: RUSTPYTHON
170169
def test_future_barry_as_flufl(self):
171170
console = InteractiveColoredConsole()
172171
f = io.StringIO()

‎Lib/test/test_super.py‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -90,7 +90,6 @@ def nested():
9090

9191
self.assertEqual(E().f(), 'AE')
9292

93-
@unittest.expectedFailure # TODO: RUSTPYTHON
9493
def test_various___class___pathologies(self):
9594
# See issue #12370
9695
class X(A):

‎Lib/test/test_symtable.py‎

Lines changed: 0 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -299,7 +299,6 @@ def test_symbol_lookup(self):
299299

300300
self.assertRaises(KeyError, self.top.lookup, "not_here")
301301

302-
@unittest.expectedFailure # TODO: RUSTPYTHON
303302
def test_namespaces(self):
304303
self.assertTrue(self.top.lookup("Mine").is_namespace())
305304
self.assertTrue(self.Mine.lookup("a_method").is_namespace())
@@ -360,7 +359,6 @@ def test_name(self):
360359
self.assertEqual(self.spam.lookup("x").get_name(), "x")
361360
self.assertEqual(self.Mine.get_name(), "Mine")
362361

363-
@unittest.expectedFailure # TODO: RUSTPYTHON; AssertionError: Tuples differ: () != ('a_method',)
364362
def test_class_get_methods(self):
365363
deprecation_mess = (
366364
re.escape('symtable.Class.get_methods() is deprecated '
@@ -442,7 +440,6 @@ def check_body(body, expected_methods):
442440
check_body('\n'.join((gen, func)), ('genexpr',))
443441
check_body('\n'.join((func, gen)), ('genexpr',))
444442

445-
@unittest.expectedFailure # TODO: RUSTPYTHON; SyntaxError: name 'x' is parameter and global
446443
def test_filename_correct(self):
447444
### Bug tickler: SyntaxError file name correct whether error raised
448445
### while parsing or building symbol table.
@@ -474,7 +471,6 @@ def test_single(self):
474471
def test_exec(self):
475472
symbols = symtable.symtable("def f(x): return x", "?", "exec")
476473

477-
@unittest.expectedFailure # TODO: RUSTPYTHON; TypeError: Expected type 'str' but 'bytes' found.
478474
def test_bytes(self):
479475
top = symtable.symtable(TEST_CODE.encode('utf8'), "?", "exec")
480476
self.assertIsNotNone(find_block(top, "Mine"))
@@ -563,7 +559,6 @@ def test_loopvar_in_only_one_scope(self):
563559
class CommandLineTest(unittest.TestCase):
564560
maxDiff = None
565561

566-
@unittest.expectedFailure # TODO: RUSTPYTHON; TypeError: Expected type 'str' but 'bytes' found.
567562
def test_file(self):
568563
filename = os_helper.TESTFN
569564
self.addCleanup(os_helper.unlink, filename)

0 commit comments

Comments
 (0)