Repository navigation
test_capi fails if run with --forever argument #105273
Description
Activity
Seems to be introduced in #105100
cc @markshannonFixed by #105683
Reacted by Erlend E. Aasland and Kirill PodoprigoraI am testing heads/main: 6a80664 and the issue happens again.
./python -m test -R 3:3 -v test_capi -m test_long_loop -m test_counter_optimizer 0:00:00 load avg: 0.62 Run tests sequentially 0:00:00 load avg: 0.62 [1/1] test_capi beginning 6 repetitions 123456 test_counter_optimizer (test.test_capi.test_misc.TestOptimizerAPI.test_counter_optimizer) ... ok test_long_loop (test.test_capi.test_misc.TestOptimizerAPI.test_long_loop) Check that we aren't confused by EXTENDED_ARG ... ok ---------------------------------------------------------------------- Ran 2 tests in 0.001s OK .test_counter_optimizer (test.test_capi.test_misc.TestOptimizerAPI.test_counter_optimizer) ... FAIL test_long_loop (test.test_capi.test_misc.TestOptimizerAPI.test_long_loop) Check that we aren't confused by EXTENDED_ARG ... FAIL ====================================================================== FAIL: test_counter_optimizer (test.test_capi.test_misc.TestOptimizerAPI.test_counter_optimizer) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/xxx/cpython/Lib/test/test_capi/test_misc.py", line 2386, in test_counter_optimizer self.assertEqual(opt.get_count(), 1000) AssertionError: 0 != 1000 ====================================================================== FAIL: test_long_loop (test.test_capi.test_misc.TestOptimizerAPI.test_long_loop) Check that we aren't confused by EXTENDED_ARG ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/xxx/cpython/Lib/test/test_capi/test_misc.py", line 2408, in test_long_loop self.assertEqual(opt.get_count(), 10) AssertionError: 0 != 10 ---------------------------------------------------------------------- Ran 2 tests in 0.002s FAILED (failures=2) test test_capi failed test_capi failed (2 failures) == Tests result: FAILURE == 1 test failed: test_capi Total duration: 76 ms Tests result: FAILURE
Fixed by #105683
Unfortunately not.
Reacted by Alex WaygoodJust got hit by this (it makes the refleak buildbots fail in #105924).
I've got a hunch the problem is in deopt_code() (in codeobject.c), but I can't quite nail it. I put a printf() there when opcode == ENTER_EXECUTOR, and in the loop in the test which executes five times, it prints only four times?! Then the second time the whole test is re-run, it prints never at all (but the test fails after the first time through the loop). It appears the first iteration (out of five in the test --
for repeat in range(5)) doesn't cause any executor to be cleared when clear_executors() setsfunc.__code__ = func.__code__.replace(). But the first iteration clearly has an executor, because the assert passes (the first time the whole test is run).Not sure where this really points, but the logic in deopt_code() is subtle (it gets the opcode from
_Py_GetBaseOpcode(code, i), then patchesinstructions[i]). I get lost when I try to understand_Py_GetBaseOpcode().This was now fixed by @vstinner in #106171. His fix is to create a whole new function object (by using
exec). This is needed because the original code object is kept alive by the parent code object -- and the only way to clear a code object'sco_executorsarray is to cause the code object to be finalized.(I tried putting the assignment to
func.__code__in front of theyield, but it didn't seem to work.)Arguably we ought to have an explicit API for this, but that can wait.
Reacted by Kirill Podoprigora
Tried on current main:
Fails on test_capi/test_misc/TestOptimizerAPI.test_counter_optimizer