Repository navigation
test__xxsubinterpreters crashed on x86 Gentoo Refleaks 3.x #77796
Description
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = 'https://github.com/ericsnowcurrently'
closed_at = <Date 2018-06-14.14:49:49.798>
created_at = <Date 2018-05-23.12:25:16.282>
labels = ['interpreter-core', '3.8', 'tests']
title = 'test__xxsubinterpreters crashed on x86 Gentoo Refleaks 3.x'
updated_at = <Date 2018-06-14.15:59:56.384>
user = 'https://github.com/vstinner'bugs.python.org fields:
activity = <Date 2018-06-14.15:59:56.384>
actor = 'vstinner'
assignee = 'eric.snow'
closed = True
closed_date = <Date 2018-06-14.14:49:49.798>
closer = 'eric.snow'
components = ['Interpreter Core', 'Tests']
creation = <Date 2018-05-23.12:25:16.282>
creator = 'vstinner'
dependencies = []
files = []
hgrepos = []
issue_num = 33615
keywords = ['patch']
message_count = 25.0
messages = ['317394', '317415', '317462', '317477', '317637', '317641', '317645', '317646', '317647', '317961', '318038', '318094', '318332', '318346', '318349', '318478', '318653', '318660', '318692', '318995', '318996', '319067', '319466', '319470', '319526']
nosy_count = 4.0
nosy_names = ['vstinner', 'ned.deily', 'eric.snow', 'pablogsal']
pr_nums = ['7251', '7288', '7503', '7552']
priority = None
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue33615'
versions = ['Python 3.8']
Activity
vstinner commentedon May 23, 2018
http://buildbot.python.org/all/#/builders/1/builds/232
(...)
3:28:16 load avg: 3.67 [401/416/3] test__xxsubinterpreters crashed (Exit code -6) -- running: test_asyncio (4631 sec)
python: Modules/gcmodule.c:277: visit_decref: Assertion `_PyGCHead_REFS(gc) != 0' failed.
Fatal Python error: Aborted
Current thread 0xb748a700 (most recent call first):
File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/unittest/case.py", line 615 in run
File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/unittest/case.py", line 663 in __call__
File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/unittest/suite.py", line 122 in run
File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/unittest/suite.py", line 84 in __call__
File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/unittest/suite.py", line 122 in run
File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/unittest/suite.py", line 84 in __call__
File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/unittest/suite.py", line 122 in run
File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/unittest/suite.py", line 84 in __call__
File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/test/support/init.py", line 1781 in run
File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/test/support/init.py", line 1882 in _run_suite
File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/test/support/init.py", line 1972 in run_unittest
File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/test/libregrtest/runtest.py", line 175 in test_runner
File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/test/libregrtest/runtest.py", line 176 in runtest_inner
File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/test/libregrtest/runtest.py", line 140 in runtest
File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/test/libregrtest/runtest_mp.py", line 67 in run_tests_slave
File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/test/libregrtest/main.py", line 517 in _main
File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/test/libregrtest/main.py", line 510 in main
File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/test/libregrtest/main.py", line 585 in main
File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/test/regrtest.py", line 46 in _main
File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/test/regrtest.py", line 50 in <module>
File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/runpy.py", line 85 in _run_code
File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/runpy.py", line 193 in _run_module_as_main
(...)
Re-running test 'test__xxsubinterpreters' in verbose mode
(...)
test_recv_not_found (test.test__xxsubinterpreters.ChannelTests) ... ok
test_run_string_arg_resolved (test.test__xxsubinterpreters.ChannelTests) ... python: Modules/gcmodule.c:277: visit_decref: Assertion `_PyGCHead_REFS(gc) != 0' failed.
Fatal Python error: Aborted
Current thread 0xb74b9700 (most recent call first):
File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/unittest/case.py", line 615 in run
File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/unittest/case.py", line 663 in __call__
File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/unittest/suite.py", line 122 in run
File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/unittest/suite.py", line 84 in __call__
File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/unittest/suite.py", line 122 in run
File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/unittest/suite.py", line 84 in __call__
File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/unittest/suite.py", line 122 in run
File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/unittest/suite.py", line 84 in __call__
File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/unittest/runner.py", line 176 in run
File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/test/support/init.py", line 1882 in _run_suite
File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/test/support/init.py", line 1972 in run_unittest
File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/test/libregrtest/runtest.py", line 175 in test_runner
File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/test/libregrtest/runtest.py", line 176 in runtest_inner
File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/test/libregrtest/runtest.py", line 140 in runtest
File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/test/libregrtest/main.py", line 291 in rerun_failed_tests
File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/test/libregrtest/main.py", line 540 in _main
File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/test/libregrtest/main.py", line 510 in main
File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/test/libregrtest/main.py", line 585 in main
File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/test/main.py", line 2 in <module>
File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/runpy.py", line 85 in _run_code
File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/runpy.py", line 193 in _run_module_as_main
ericsnowcurrently commentedon May 23, 2018
I'll take a look. Thanks!
ericsnowcurrently commentedon May 23, 2018
FTR, this started happening after the following commit:
commit 6d2cd90
Author: Eric Snow <ericsnowcurrently@gmail.com>
Date: Wed May 16 15:04:57 2018 -0400
ericsnowcurrently commentedon May 23, 2018
There are a couple of other buildbots with the same failure:
ARMv7 Ubuntu 3.x: http://buildbot.python.org/all/#/builders/106/builds/1066
PPC64 AIX 3.x: http://buildbot.python.org/all/#/builders/10/builds/1005
vstinner commentedon May 24, 2018
It would be nice to fix this bug before Python 3.7.0 final: either skip the test, or fix it.
Since the functions are still private, skipping a single test (until it's fixed) should be fine.
ned-deily commentedon May 24, 2018
Victor:
These tests (and failures) are only on master / 3.x, not 3.7, right? If so, they have no bearing on 3.7.0.
ericsnowcurrently commentedon May 24, 2018
Correct. These failures are only on master.
vstinner commentedon May 24, 2018
Oh, I didn't know that it was a 3.8-only issue.
ericsnowcurrently commentedon May 24, 2018
no worries :)
3 remaining items
ericsnowcurrently commentedon May 31, 2018
New changeset 110bc01 by Eric Snow in branch 'master':
bpo-33615: Temporarily disable a test that is triggering crashes on a few buildbots. (gh-7288)
110bc01
vstinner commentedon May 31, 2018
Now the test runs but doesn't crash anymore: bpo-33724.
ericsnowcurrently commentedon May 31, 2018
FYI, I plan on closing this issue only *after* I've re-enabled the crashing test and it passes. :)
ericsnowcurrently commentedon Jun 2, 2018
New changeset 6379913 by Eric Snow in branch 'master':
bpo-33615: Re-enable a subinterpreter test. (gh-7251)
6379913
ericsnowcurrently commentedon Jun 4, 2018
This appears to be recurring on the "x86 Gentoo Refleaks 3.x" builder still. I was thrown off by the success of the first run after I landed my fix:
http://buildbot.python.org/all/#/builders/1/builds/241
FYI, the other buildbots having this issue before (e.g. "ARMv7 Ubuntu 3.x") are still passing.
ericsnowcurrently commentedon Jun 4, 2018
FTR, bpo-33724 is related
pablogsal commentedon Jun 4, 2018
Same error in AMD64 Windows10 3.x:
http://buildbot.python.org/all/#/builders/3/builds/941
vstinner commentedon Jun 8, 2018
New changeset c4f3cb7 by Victor Stinner in branch 'master':
bpo-33615: Skip test__xxsubinterpreters (GH-7503)
c4f3cb7
vstinner commentedon Jun 8, 2018
test__xxsubinterpreters prevents to get results from Gentoo Refleaks 3.x and Windows Refleaks 3.x, it also broke multiple CIs and it introduced random failures. For all these reasons, I skipped the test. See the general policy for CIs:
https://mail.python.org/pipermail/python-dev/2018-May/153753.html
Eric: if you need CIs to check if a change fix test__xxsubinterpreters, they are ways to trigger custom builds, but I don't recall how to do that :-D You should be able to do that on buildbots at least, the devguide explains how to do it, or ask maybe Zachary Ware. Until the crash is fixed, I would prefer to leave the test skipped.
ericsnowcurrently commentedon Jun 8, 2018
Yeah, I did a custom build the other day. Sorry about the delay in disabling the test again and thanks for getting it done.
ericsnowcurrently commentedon Jun 13, 2018
New changeset ab4a198 by Eric Snow in branch 'master':
bpo-33615: Re-enable subinterpreter tests. (bpo-7552)
ab4a198
ericsnowcurrently commentedon Jun 13, 2018
I've re-enabled the subinterpreter tests, but left the one problem test (ChannelTests.test_run_string_arg_resolved) disabled. I also changed all uses of %lld to use PRId64 instead. (Thanks, Victor, for the suggestion.)
The buildbots look good. I'll keep an eye on "x86 Gentoo Refleaks 3.x" for the next time it runs (starts every 24 hours; 10 hours from now). When that passes I'll close this issue.
vstinner commentedon Jun 14, 2018
It's nice to see this issue fixed :-)