Repository navigation
regrtest: Add --fail-env-changed option #74949
Description
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = None
closed_at = <Date 2017-06-27.21:37:48.617>
created_at = <Date 2017-06-26.12:17:09.833>
labels = ['3.7', 'type-feature', 'tests']
title = 'regrtest: Add --fail-env-changed option'
updated_at = <Date 2017-07-05.13:44:55.955>
user = 'https://github.com/vstinner'bugs.python.org fields:
activity = <Date 2017-07-05.13:44:55.955>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2017-06-27.21:37:48.617>
closer = 'vstinner'
components = ['Tests']
creation = <Date 2017-06-26.12:17:09.833>
creator = 'vstinner'
dependencies = []
files = ['46976']
hgrepos = []
issue_num = 30764
keywords = []
message_count = 21.0
messages = ['296883', '296885', '296902', '296908', '296911', '296913', '296958', '296960', '296965', '296966', '296972', '297028', '297036', '297043', '297046', '297075', '297549', '297558', '297567', '297577', '297744']
nosy_count = 1.0
nosy_names = ['vstinner']
pr_nums = ['2402', '2405', '2410', '2411', '2412', '2423', '2441', '2442', '2444', '2521', '2536', '2539', '2540', '2541']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue30764'
versions = ['Python 2.7', 'Python 3.5', 'Python 3.6', 'Python 3.7']
Activity
vstinner commentedon Jun 26, 2017
The be able to debug bpo-30280 using my bisection script:
https://github.com/haypo/misc/blob/205fea00d509ddd1cffa30c5c6f2da012f3ec54e/python/bisect_test.py
I would like to be able to mark tests which alter the environment as failed, using an explicit option.
It would be nice to be able to enable this option by default, or even remove the option and just make the behaviour as the default, but it would require to fix some issues first (like bpo-30280).
vstinner commentedon Jun 26, 2017
I implemented the feature:
#2402
My PR changes also the exit code from 1 to 2 (tests failed) or 3 (interrupted), to be able to distinguish how regrtest failed. It should also help to bisect failing tests.
vstinner commentedon Jun 26, 2017
I updated my PR to use the exit code 130 when regrtest is interrupted. Serhiy Storchaka found a reference:
"On Linux scripts terminated with Ctrl-C return exit code 130 (http://tldp.org/LDP/abs/html/exitcodes.html)."
vstinner commentedon Jun 26, 2017
New changeset cdee3f1 by Victor Stinner in branch 'master':
bpo-30764: test_subprocess uses SuppressCrashReport (bpo-2405)
cdee3f1
vstinner commentedon Jun 26, 2017
New changeset 849b062 by Victor Stinner in branch '3.5':
bpo-30764: test_subprocess uses SuppressCrashReport (bpo-2405) (bpo-2411)
849b062
vstinner commentedon Jun 26, 2017
New changeset 63f54c6 by Victor Stinner in branch 'master':
bpo-30764: regrtest: add --fail-env-changed option (bpo-2402)
63f54c6
vstinner commentedon Jun 26, 2017
New changeset 9ad50d9 by Victor Stinner in branch '3.6':
bpo-30764: test_subprocess uses SuppressCrashReport (bpo-2405) (bpo-2410)
9ad50d9
vstinner commentedon Jun 26, 2017
New changeset 2097b9e by Victor Stinner in branch '2.7':
[2.7] bpo-30764: test_subprocess uses SuppressCrashReport (bpo-2405) (bpo-2412)
2097b9e
vstinner commentedon Jun 26, 2017
I tried to implement test.support.SuppressCrashReport on Windows using only ctypes for Python 2.7, but I failed: see attached suppress_crash.py. So I created #2423 which adds new functions and constants to the _testcapi module. IMHO it's more reliabe to do that, but it adds more C code.
vstinner commentedon Jun 26, 2017
Test failure on x86-64 Sierra 2.7 buildbot:
http://buildbot.python.org/all/builders/x86-64%20Sierra%202.7/builds/111/steps/test/logs/stdio
skipped 'mswindows only'
======================================================================
FAIL: test_child_terminated_in_stopped_state (test.test_subprocess.POSIXProcessTestCase)
Test wait() behavior when waitpid returns WIFSTOPPED; bpo-29335.
----------------------------------------------------------------------
vstinner commentedon Jun 26, 2017
Oops, I used the wrong bpo number for the test_subprocess fixes: the right bpo is bpo-30448!
vstinner commentedon Jun 27, 2017
New changeset 95a91df by Victor Stinner in branch '2.7':
bpo-30764: Windows support.SuppressCrashReport (bpo-2423)
95a91df
vstinner commentedon Jun 27, 2017
New changeset 35d2ca2 by Victor Stinner in branch '3.6':
[3.6] bpo-30523, bpo-30764, bpo-30776: Sync regrtest from master (bpo-2441)
35d2ca2
vstinner commentedon Jun 27, 2017
New changeset de1850b by Victor Stinner in branch '3.5':
[3.5] bpo-30523, bpo-30764, bpo-30776: Sync regrtest from master (bpo-2442)
de1850b
vstinner commentedon Jun 27, 2017
New changeset fea98bf by Victor Stinner in branch '2.7':
[2.7] bpo-30523, bpo-30764, bpo-30776: Sync regrtest from master (bpo-2444)
fea98bf
vstinner commentedon Jun 27, 2017
The initial issue, the "regrtest: Add --fail-env-changed option" feature, has been implemented in 2.7, 3.5; 3.6 and master branches. So I close this issue.
Again, this issue got hijacked by the test_subprocess change, bpo-30448, because I wrote the wrong bpo number in my commits ;-)
vstinner commentedon Jul 3, 2017
New changeset 5e87592 by Victor Stinner in branch 'master':
bpo-30764: Fix regrtest --fail-env-changed --forever (bpo-2536)
5e87592
vstinner commentedon Jul 3, 2017
New changeset 4132adb by Victor Stinner in branch '3.6':
bpo-30764: Fix regrtest --fail-env-changed --forever (bpo-2536) (bpo-2539)
4132adb
vstinner commentedon Jul 3, 2017
New changeset 0b12107 by Victor Stinner in branch '3.5':
[3.5] bpo-29512, bpo-30764, bpo-30776: Backport regrtest enhancements from 3.6 to 3.5 (bpo-2540)
0b12107
vstinner commentedon Jul 3, 2017
New changeset d7955b8 by Victor Stinner in branch '2.7':
[2.7] bpo-29512, bpo-30764: Backport regrtest enhancements from 3.5 to 2.7 (bpo-2541)
d7955b8
vstinner commentedon Jul 5, 2017
New changeset 8207c17 by Victor Stinner in branch 'master':
Revert "bpo-30822: Fix testing of datetime module." (bpo-2588)
8207c17