Repository navigation
test_pty fails when using setsid() #82728
Description
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = None
closed_at = <Date 2019-12-09.11:00:12.673>
created_at = <Date 2019-10-21.11:38:39.966>
labels = ['tests', '3.9']
title = 'test_pty fails when using setsid()'
updated_at = <Date 2019-12-09.11:15:27.344>
user = 'https://github.com/vstinner'bugs.python.org fields:
activity = <Date 2019-12-09.11:15:27.344>
actor = 'miss-islington'
assignee = 'none'
closed = True
closed_date = <Date 2019-12-09.11:00:12.673>
closer = 'vstinner'
components = ['Tests']
creation = <Date 2019-10-21.11:38:39.966>
creator = 'vstinner'
dependencies = []
files = []
hgrepos = []
issue_num = 38547
keywords = ['patch']
message_count = 12.0
messages = ['355059', '357402', '357403', '357414', '357415', '357416', '357419', '358065', '358066', '358067', '358068', '358070']
nosy_count = 4.0
nosy_names = ['db3l', 'vstinner', 'pablogsal', 'miss-islington']
pr_nums = ['17519', '17520', '17521']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue38547'
versions = ['Python 3.9']
Activity
vstinner commentedon Oct 21, 2019
regrtest has been modified in bpo-38502 to use setsid() when using multiprocessing mode (-jN command line option).
Problem: David Bolen identified that test_pty started to fail on his bolen-ubuntu worker (Ubuntu 18.04.3) since my commit ecb035c.
https://buildbot.python.org/all/#/builders/141/builds/2679
0:19:05 load avg: 1.81 [234/419/1] test_pty crashed (Exit code -1) -- running: test_unicodedata (55.5 sec)
I can reproduce the issue locally:
---
== Tests result: FAILURE ==
1 test failed:
test_pty
Total duration: 383 ms
Tests result: FAILURE
---
It's surprising that there is no output!
I would prefer to keep process groups in regrtest, it's really helpful to be able to kill all processes spawned by a test worker process.
I'm not sure how/why PTY depends is incompatible with setsid().
pablogsal commentedon Nov 24, 2019
This also happens when running the test suite with high parallelism:
./python -m test -j 20
This fails with:
== Tests result: FAILURE ==
398 tests OK.
2 tests failed:
test_embed test_pty
pablogsal commentedon Nov 24, 2019
Indeed, almost all buildbots have to repeat the test_pty.
I think this needs to be fixed or process groups in regrtest should be limited or reverted.
vstinner commentedon Nov 24, 2019
What do you mean by "limited"?
Process groups really help to prevent to leak grandchild processes in multiprocessing tests, when tests are interrupted manually by CTRL+C or by a timeout (sadly, only when the timeout is handled by regrtest, not when it's handled by faulthandler).
See bpo-38502 for the rationale.
pablogsal commentedon Nov 24, 2019
I mean to deactivate it by default and make opt-in.
I love process groups and they are awesome. But having these test being re-run on every buildbot and failing on my machine when just running test with -j is very annoying.
vstinner commentedon Nov 24, 2019
Can't we fix test_pty?
db3l commentedon Nov 24, 2019
I think fixing the underlying pty issue should certainly be the goal, but the question is whether the process group change should remain active in the meantime, as its presence is causing a regression in the tests. I think such cases in the past are usually rolled back, right?
I was originally on the fence since process groups address a real problem, especially in interactive testing, while creating an arguably aesthetic issue for my case of the buildbots (a warning rather than failure).
But Pablo's point about a normal manual full test run failing (not a warning as with the buildbots) feels persuasive since that's probably as common as the issue being addressed by the change. Even if pre-existing, the pty failure is exposed by the process group change, so it might be best for the process group change to wait on fixing the pty issue.
I don't know how to weigh the relative impact though, e.g,. how many people are likely to run into each failure case. There's probably more people doing a normal test run than breaking out of such tests though. At the least, it's a worst impact than just the warnings on the buildbots.
Perhaps an intermediate fallback could be gating the process group change behind a regrtest option (opt-in) which could then preserve its benefits upon request, without negatively impacting the default test process, whether manual or on the buildbots.
At least until resource is available to resolve the pty issue.
vstinner commentedon Dec 9, 2019
I wrote PR 17519 which fix the bug. We just have to ignore SIGHUP signal.
vstinner commentedon Dec 9, 2019
New changeset a1838ec by Victor Stinner in branch 'master':
bpo-38547: Fix test_pty if the process is the session leader (GH-17519)
a1838ec
vstinner commentedon Dec 9, 2019
Ok, I merged my fix to master. The backport to 3.7 and 3.8 will follow quickly. I close the issue.
Sorry for the inconvenience ;-)
miss-islington commentedon Dec 9, 2019
New changeset b9f4b49 by Miss Islington (bot) in branch '3.7':
bpo-38547: Fix test_pty if the process is the session leader (GH-17519)
b9f4b49
miss-islington commentedon Dec 9, 2019
New changeset d08fd29 by Miss Islington (bot) in branch '3.8':
bpo-38547: Fix test_pty if the process is the session leader (GH-17519)
d08fd29