Repository navigation
test_subprocess: test_child_terminated_in_stopped_state() leaks a zombie process #75356
Description
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = None
closed_at = <Date 2017-08-11.12:38:59.290>
created_at = <Date 2017-08-10.09:20:53.621>
labels = ['3.7', 'tests']
title = 'test_subprocess: test_child_terminated_in_stopped_state() leaks a zombie process'
updated_at = <Date 2017-08-11.12:38:59.289>
user = 'https://github.com/vstinner'bugs.python.org fields:
activity = <Date 2017-08-11.12:38:59.289>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2017-08-11.12:38:59.290>
closer = 'vstinner'
components = ['Tests']
creation = <Date 2017-08-10.09:20:53.621>
creator = 'vstinner'
dependencies = []
files = []
hgrepos = []
issue_num = 31173
keywords = []
message_count = 7.0
messages = ['300066', '300068', '300070', '300071', '300148', '300149', '300161']
nosy_count = 2.0
nosy_names = ['gregory.p.smith', 'vstinner']
pr_nums = ['3055', '3070', '3071']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue31173'
versions = ['Python 2.7', 'Python 3.6', 'Python 3.7']
Activity
vstinner commentedon Aug 10, 2017
The test_child_terminated_in_stopped_state() test creates a child process which calls ptrace(PTRACE_ME, 0, 0) and then crashs using SIGSEGV. The problem is that even if we read the exit status using os.waitpid() through subprocess, the process remains alive in the "t (tracing stop)" state.
I would prefer to not use ptrace() is an unit test since this API is very low-level and it's hard to use it correctly.
I suggest to either remove the functional test, or to rewrite it as an unit test using mocks to test bpo-29335 without ptrace().
haypo@selma$ ./python -m test -m test_child_terminated_in_stopped_state -F test_subprocess
Run tests sequentially
0:00:00 load avg: 0.95 [ 1] test_subprocess
0:00:00 load avg: 0.95 [ 2] test_subprocess
0:00:01 load avg: 0.96 [ 3] test_subprocess
0:00:01 load avg: 0.96 [ 4] test_subprocess
0:00:02 load avg: 0.96 [ 5] test_subprocess
0:00:03 load avg: 0.96 [ 6] test_subprocess
0:00:03 load avg: 0.96 [ 7] test_subprocess
0:00:04 load avg: 0.96 [ 8] test_subprocess
0:00:05 load avg: 0.96 [ 9] test_subprocess
0:00:05 load avg: 0.96 [ 10] test_subprocess
^Z
[1]+ Stoppé ./python -m test -m test_child_terminated_in_stopped_state -F test_subprocess
haypo@selma$ ps
PID TTY TIME CMD
30359 pts/0 00:00:00 bash
31882 pts/0 00:00:00 python
31885 pts/0 00:00:00 python
31888 pts/0 00:00:00 python
31892 pts/0 00:00:00 python
31895 pts/0 00:00:00 python
31898 pts/0 00:00:00 python
31901 pts/0 00:00:00 python
31904 pts/0 00:00:00 python
31907 pts/0 00:00:00 python
31910 pts/0 00:00:00 python
31912 pts/0 00:00:00 python
31920 pts/0 00:00:00 ps
haypo@selma$ grep Stat /proc/31885/status
State: t (tracing stop)
vstinner commentedon Aug 10, 2017
#3055 removes the functional test and replaces it with an unit test which mocks os.waitpid() using a new _testcapi.W_STOPCODE() function to test the WIFSTOPPED() path.
The functional test created a core dump, but it's now fixed using SuppressCrashReport. It leaks a zombie process in a special state, the process is traced and cannot be killed. I tried to wait for the process a second time, but it's not enough to "close" it. I guess that we would have to write a little debugger to attach the process in the parent process. IMHO it's overcomplicated just to check that subprocess calls WIFSTOPPED().
vstinner commentedon Aug 10, 2017
I chose to only add W_STOPCODE() to _testcapi rather than the os module, because I don't want to have to document this function. I don't think that anyone needs such function, usually we only need to consume process statuses, not to produce them. The only use case is to write an unit test.
This issue is part of bpo-31160 which ensures that unit tests don't leak child processes. This issue is part of my large project of reducing the fail rate on CIs (Travis CI, AppVeyor, buildbots):
https://haypo.github.io/python-buildbots-2017q2.html
I will now merge my PR 3055 to be able to unblock my work on CIs. But I will wait for feedback from Gregory before backporting this fix to 2.7 and 3.6.
vstinner commentedon Aug 10, 2017
New changeset 7b7c6dc by Victor Stinner in branch 'master':
bpo-31173: Rewrite WSTOPSIG test of test_subprocess (bpo-3055)
7b7c6dc
vstinner commentedon Aug 11, 2017
New changeset bc69d00 by Victor Stinner in branch '3.6':
bpo-31173: Rewrite WSTOPSIG test of test_subprocess (bpo-3055) (bpo-3070)
bc69d00
vstinner commentedon Aug 11, 2017
Reopen. I forgot Python 2.7.
vstinner commentedon Aug 11, 2017
New changeset 4dea065 by Victor Stinner in branch '2.7':
bpo-31173: Rewrite WSTOPSIG test of test_subprocess (bpo-3055) (bpo-3071)
4dea065