Interrupted system calls are not retried

@loewis

BPO	9867
Nosy	@loewis, @ronaldoussoren, @pitrou, @benjaminp, @ned-deily, @mitsuhiko, @Trundle
Superseder	bpo-12268: file readline, readlines & readall methods can lose data on EINTR

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2012-07-12.21:35:33.709>
created_at = <Date 2010-09-16.02:02:52.080>
labels = ['interpreter-core', 'type-bug']
title = 'Interrupted system calls are not retried'
updated_at = <Date 2012-07-12.21:36:23.243>
user = 'https://github.com/mitsuhiko'

bugs.python.org fields:

activity = <Date 2012-07-12.21:36:23.243>
actor = 'pitrou'
assignee = 'none'
closed = True
closed_date = <Date 2012-07-12.21:35:33.709>
closer = 'pitrou'
components = ['Interpreter Core']
creation = <Date 2010-09-16.02:02:52.080>
creator = 'aronacher'
dependencies = []
files = []
hgrepos = []
issue_num = 9867
keywords = []
message_count = 20.0
messages = ['116504', '116505', '116521', '116524', '116525', '116529', '116530', '116532', '116534', '116538', '116539', '116540', '116541', '116544', '116545', '116546', '116547', '116548', '116549', '165335']
nosy_count = 10.0
nosy_names = ['loewis', 'ronaldoussoren', 'exarkun', 'pitrou', 'benjamin.peterson', 'ned.deily', 'stutzbach', 'aronacher', 'Trundle', 'DasIch']
pr_nums = []
priority = 'normal'
resolution = 'duplicate'
stage = None
status = 'closed'
superseder = '12268'
type = 'behavior'
url = 'https://bugs.python.org/issue9867'
versions = ['Python 2.7']

Currently Python does not check fread and other IO calls for EINTR. This usually is not an issue, but on OS X a continued program will be sent an SIGCONT signal which causes fread to be interrupted.

Testcase:

mitsuhiko@nausicaa:~$ python2.7
Python 2.7 (r27:82508, Jul  3 2010, 21:12:11) 
[GCC 4.0.1 (Apple Inc. build 5493)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from signal import SIGCONT, signal
>>> def show_signal(*args):
...  print 'Got SIGCONT'
...  
>>> signal(SIGCONT, show_signal)
0
>>> import sys
>>> sys.stdin.read()
^Z
[1]+  Stopped                 python2.7
mitsuhiko@nausicaa:~$ fg
python2.7
Got SIGCONT
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
IOError: [Errno 4] Interrupted system call
>>>

Expected behavior: on fg it should continue to read. The solution would be to loop all calls to fread and friends until errno is no longer EINTR. Now the question is how to best do that. I can't think of a portable way to define a macro that continues to run an expression until errno is EINTR, maybe someone else has an idea.

Otherwise it would be possible to just put the loops by hand around each fread/fgetc etc. call, but that would make the code quite a bit more ugly.

Technically I suppose the problem applies to all platforms, on OS X it's just easier to trigger.

The test fails exactly the same way using a python 2.6.6 on a current Debian (testing) Linux 2.6.32 so I think it better to remove the OS X from the title. Also the versions field refers to where a potential fix might be applied; that rules out 2.5 and 2.6 since it is not a security problem.

I was also curious if calling signal.siginterrupt for SIGCONT had any effect on this. Neither False nor True on either OS X or linux seemed to change the behavior.

I fail to see why this is a bug. If the system call is interrupted, why should Python not report that?

One could argue of course that every user of Python should handle EINTR, but that's something I think should be solved in the IO library because very few people know that one is supposed to restart syscalls on EINTR on POSIX systems.

Ruby for instance handles EINTR properly:

mitsuhiko@nausicaa:$ ruby -e 'puts $stdin.read.inspect'
^Z
[1]+ Stopped
mitsuhiko@nausicaa:$ fg
ruby -e 'puts $stdin.read.inspect'
test
"test\n"

So does perl:

mitsuhiko@nausicaa:$ perl -e 'chomp($x = <STDIN>); print $x'
^Z
[1]+ Stopped
mitsuhiko@nausicaa:$ fg
perl -e 'chomp($x = <STDIN>); print $x'
test
test

Interestingly even PHP handles that properly.

One could argue of course that every user of Python should handle
EINTR, but that's something I think should be solved in the IO
library because very few people know that one is supposed to restart
syscalls on EINTR on POSIX systems.

Ruby for instance handles EINTR properly:

Hmm. So under what conditions should it continue, and under what
conditions should it raise an exception (when errno is EINTR)?

Hmm. So under what conditions should it continue, and under what
conditions should it raise an exception (when errno is EINTR)?

EINTR indicates a temporary failure. In that case it should always retry.

A common macro for handling that might look like this:

#define RETRY_ON_EINTR(x) ({ \
  typeof(x) rv; \
  do { rv = x; } while (rv < 0 && errno == EINTR); \
  rv;\
})

But from what I understand, braces in parentheses are a GCC extension.

Am 16.09.10 14:06, schrieb Armin Ronacher:

Armin Ronacher<armin.ronacher@active-4.com> added the comment:

> Hmm. So under what conditions should it continue, and under what
> conditions should it raise an exception (when errno is EINTR)?

EINTR indicates a temporary failure. In that case it should always retry.

But Ruby doesn't. If you send SIGINT, it will print

-e:1:in `read': Interrupt
from -e:1

If you send SIGHUP, it will print

Hangup

So it is surely more complex than "always retry".

Wouldn't retrying on EINTR cause havoc when you try to interrupt a process?

That is: what would happen with the proposed patch when a python script does a read that takes a very long time and the user tries to interrupt the script (by using Ctrl+C to send a SIGTERM)?

If I my understanding of is correct the patch will ensure that the process does not get interupted because the default SIGTERM handler just sets a flag that's periodicly checked in the python interpreter loop. With the proposed patch python would not get around to checking that flag until the I/O operation is finished.

The following minimal C code shows how EINTR can be handled:

#include <stdlib.h>
#include <stdio.h>
#include <errno.h>
#include <signal.h>

#define BUFFER_SIZE 1024


int
main()
{
    char buffer[BUFFER_SIZE];
    printf("PID = %d\n", getpid());
    while (1) {
        int rv = fgetc(stdin);
        if (rv < 0) {
            if (feof(stdin))
                break;
            if (errno == EINTR)
                continue;
            printf("Call failed with %d\n", errno);
            return 1;
        }
        else
            fputc(rv, stdout);
    }
    return 0;
}

Test application:

mitsuhiko@nausicaa:/tmp$ ./a.out
PID = 22806
Terminated
mitsuhiko@nausicaa:/tmp$ ./a.out
PID = 22809

mitsuhiko@nausicaa:/tmp$ ./a.out
PID = 22812
^Z
[2]+ Stopped ./a.out
mitsuhiko@nausicaa:/tmp$ fg
./a.out
test
test
foo
foo

First signal sent was TERM, second was INT. Last case was sending to background, receiving the ignored SIGCONT signal, fgetc returning -1 and fgetc being called again because of errno being EINTR.

Wouldn't retrying on EINTR cause havoc when you try to interrupt a process?

All your C applications are doing it, why should Python cause havok there? Check the POSIX specification on that if you don't trust me.

That is: what would happen with the proposed patch when a python script
does a read that takes a very long time and the user tries to interrupt
the script (by using Ctrl+C to send a SIGTERM)?
EINTR is only returned if nothing was read so far and the call was interrupted in case of fread.

Here a quick explanation from the GNU's libc manual:
http://www.gnu.org/s/libc/manual/html_node/Interrupted-Primitives.html

There is a funny story related to that though :)

"BSD avoids EINTR entirely and provides a more convenient approach:
to restart the interrupted primitive, instead of making it fail."

BSD does, but the Mach/XNU kernel combo on OS X is not. Which is why all the shipped BSD tools have that bug, but if you run their GNU equivalents on OS X everything work as expected.

Some parts of the stdlib already retry manually (such as SocketIO, subprocess, multiprocessing, socket.sendall), so it doesn't sound unreasonable for the IO lib to retry too.

There are/were other people complaining in similar cases: bpo-7978, bpo-1628205.

On 16 Sep, 2010, at 14:36, Armin Ronacher wrote:

Armin Ronacher <armin.ronacher@active-4.com> added the comment:

> Wouldn't retrying on EINTR cause havoc when you try to interrupt a process?

All your C applications are doing it, why should Python cause havok there? Check the POSIX specification on that if you don't trust me.

> That is: what would happen with the proposed patch when a python script
> does a read that takes a very long time and the user tries to interrupt
> the script (by using Ctrl+C to send a SIGTERM)?
EINTR is only returned if nothing was read so far and the call was interrupted in case of fread.

Here a quick explanation from the GNU's libc manual:
http://www.gnu.org/s/libc/manual/html_node/Interrupted-Primitives.html

You conveniently didn't quote the part of my message where I explained why I think there may be a problem.

CPython's signal handlers just set a global flag to indicate that a signal occurred and run the actual signal handler later on from the main interpreter loop, see signal_handler in Modules/signal.c and intcatcher in Parser/intrcheck.c.

The latter contains the default handler for SIGINT and that already contains code that deals with SIGINT not having any effect (when you sent SIGINT twice in a row without CPython running pending calls the interpreter gets aborted).

Because Python's signal handlers only set a flag and do the actual action later on blindly rerunning system calls when errno == EINTR may result in programs that don't seem to react to signals at all.

On 16 Sep, 2010, at 14:38, Armin Ronacher wrote:

Armin Ronacher <armin.ronacher@active-4.com> added the comment:

There is a funny story related to that though :)

"BSD avoids EINTR entirely and provides a more convenient approach:
to restart the interrupted primitive, instead of making it fail."

BSD does, but the Mach/XNU kernel combo on OS X is not. Which is why all the shipped BSD tools have that bug, but if you run their GNU equivalents on OS X everything work as expected.

setting the SA_RESTART in the call to sigaction should work (on OSX HAVE_SIGACTION is defined), unless the manpage is lying.

Ronald

Because Python's signal handlers only set a flag and do the actual
action later on blindly rerunning system calls when errno == EINTR may
result in programs that don't seem to react to signals at all.

You just need to call PyErr_CheckSignals() and check its result.

setting the SA_RESTART in the call to sigaction should work (on OSX HAVE_SIGACTION is defined), unless the manpage is lying.

It should work, haven't tried. From what I understand on a BSD system, retrying is the default.

You conveniently didn't quote the part of my message where I explained
why I think there may be a problem.
I understand that, but there are already cases in Python where EINTR is handled properly. In fact, quoting socketmodule.c:

if (res == EINTR && PyErr_CheckSignals())

On 16 Sep, 2010, at 15:40, Armin Ronacher wrote:

Armin Ronacher <armin.ronacher@active-4.com> added the comment:

> You conveniently didn't quote the part of my message where I explained
> why I think there may be a problem.
I understand that, but there are already cases in Python where EINTR is handled properly. In fact, quoting socketmodule.c:

if (res == EINTR && PyErr_CheckSignals())

This looks fine.

Ronald

This has been fixed in issue bpo-10956 (Python 3 and the io module) and issue bpo-12268 (Python 2's file objects).

Navigation Menu

Repository navigation

Interrupted system calls are not retried #54076

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Interrupted system calls are not retried #54076

Description

Activity

mitsuhiko commented on Sep 16, 2010

ned-deily commented on Sep 16, 2010

loewis commented on Sep 16, 2010

mitsuhiko commented on Sep 16, 2010

mitsuhiko commented on Sep 16, 2010

loewis commented on Sep 16, 2010

mitsuhiko commented on Sep 16, 2010

loewis commented on Sep 16, 2010

ronaldoussoren commented on Sep 16, 2010

mitsuhiko commented on Sep 16, 2010

mitsuhiko commented on Sep 16, 2010

mitsuhiko commented on Sep 16, 2010

pitrou commented on Sep 16, 2010

ronaldoussoren commented on Sep 16, 2010

ronaldoussoren commented on Sep 16, 2010

pitrou commented on Sep 16, 2010

mitsuhiko commented on Sep 16, 2010

mitsuhiko commented on Sep 16, 2010

ronaldoussoren commented on Sep 16, 2010

pitrou commented on Jul 12, 2012

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions