input() uses sys.__stdout__ instead of sys.stdout for prompt

@pitrou

BPO	24402
Nosy	@pitrou, @taleinat, @berkerpeksag, @vadmium, @serhiy-storchaka, @eryksun, @zhangyangyu
Files	issue24402.fix_only.patch: patch with just a fix (no tests) input-terminal.patch: Now with test case

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/vadmium'
closed_at = <Date 2015-10-10.10:04:49.822>
created_at = <Date 2015-06-07.09:30:52.044>
labels = ['interpreter-core', 'type-bug']
title = 'input() uses sys.__stdout__ instead of sys.stdout for prompt'
updated_at = <Date 2015-10-10.10:04:49.821>
user = 'https://bugs.python.org/KeitaKita'

bugs.python.org fields:

activity = <Date 2015-10-10.10:04:49.821>
actor = 'martin.panter'
assignee = 'martin.panter'
closed = True
closed_date = <Date 2015-10-10.10:04:49.822>
closer = 'martin.panter'
components = ['Interpreter Core']
creation = <Date 2015-06-07.09:30:52.044>
creator = 'Keita Kita'
dependencies = []
files = ['39697', '40707']
hgrepos = []
issue_num = 24402
keywords = ['patch']
message_count = 15.0
messages = ['244949', '244957', '245279', '245280', '252436', '252455', '252458', '252676', '252677', '252683', '252686', '252688', '252689', '252690', '252699']
nosy_count = 9.0
nosy_names = ['pitrou', 'taleinat', 'python-dev', 'berker.peksag', 'martin.panter', 'serhiy.storchaka', 'eryksun', 'xiang.zhang', 'Keita Kita']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue24402'
versions = ['Python 3.4', 'Python 3.5', 'Python 3.6']

On Python 3.4, input() does not use wrapped sys.stdout. For example, the following script should print "Wrapped stdout" by print() and input() because sys.stdout is replaced with Writer object.

--------

import io
import sys


class Writer:
    def __getattr__(self, name):
        return getattr(sys.__stdout__, name)

    def write(self, data):
        if data != '\n':
            sys.__stdout__.write('Wrapped stdout\n')

    def fileno():
        raise OSError()


sys.stdout = Writer()

print('print')
input('input')

But the script does not print "Wrapped stdout" as prompt of input(). The script
prints the following.

-----
Wrapped stdout
input
----

Although, when sys.stdin is also wrapped, input() will use sys.stdout for prompt. For example, the following script prints "Wrapped stdout" by print() and input().

----

import io
import sys


class Writer:
    def __getattr__(self, name):
        return getattr(sys.__stdout__, name)

    def write(self, data):
        if data != '\n':
            sys.__stdout__.write('Wrapped stdout\n')

    def fileno():
        raise OSError()


class Reader:
    def __getattr__(self, name):
        return getattr(sys.__stdin__, name)

    def read(self, size):
        return sys.__stdin__.read(size)

    def fileno():
        raise OSError()


sys.stdout = Writer()
sys.stdin = Reader()

print('print')
input('input')

The script prints the following.

-----
Wrapped stdout
Wrapped stdout
----

This looks like an oversight. In the implementation in bltinmodule.c, if checking sys.stdout.fileno() fails, it clears the error without setting tty = 0. It's the same in 3.5.

    /* We should only use (GNU) readline if Python's sys.stdin and
       sys.stdout are the same as C's stdin and stdout, because we
       need to pass it those. */
    tmp = _PyObject_CallMethodId(fin, &PyId_fileno, "");
    if (tmp == NULL) {
        PyErr_Clear();
        tty = 0;
    }
    else {
        fd = PyLong_AsLong(tmp);
        Py_DECREF(tmp);
        if (fd < 0 && PyErr_Occurred())
            return NULL;
        tty = fd == fileno(stdin) && isatty(fd);
    }
    if (tty) {
        tmp = _PyObject_CallMethodId(fout, &PyId_fileno, "");
        if (tmp == NULL)
            PyErr_Clear();
        else {
            fd = PyLong_AsLong(tmp);
            Py_DECREF(tmp);
            if (fd < 0 && PyErr_Occurred())
                return NULL;
            tty = fd == fileno(stdout) && isatty(fd);
        }
    }

Okay, so the change is straightforward. With this change the entire test suite passes on my machine.

We'll need a test for this for the test suite, and that's far less straightforward. It seems that test.test_builtin.BuiltinTest.check_input_tty() could be a starting point. Could someone come up with a working test for this?

Here's a patch with just the required fix (no tests).

Patch itself looks good.

This patch includes a test case using pty.fork() to ensure stdin is a terminal, so the bug would be triggered. It is a bit tricky, but hopefully works on Unix platforms. I have only tested it on Linux.

LGTM. See also minor note on Rietveld.

I think it is worth to extract all input() tests to separate class.

Okay I plan to commit a modified version where I moved the input() test methods that use pseudo terminals to a new PtyTests class. The test_input() method stays in the original class, because it shares a helper method with other tests.

I am resisting dropping that decode() step, because it makes any error message from the forked child hard to read:

AssertionError: 256 != 0 : b'quux\r\nTraceback (most recent call last):\r\n  File "/media/disk/home/proj/python/cpython/Lib/test/test_builtin.py", line 1593, in test_input_no_stdout_fileno\r\n    input("prompt")\r\nTypeError: bad argument type for built-in operation\r\n'

New changeset 425d81d10b13 by Martin Panter in branch '3.4':
Issue bpo-24402: Fix input() when stdout.fileno() fails; diagnosed by Eryksun
https://hg.python.org/cpython/rev/425d81d10b13

New changeset e97d940a6543 by Martin Panter in branch '3.5':
Issue bpo-24402: Merge input() fix from 3.4 into 3.5
https://hg.python.org/cpython/rev/e97d940a6543

New changeset bcc0f8eb6797 by Martin Panter in branch 'default':
Issue bpo-24402: Merge input() fix from 3.5
https://hg.python.org/cpython/rev/bcc0f8eb6797

Seems my test hangs the Free BSD 9 buildbot, although Free BSD 10 apparently passed.

http://buildbot.python.org/all/builders/AMD64%20FreeBSD%209.x%203.4/builds/1128/steps/test/logs/stdio

[390/390] test_builtin
Timeout (1:00:00)!
Thread 0x0000000801407400 (most recent call first):
File "/usr/home/buildbot/python/3.4.koobs-freebsd9/build/Lib/test/test_builtin.py", line 1585 in test_input_no_stdout_fileno
File "/usr/home/buildbot/python/3.4.koobs-freebsd9/build/Lib/unittest/case.py", line 580 in run
File "/usr/home/buildbot/python/3.4.koobs-freebsd9/build/Lib/unittest/case.py", line 628 in __call__

This test also fails on Ubuntu.

[1/1] test_builtin
test test_builtin failed -- Traceback (most recent call last):
  File "/home/angwer/my-cpython/Lib/test/test_builtin.py", line 1588, in test_input_no_stdout_fileno
    self.assertEqual(status, 0, output)
AssertionError: 256 != 0 : quux
Traceback (most recent call last):
  File "/home/angwer/my-cpython/Lib/test/test_builtin.py", line 1593, in test_input_no_stdout_fileno
    input("prompt")
TypeError: bad argument type for built-in operation

1 test failed:
test_builtin

I wrote this on Arch Linux, so I doubt your Ubuntu failure is valid. The error you see is what I would expect if the bug were not actually fixed. I suspect you need to recompile /Python/bltinmodule.c, which contains the bug fix.

However, the test also hangs OS X buildbots. I can only guess at why it is hanging, either at the waitpid() line or the os.read() line. I am looking at refactoring my test to use the existing check_input_tty() infrastructure. It seems over the top by adding a separate pipe back to the parent process, but presumably will workaround whatever is causing the hang.

It's my fault. I forget to recompile. Now it works.

New changeset d8dd9015b086 by Martin Panter in branch '3.4':
Issue bpo-24402: Factor out PtyTests.run_child() in input() tests
https://hg.python.org/cpython/rev/d8dd9015b086

New changeset 6a8f96b46dce by Martin Panter in branch '3.5':
Issue bpo-24402: Merge potential test fix from 3.4 into 3.5
https://hg.python.org/cpython/rev/6a8f96b46dce

New changeset cb574ee7231e by Martin Panter in branch 'default':
Issue bpo-24402: Merge potential test fix from 3.5
https://hg.python.org/cpython/rev/cb574ee7231e

My second attempt seems to have avoided the hang.

For the record, the backtrace from 3.4.koobs-freebsd9 did say it was hanging at the os.waitpid() line, but I still don’t understand why.

Navigation Menu

Repository navigation

input() uses sys.stdout instead of sys.stdout for prompt #68590

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

input() uses sys.__stdout__ instead of sys.stdout for prompt #68590

Description

Activity

KeitaKita commented on Jun 7, 2015

eryksun commented on Jun 7, 2015

taleinat commented on Jun 12, 2015

taleinat commented on Jun 12, 2015

vadmium commented on Oct 6, 2015

vadmium commented on Oct 7, 2015

serhiy-storchaka commented on Oct 7, 2015

vadmium commented on Oct 10, 2015

python-dev commented on Oct 10, 2015

vadmium commented on Oct 10, 2015

zhangyangyu commented on Oct 10, 2015

vadmium commented on Oct 10, 2015

zhangyangyu commented on Oct 10, 2015

python-dev commented on Oct 10, 2015

vadmium commented on Oct 10, 2015

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions

input() uses sys.stdout instead of sys.stdout for prompt #68590