Repository navigation
Use interactive input even if stdout is redirected #69017
Description
| BPO | 24829 |
|---|---|
| Nosy | @rhettinger, @bitdancer, @vadmium, @eryksun |
| Dependencies |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = None
closed_at = None
created_at = <Date 2015-08-08.14:48:56.820>
labels = ['type-bug', '3.8', '3.9', '3.10']
title = 'Use interactive input even if stdout is redirected'
updated_at = <Date 2021-03-15.21:02:22.443>
user = 'https://bugs.python.org/Drekin'bugs.python.org fields:
activity = <Date 2021-03-15.21:02:22.443>
actor = 'eryksun'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = []
creation = <Date 2015-08-08.14:48:56.820>
creator = 'Drekin'
dependencies = ['17620']
files = []
hgrepos = []
issue_num = 24829
keywords = []
message_count = 12.0
messages = ['248271', '249098', '249144', '249145', '249148', '249149', '249154', '249157', '249166', '249294', '249454', '249469']
nosy_count = 5.0
nosy_names = ['rhettinger', 'r.david.murray', 'martin.panter', 'eryksun', 'Drekin']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue24829'
versions = ['Python 3.8', 'Python 3.9', 'Python 3.10']
Activity
Drekin commentedon Aug 8, 2015
Currently, if one redirects stdout, readline hook is not used (see https://hg.python.org/cpython/file/default/Parser/myreadline.c#l208). I would assume that if I run Python as "py -i > output.txt", I can use GNU readline or other readline hook for input just like normal; only the output is redirected to the file.
vadmium commentedon Aug 25, 2015
I think Gnu Readline uses standard output for echoing input, prompts, controlling the cursor, etc. You can already see this by redirecting to a separate terminal. In Linux:
$ python > /dev/pts/2So I think your assumption is not valid. Perhaps you can redirect standard error instead? (python -i . . . 2> output.txt, at least in Linux.) Or maybe use some terminal logging program, like the “script” Unix command. But I’m not sure there is a good use case for enabling Readline without a terminal to output to.
Drekin commentedon Aug 25, 2015
Regarding the prompt, I think it should be always written to sys.stdout, so we may hypothetically add a wrapper around GNU readline writing the prompt to sys.stdout and calling the actual readline with empty prompt.
So you are telling me that the characters displayed in console when entering input are actually written to stdout in Linux (I think it's not the case in Windows)? Maybe it woudn't matter if they were written to console even when stdout is redirected.
I'm not saying it is extra useful to redirect stdout while providing interactive input, I was just wondering what should be the right behavior. And from my point of view motivated as explained above, readlinehook should be used even with redirected stdout, at least in Windows.
bitdancer commentedon Aug 25, 2015
The behavior of programs definitely differs if stdout is redirected. For example on linux a program might output color codes to stdout normally, but if it is redirected to a non-tty, no color codes would be output.
Windows console handling of unicode is an entirely separate problem for which there are open issues and some possible solutions being worked on.
Drekin commentedon Aug 25, 2015
R. David Murray: I understand that the behavior of programs differ if stdout is redirected. I just claim that at least in Windows, stdin and stdout are completely independent, so redirecting *stdout* shouldn't affect how data are read from *stdin*.
I know that Unicode handling in Windows is a completely different issue. I was just explaining how I ran into this issue. During my work on
win-unicode-consoleI found out there is some (at least for me) suprising behavior regarding the stdio implementation in Python. So I raised issues (this one, bpo-17620, bpo-18597, bpo-18838).bitdancer commentedon Aug 25, 2015
Well, I think we're constrained by the behavior of readline here.
Drekin commentedon Aug 25, 2015
Behavior of which readline? GNU readline? Note that it is only one particular implmentation on one particular platform while Python should be as multiplatform as possible / viable.
And what behavior it is? More precisely, what incorrect behavior in Linux would it bring if the condition in
was replaced by
!isatty (fileno (sys_stdin))?bitdancer commentedon Aug 25, 2015
It's the one we started with, and the one in most common use.
I don't even know if there would be a problem. The thing to do would be to try it and find out (on both linux and windows).
vadmium commentedon Aug 25, 2015
In Linux, the original stdin, stdout, stderr file descriptors refer to the same console by default, and you use normal file reads and writes on them. In Linux when Python uses Gnu Readline, the displayed input characters are indeed written to the original stdout file descriptor. For comparison, when Readline is not used, the displayed characters go to the original stdin console, because they are handled at a lower OS level before they are even read by Python and Readline.
By experimentation, the Bash shell seems to work similarly, except it uses stderr rather than stdout. You can freely redirect stdout, but if you redirect stderr it affects the output seen on the terminal. It may be possible to write to stdin to avoid the input and output being split up, but I think writing to an input-only file descriptor is a bad idea.
If you dropped the isatty() check for stdout in Linux, you may end up with the output of Readline in the file (assuming Readline is okay with this). The file would include all the cursor positioning codes, backspaces, etc, that Readline generates. But I haven’t actually tried this.
eryksun commentedon Aug 28, 2015
Yes, that's what happens.
It seems to me the isatty check could be moved to call_readline in Modules/readline.c. This way PyOS_ReadlineFunctionPointer decides whether to call PyOS_StdioReadline. (See bpo-512981 for the patch that introduced the isatty check.)
A related issue in Windows is that when stdout isn't a console, the text encoding defaults to the locale's ANSI encoding (e.g. cp1252). UTF-8 can be forced by setting the environment variable PYTHONIOENCODING=utf-8.
Also of concern is the pyreadline module. Its [Console class] assumes that the process standard handles are console handles. This is generally valid given the isatty check. OTOH, when I removed this check and redirected stdout to a file, pyreadline crashed in an endless loop.
pyreadline's hook could punt to PyOS_StdioReadline if stdin and stdout don't both refer to the console. The problem there is calling the CRT's _fileno (POSIX) function in Python. Maybe the os module could wrap fileno in Modules/posixmodule.c. In addition to accepting a FILE pointer, os.fileno could accept an enum of C_STDIN, C_STDOUT, and C_STDERR.
rhettinger commentedon Sep 1, 2015
I recommend this feature request be rejected. The current behavior has been in place for a long time. Changing it may cause more harm than good (especially if the behavior between operating systems becomes more divergent).
Drekin commentedon Sep 1, 2015
How about reconsidering in the case that the machinery around PyOS_Readline is rewritten as I suggest in bpo-17620 ?
sswam commentedon Oct 17, 2024
Readline is broken here, it should use stderr for line editing not stdout.