Repository navigation
os.execvpe() doesn't support surrogates in env #52638
Description
| BPO | 8391 |
|---|---|
| Nosy | @vstinner |
| Files |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = None
closed_at = <Date 2010-04-23.21:45:07.683>
created_at = <Date 2010-04-14.00:01:36.067>
labels = ['library', 'expert-unicode']
title = "os.execvpe() doesn't support surrogates in env"
updated_at = <Date 2010-04-25.22:40:52.782>
user = 'https://github.com/vstinner'bugs.python.org fields:
activity = <Date 2010-04-25.22:40:52.782>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2010-04-23.21:45:07.683>
closer = 'vstinner'
components = ['Library (Lib)', 'Unicode']
creation = <Date 2010-04-14.00:01:36.067>
creator = 'vstinner'
dependencies = []
files = ['16940']
hgrepos = []
issue_num = 8391
keywords = ['patch']
message_count = 9.0
messages = ['103100', '103101', '103107', '103277', '103278', '103279', '103459', '104055', '104175']
nosy_count = 2.0
nosy_names = ['vstinner', 'Arfrever']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = None
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue8391'
versions = ['Python 3.1', 'Python 3.2']
Activity
vstinner commentedon Apr 14, 2010
It would be nice to support the PEP-383 (surrogateescape) for environment variables in os.execvpe(). Attached patch uses PyUnicode_AsEncodedString(val, Py_FileSystemDefaultEncoding, "surrogateescape") to encode an environment variable value.
I'm not sure that PyUnicode_AsEncodedString(val, Py_FileSystemDefaultEncoding, "surrogateescape") does always return a PyBytes object.
I not patched environment keys, but it might be useful.
vstinner commentedon Apr 14, 2010
See also issue bpo-4036.
vstinner commentedon Apr 14, 2010
See also bpo-8393.
vstinner commentedon Apr 16, 2010
My patch doesn't work for types bytes and bytearray.
I noticed that py3k uses surrogateescape to encode environment variable values ;-)
vstinner commentedon Apr 16, 2010
Other notes: Environment variable *names* use also surrogateescape "encoding". os.spawnve() and os.spawnvpe() should also be patched (the code should also be factorized).
vstinner commentedon Apr 16, 2010
New version of the patch:
Because of the factorization, the error messages doesn't contain the function name anymore. spawnve() and spawnvpe() omit BEGINLIBPATH and ENDLIBPATH, as execve(): "that Would Confuse Programs if Passed On". I suppose that if execve() ignore them, spawn*e() should also ignore them.
I don't have an OS/2, so I'm unable to test my patch on this OS :-/
Note: The patch fixes also subprocess to support bytes and bytearray in the environment dictionary.
vstinner commentedon Apr 18, 2010
Current code of execve() has a bug: it uses the length of the environment variable value in *characters* and not in *bytes* to allocate the "p" buffer. I remember that someone wrote a comment somewhere about that... The result is that the environment variable value is truncated by 1 byte.
Example (copy of http://dpaste.com/184803/):
-----------
declare -p VAR
$ python2.6 test.py
declare -x VAR="ćd"
$ python3.1 test.py
declare -x VAR="ć"
-----------
vstinner commentedon Apr 23, 2010
Commited: r80421 (py3k), blocked in 3.1 (80422). The commit fixes also os.getenv() to support bytes environment name.
vstinner commentedon Apr 25, 2010
I blocked the fix in Python 3.1 because it's non trivial and I prefer to avoid complex changes in Python 3.1. But then I realized that Python 3.1 has two bugs about environment variables.
It uses sys.getfilesystemencoding()+surrogateecape to decode variables and sys.getdefaultencoding()+strict to encode variables: the encoding is different!
It counts the number of *characters* to allocate the *byte* string buffer and so non-ASCII values are truncated.
So I decided to backport the fix: r80494.