Repository navigation
py3k fails under Windows if "-c" or "-m" is given a non-ascii value #47955
Description
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = None
closed_at = <Date 2008-11-11.23:05:34.430>
created_at = <Date 2008-08-27.18:05:40.472>
labels = ['type-bug', 'release-blocker']
title = 'py3k fails under Windows if "-c" or "-m" is given a non-ascii value'
updated_at = <Date 2019-01-10.12:01:30.676>
user = 'https://github.com/pitrou'bugs.python.org fields:
activity = <Date 2019-01-10.12:01:30.676>
actor = 'ossdev07'
assignee = 'none'
closed = True
closed_date = <Date 2008-11-11.23:05:34.430>
closer = 'amaury.forgeotdarc'
components = []
creation = <Date 2008-08-27.18:05:40.472>
creator = 'pitrou'
dependencies = []
files = ['11273', '11411', '11422', '11424', '11545']
hgrepos = []
issue_num = 3705
keywords = ['patch']
message_count = 15.0
messages = ['72036', '72040', '72682', '72692', '72714', '72720', '72771', '72773', '72800', '72826', '72828', '73533', '75764', '75765', '75768']
nosy_count = 4.0
nosy_names = ['loewis', 'amaury.forgeotdarc', 'pitrou', 'benjamin.peterson']
pr_nums = ['11497', '11497', '11497']
priority = 'release blocker'
resolution = 'fixed'
stage = None
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue3705'
versions = ['Python 3.0']
Activity
pitrou commentedon Aug 27, 2008
The explanation is quite simple: in Py_Main, the arguments are converted
from wide to byte strings, but the required length of the byte string is
assumed equal to that of the wide string.
Which gives:
$ ./python -c "print('à')" Fatal Python error: not enough memory to copy -c argument Erreur de segmentation (core dumped) $ ./python -m à Fatal Python error: not enough memory to copy -m argument Erreur de segmentation (core dumped)pitrou commentedon Aug 27, 2008
Here is a patch which works under Linux. Under Windows it doesn't choke
when converting arguments anymore, but it fails later in the process (in
the parser for '-c', in the importing logic for '-m').
Here is an example:
benjaminp commentedon Sep 6, 2008
Hmm. I suppose anything is better than segfaulting. I think the patch is
fine for now, though.
pitrou commentedon Sep 6, 2008
Committed in r66269.
[-]py3k aborts if "-c" or "-m" is given a non-ascii value[/-][+]py3k fails under Windows if "-c" or "-m" is given a non-ascii value[/+]amauryfa commentedon Sep 6, 2008
This patch corrects the "-m" case on windows: the path has to be
decoded/recoded using the filesystem encoding, and not the default utf-8.
Review is needed, of course.
pitrou commentedon Sep 6, 2008
Looks good and works under Linux.
One small nit, you could just as well use "NN(ssi)" for the
Py_BuildValue and remove Py_DECREF(fob), so as to be more consistent.
amauryfa commentedon Sep 8, 2008
Updated patch.
amauryfa commentedon Sep 8, 2008
./python -c "print('à')"
does not work on my Linux machine with latest py3k (r66303), certainly
because my terminal uses a latin-1 encoding: wcstombs will convert the
argument back to the terminal encoding, whereas PyRun_SimpleString
expects a UTF-8 string.
I join another patch, which propagates the wchar_t as far as possible,
and encodes it as utf-8; with test.
This also corrects the Windows case.
benjaminp commentedon Sep 8, 2008
I think the patch good; go ahead.
amauryfa commentedon Sep 9, 2008
Applied both patches as r66331.
amauryfa commentedon Sep 9, 2008
Unfortunately, my patch does not work: see the compile warnings in "main.c":
http://www.python.org/dev/buildbot/3.0/x86%20osx.5%203.0/builds/344/step-compile/0
I reverted the change, and will try something else...
amauryfa commentedon Sep 21, 2008
Today I learned something: wchar_t can be 2 or 4 bytes, PyUNICODE can be
2 or 4 bytes, and all combinations are possible.
My error was to use PyUnicode_FromUnicode on a wchar_t*; PyUnicode_FromWideChar is the obvious function to use.
Attached a new patch (command_unicode_2.patch) for review.
amauryfa commentedon Nov 11, 2008
Raising to release blocker, just to trigger another review...
benjaminp commentedon Nov 11, 2008
Go ahead.
amauryfa commentedon Nov 11, 2008
Fixed as r67190. Thanks for the review.