Repository navigation
Tkinter binding involving Control-spacebar raises unicode error #45369
Description
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = 'https://github.com/kbkaiser'
closed_at = <Date 2011-05-11.21:20:52.642>
created_at = <Date 2007-08-26.20:48:31.672>
labels = ['expert-IDLE', 'type-bug', 'expert-tkinter', 'expert-unicode']
title = 'Tkinter binding involving Control-spacebar raises unicode error'
updated_at = <Date 2012-06-05.02:30:35.417>
user = 'https://github.com/kbkaiser'bugs.python.org fields:
activity = <Date 2012-06-05.02:30:35.417>
actor = 'Mike.Perry'
assignee = 'kbk'
closed = True
closed_date = <Date 2011-05-11.21:20:52.642>
closer = 'kbk'
components = ['IDLE', 'Tkinter', 'Unicode']
creation = <Date 2007-08-26.20:48:31.672>
creator = 'kbk'
dependencies = []
files = ['8323', '8330', '12054', '12094', '14674', '14675', '14677', '21954']
hgrepos = []
issue_num = 1028
keywords = ['patch', 'needs review']
message_count = 45.0
messages = ['55311', '55312', '55313', '55321', '55325', '55330', '75999', '76032', '76034', '76035', '76041', '76043', '76164', '76165', '76173', '76178', '76179', '76180', '76701', '76846', '77209', '81060', '81989', '90665', '90707', '91405', '91408', '91420', '92678', '99441', '101508', '110080', '116773', '118400', '128441', '133494', '135707', '135788', '135795', '135806', '135807', '153250', '153252', '161718', '162321']
nosy_count = 11.0
nosy_names = ['terry.reedy', 'kbk', 'vstinner', 'ned.deily', 'r.david.murray', 'Bernt.R\xc3\xb8skar.Brenna', 'BreamoreBoy', 'jsprunck', 'mgstrein', 'python-dev', 'Mike.Perry']
pr_nums = []
priority = 'high'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue1028'
versions = ['Python 3.2', 'Python 3.3']
Activity
kbkaiser commentedon Aug 26, 2007
The control-spacebar binding is used in IDLE to
force open the completions window. It's causing
IDLE to exit with a utf8 decode error. Attached
is
a Tkinter cut-down
exhibiting the problem and a patch.
The cutdown runs ok on 2.6 but not on py3k because
the latter uses PyUnicode_FromString on all the
arguments and errs out when it encounters a
character outside the utf-8 range.
Strangely, on my system, control-spacebar is
sending a two byte
string, C0E8 via the %A parameter. Control-2
does
the same. Other keys with combinations of
modifier
keys send one byte.
Linux trader 2.6.18-ARCH #1 SMP PREEMPT Sun Nov
19
09:14:35 CET 2006 i686 Intel(R) Pentium(R) 4 CPU
2.40GHz GenuineIntel GNU/Linux
Can the problem be confirmed?
Using PyUnicode_FromUnicode on %A works because
the
unicode string is copied instead of decoded, and
that parameter is supposed to be unicode, in any
case.
The patch fixes the problem on my system but
should
be reviewed, especially whether the cast in the
call
to PyUnicode_FromUnicode is suitably cross-
platform.
Assigning to Neal since he's working a lot of
Unicode issues right now. I can check it in if I
get approval.
kbkaiser commentedon Aug 26, 2007
Heh, I see we have the same damn problem SF had: when a comment is
edited,
it doesn't re-wrap properly when submitted. You have to remove the
returns
manually after editing.
kbkaiser commentedon Aug 26, 2007
Nope, you have to make sure not to type too wide.
kbkaiser commentedon Aug 26, 2007
Well, maybe someday Tk will send a multibyte unicode
character. Update the patch.
nnorwitz commentedon Aug 26, 2007
I can confirm the problem and that your patch fixes the problem. Go
ahead and check it in. Thanks!
kbkaiser commentedon Aug 27, 2007
OK, thanks for the review! I suppose Tk is sending a bad string.
r57540
ocean-city commentedon Nov 18, 2008
Sorry, I reverted r57540 because it caused segfault at IDLE exit. (See
bpo-4313) I reopened this issue.
gpolo commentedon Nov 19, 2008
I can reproduce it here with tk8.4, using tk8.5 doesn't cause this.
gpolo commentedon Nov 19, 2008
Here is a patch that doesn't use magic numbers :P
I didn't hit the problem described in bpo-4313 with this one, and
PythonCmd should be doing this anyway, but ideally we should move to
Tcl_CreateObjCommand.
30 remaining items
bitdancer commentedon Apr 11, 2011
Nudge: report on the Ubuntu bug tracker that this is still an issue with 3.2:
https://bugs.launchpad.net/bugs/517552
kbkaiser commentedon May 10, 2011
Tcl/Tk uses modified utf-8 internally. This includes using 0xC080, a multibyte Unicode null character, for embedded nulls that work with C's null terminated strings. Java does the same.
Note that typing Ctrl-space and Ctrl-2 are conventional ways to enter a null from the keyboard. That's the reason a null char is associated with those key combinations.
When Tcl exports Unicode, it is supposed to be strict utf-8. Until Tcl8.5, the %A (Unicode character corresponding to an event) was incorrectly leaking the modified Unicode null.
_tkinter.c.2.patch is narrowly focused: if PythonCmd raises a UnicodeDecodeError and if the string passed in an arg is 0xC080, it is replaced with the Unicode null 0x00.
python-dev commentedon May 11, 2011
New changeset 82cfbe2ddfbb by Kurt B. Kaiser in branch '3.1':
Issue bpo-1028: Tk returns invalid Unicode null in %A: UnicodeDecodeError.
http://hg.python.org/cpython/rev/82cfbe2ddfbb
vstinner commentedon May 11, 2011
I'm working on bpo-2857 which adds the "Modified UTF-8" ("utf-8-java"?) codec to Python. We can maybe use it instead of raising an error in 3.3?
kbkaiser commentedon May 11, 2011
r70039 3.1 forward ported > 3.2 > default. Will be in 3.2.1.
kbkaiser commentedon May 11, 2011
Having a modified utf-8 codec will be useful. That said, it is an error
for Tcl/Tk to expose modified utf-8 externally, and that was fixed at
some point in Tk8.5. Since Tk is no longer sending 0xC080 for the %A
char, switching codecs in _tkinter.c won't accomplish anything.
This fix was to correct a long-standing problem in IDLE using Tk8.4,
which is most easily solved by catching the leaked invalid null in
_tkinter.c.
It seems to me that, once you switch to modified utf-8 and allow the
embedded nulls, you have to make sure everything you are doing uses
the modified utf-8 encoding/decoding.
MikePerry commentedon Feb 13, 2012
Hello,
I am still able to reproduce this issue with Python 3.2.2. It seems as if this bug was closed with a the note:
r70039 3.1 forward ported > 3.2 > default. Will be in 3.2.1.
This leads me to believe that either 3.2.2 has a regression or the patch never made it into 3.2.1.
Can anyone chime in with some more details?
Thanks,
Mike
MikePerry commentedon Feb 13, 2012
Figured I should capture the exception. See below.
3.2.2+ (default, Jan 8 2012, 07:22:26)
[GCC 4.6.2]
terryjreedy commentedon May 27, 2012
On Win 7, a<cntl-space> brings up the box on all the latest releases: 2.7.3, 3.2.3, and 3.3.0a3. (These all come with recent tk 8.5.x.)
Mike, please retest with 3.2.3 and specify os and tk version and exactly what you entered if there is still a problem.
MikePerry commentedon Jun 5, 2012
Looking good in 3.2.3! Tested on Debian Wheezy using packages
python3-tk 3.2.3-1 and idle3 3.2.3~rc1-2.
/*
* Mike Perry
* mike@cogs....com
*/
On Sun, May 27, 2012 at 4:10 PM, Terry J. Reedy <report@bugs.python.org> wrote: