Repository navigation
Double coding cookie #70768
Description
| BPO | 26581 |
|---|---|
| Nosy | @malemburg, @gvanrossum, @loewis, @vstinner, @serhiy-storchaka |
| Files |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = 'https://github.com/serhiy-storchaka'
closed_at = <Date 2016-03-20.21:52:08.611>
created_at = <Date 2016-03-17.12:00:36.043>
labels = ['interpreter-core', 'type-bug', 'library']
title = 'Double coding cookie'
updated_at = <Date 2016-03-20.21:52:08.611>
user = 'https://github.com/serhiy-storchaka'bugs.python.org fields:
activity = <Date 2016-03-20.21:52:08.611>
actor = 'serhiy.storchaka'
assignee = 'serhiy.storchaka'
closed = True
closed_date = <Date 2016-03-20.21:52:08.611>
closer = 'serhiy.storchaka'
components = ['Interpreter Core', 'Library (Lib)']
creation = <Date 2016-03-17.12:00:36.043>
creator = 'serhiy.storchaka'
dependencies = []
files = ['42185']
hgrepos = []
issue_num = 26581
keywords = ['patch']
message_count = 8.0
messages = ['261909', '262051', '262052', '262053', '262054', '262089', '262090', '262092']
nosy_count = 6.0
nosy_names = ['lemburg', 'gvanrossum', 'loewis', 'vstinner', 'python-dev', 'serhiy.storchaka']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue26581'
versions = ['Python 2.7', 'Python 3.5', 'Python 3.6']
Activity
serhiy-storchaka commentedon Mar 17, 2016
When Python source file contains double coding cookies on different lines, the first wins. When it contains double coding cookies on the same line, the last wins.
PEP-263 was sufficiently vague about this. Now this is clarified (22490711c870). The first coding cookie should always win.
Proposed patch fixes Python tokenizer, the tokenize module, and other places. Tests are taken from bpo-25643.
serhiy-storchaka commentedon Mar 19, 2016
I just tested with Emacs, and it looks that when specify different codings on two different lines, the first coding wins, but when specify different codings on the same line, the last coding wins.
Therefore current CPython behavior can be correct, and the regular expression in PEP-263 should be changed to use greedy repetition.
gvanrossum commentedon Mar 19, 2016
Do you have write permission to the PEP? Just update it.
serhiy-storchaka commentedon Mar 19, 2016
Yes, I have. But I were not sure what behavior should be correct in Python. On one side, always choosing the first declaration (on the same or on different lines) looks more consistent. On other side, current behavior was in CPython from the initial implementing PEP-263 in bpo-526840 and it matches Emacs behavior (if I understand this correctly).
I can update the regular expression, but may be this obscure corner case needs the verbal explanation.
gvanrossum commentedon Mar 19, 2016
Right. Please go ahead with both. I am fine with defining the current
behavior correct.
--Guido (mobile)
On Mar 19, 2016 9:37 AM, "Serhiy Storchaka" <report@bugs.python.org> wrote:
serhiy-storchaka commentedon Mar 20, 2016
Ah, I made a mistake! In 2.7 the first coding on the same line wins. And that behavior was from start. Regression was unintentionally introduced in bpo-18470.
Thus *there is* a bug in Python 3. PEP-263 doesn't need more changes, but Python tokenizer and related tools do.
Sorry for misleading.
python-dev commentedon Mar 20, 2016
New changeset 23a7481eafd4 by Serhiy Storchaka in branch 'default':
Issues bpo-25643, bpo-26581: Added new tests for detecting Python source code encoding.
https://hg.python.org/cpython/rev/23a7481eafd4
python-dev commentedon Mar 20, 2016
New changeset 1c44cea2ea8f by Serhiy Storchaka in branch '3.5':
Issue bpo-26581: Use the first coding cookie on a line, not the last one.
https://hg.python.org/cpython/rev/1c44cea2ea8f
New changeset 8506d127d482 by Serhiy Storchaka in branch '2.7':
Issue bpo-26581: Use the first coding cookie on a line, not the last one.
https://hg.python.org/cpython/rev/8506d127d482
New changeset e86cd4a872b8 by Serhiy Storchaka in branch 'default':
Issue bpo-26581: Use the first coding cookie on a line, not the last one.
https://hg.python.org/cpython/rev/e86cd4a872b8