import + coding = failure (3.1.2/win32)

@amauryfa

BPO	8988
Nosy	@amauryfa, @vstinner, @merwok, @iritkatriel
Superseder	bpo-3080: Full unicode import system
Files	pybug-import-coding.zip: bug-generating sources 'a.py' & 'b.py'

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2020-11-06.18:01:17.246>
created_at = <Date 2010-06-13.10:50:33.703>
labels = ['interpreter-core', 'type-bug']
title = 'import + coding = failure (3.1.2/win32)'
updated_at = <Date 2020-11-06.18:01:17.245>
user = 'https://bugs.python.org/gonegown'

bugs.python.org fields:

activity = <Date 2020-11-06.18:01:17.245>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2020-11-06.18:01:17.246>
closer = 'vstinner'
components = ['Interpreter Core']
creation = <Date 2010-06-13.10:50:33.703>
creator = 'gonegown'
dependencies = []
files = ['17655']
hgrepos = []
issue_num = 8988
keywords = []
message_count = 21.0
messages = ['107731', '107783', '108184', '108371', '108693', '108720', '108721', '108801', '109167', '109168', '109967', '110047', '110420', '110421', '112029', '112034', '119105', '119106', '227781', '380461', '380462']
nosy_count = 5.0
nosy_names = ['amaury.forgeotdarc', 'vstinner', 'eric.araujo', 'gonegown', 'iritkatriel']
pr_nums = []
priority = 'normal'
resolution = 'duplicate'
stage = 'resolved'
status = 'closed'
superseder = '3080'
type = 'behavior'
url = 'https://bugs.python.org/issue8988'
versions = ['Python 3.1']

I have python 3.1.2 fetched from the main site.

imagine two source files:

a.py:
-------

# coding: cp1251
import b;
print('A');

b.py:
-------
print('B');
-------

Both reside in the same directory containing at least one non-ascii character (try 0xdb) in the _path_.

import will fail with an empty error!
#coding here works fine with utf-8 and fails using any other one

now tell me how the hell can file system encoding be related to file content encoding?!

I've attached the source

This bug is maybe related to bpo-8611?

Can you try with py3k (python 3.2)?

Is there py3k for win32?
And how do I know if bpo-8611 comes from the same source?
Have no idea how they have organized the python core. I'm new to python (about 2 months) and I don't think I will use it for long. It's just not serious.

@Haypo

now tell me how the hell can file system encoding be related
to file content encoding?!

Why do you say so? I can reproduce your issue, but changing the first line of a.py:
# coding: cp1252
to:
# coding: utf-8
did not change anything.

In the meantime, you should refrain from creating directories with characters not representable in the terminal window.

@Haypo: The problem still exists with py3k at r82150.

@Amaury

@Amaury:
What you're saying about directory naming is right indeed.
But the case has begun from cyrillic letters in the NTFS path, which I do not use, but the users of my soft do. So putting the program into such directory makes the former unuseable; until the sources are in utf anyway.

I just ran this on another computer and it seemed to work with #coding in a.py. I then added this line to b.py and it failed. I played about 15 minutes inserting the line and removing and changing the directory name.
And I can tell the behaviour for me looks just random!
Though I noticed that addding #coding line to both sources fails more often.

You'll see:
--------

Traceback (most recent call last):
  File "F:\1home\С\u201e\a.py", line 1, in <module>
SyntaxError: None

And what the hell is this u201e? That should have been a letter!

File "F:\1home\С\u201e\a.py", line 1, in <module>
And what the hell is this u201e? That should have been a letter!

It's probably this symbol: http://www.eki.ee/letter/chardata.cgi?ucode=201e
but it has no representation in the console windows you are using; try "import sys; print(sys.stderr.encoding)" to print the code page used by your console.
In error messages, Python replaces unpritable characters with their "escaped" form: \uXXXX where XXXX is the character number.

But the case has begun from cyrillic letters in the NTFS path,
which I do not use, but the users of my soft do.
So putting the program into such directory makes the former unuseable;
until the sources are in utf anyway.

I agree that cyrillic letters in the path makes the program unusable. This is the bug to fix, and the zip file you attached is a good test case.

But I still don't see how "the sources are in utf" can influence this. Please show me!

Did you read my first comment? "This bug is maybe related to bpo-8611"

@Amaury

@Amaury:
Removing #coding lines or replacing them with #coding: utf-8 makes this test case working, at least on 4 computers I have been able to test this.

My initial program was consisting of roughly ten files and utf-8 made it work.

@Haypo:
"maybe"...

Here is what I did, on a machine running Windows XP, with python 3.1.1:

I used 7-zip to extract the attached zip file, in the c:\temp directory.
Then I opened a command prompt, here is an exact copy of the session:
C:>cd \temp\█

C:\temp\█>dir
Le volume dans le lecteur C s'appelle Disque dur
Le numéro de série du volume est D4BA-260C

Répertoire de C:\temp\█

03/07/2010 11:10 <REP> .
03/07/2010 11:10 <REP> ..
08/06/2010 09:13 44 a.py
08/06/2010 14:21 11 b.py
2 fichier(s) 55 octets
2 Rép(s) 58 733 801 472 octets libres

C:\temp\█>c:\Python31\python.exe a.py
Traceback (most recent call last):
  File "a.py", line 3, in <module>
    import b;
ImportError: No module named b

C:\temp\█>notepad a.py
[Replaced encoding with "utf-8", then save and quit]

C:\temp\█>c:\Python31\python.exe a.py
Traceback (most recent call last):
  File "a.py", line 2, in <module>
    import b;
ImportError: No module named b

@Amaury

@Amaury:
Just fine!
It's either another bug in python or 3.1.1 specifics.

Just fine!
It's either another bug in python or 3.1.1 specifics.

What do you mean? what is 'it'? The error I in the session above shows the bug we described first (strange letters in the path makes the program unusable), and shows that the file's encoding doesn't change this.

@Amaury

@Amaury:
error message for my bug was:
SyntaxError: None
and for your:
ImportError: No module named b

We've got at least two bugs in one testcase

Then please tell us how to reproduce the "SyntaxError" case

I wrote a patch on import machinery to support unicode characters: see bpo-9425.

I tested pybug-import-coding.zip on Windows with my import_unicode branch (see bpo-9425): it works correctly, whereas it fails with py3k (Python 3.2).

I tested the example with Python 3.2 (r85691) and the issue looks to be fixed. Can someone else confirm that?

I decompressed the ZIP archive, moved into the directory containing a.py and b.py, and called \path\to\python.exe a.py: it works.

See also bpo-3080 for the full unicode support in the import machinery.

Works for me using 3.4.1 and 3.5.0a0.

I think this can be closed as out of date.

See also bpo-3080 for the full unicode support in the import machinery.

I'm confident that I fixed this issue in bpo-3080. I mark this one as a duplicate.

Navigation Menu

Repository navigation

import + coding = failure (3.1.2/win32) #53234

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

import + coding = failure (3.1.2/win32) #53234

Description

Activity

gonegown commented on Jun 13, 2010

vstinner commented on Jun 14, 2010

gonegown commented on Jun 19, 2010

amauryfa commented on Jun 22, 2010

gonegown commented on Jun 26, 2010

amauryfa commented on Jun 26, 2010

amauryfa commented on Jun 26, 2010

vstinner commented on Jun 27, 2010

gonegown commented on Jul 3, 2010

amauryfa commented on Jul 3, 2010

gonegown commented on Jul 11, 2010

amauryfa commented on Jul 11, 2010

gonegown commented on Jul 16, 2010

amauryfa commented on Jul 16, 2010

vstinner commented on Jul 30, 2010

vstinner commented on Jul 30, 2010

vstinner commented on Oct 19, 2010

vstinner commented on Oct 19, 2010

BreamoreBoy commented on Sep 28, 2014

iritkatriel commented on Nov 6, 2020

vstinner commented on Nov 6, 2020

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions