distutils fail to install unicode-encoded files with POSIX locale

@tarekziade

BPO	10051
Nosy	@tarekziade, @merwok, @mgorny
Superseder	bpo-9561: distutils: set encoding to utf-8 for input and output files
Files	build.log: Complete Gentoo build log

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/tarekziade'
closed_at = <Date 2010-10-18.18:08:42.999>
created_at = <Date 2010-10-08.11:49:37.377>
labels = ['type-bug', 'library']
title = 'distutils fail to install unicode-encoded files with POSIX locale'
updated_at = <Date 2010-10-18.18:08:42.997>
user = 'https://github.com/mgorny'

bugs.python.org fields:

activity = <Date 2010-10-18.18:08:42.997>
actor = 'eric.araujo'
assignee = 'tarek'
closed = True
closed_date = <Date 2010-10-18.18:08:42.999>
closer = 'eric.araujo'
components = ['Distutils']
creation = <Date 2010-10-08.11:49:37.377>
creator = 'mgorny'
dependencies = []
files = ['19159']
hgrepos = []
issue_num = 10051
keywords = []
message_count = 2.0
messages = ['118188', '119049']
nosy_count = 3.0
nosy_names = ['tarek', 'eric.araujo', 'mgorny']
pr_nums = []
priority = 'normal'
resolution = 'duplicate'
stage = 'resolved'
status = 'closed'
superseder = '9561'
type = 'behavior'
url = 'https://bugs.python.org/issue10051'
versions = ['Python 3.1', 'Python 3.2']

I was reported an issue where the user was unable to install my package [1] using python3.1. I was able to guess the issue comes from locale being set to POSIX and reproduce it.

It seems that the reasons are utf8-encoded characters in the installed script file. All the project files carry encoding declaration as per PEP-0263. They install fine with python2 but fail both with 3.1 and 3.2.

I think the build system should be locale-agnostic, and work correctly even if LC_ALL=C. If it's necessary to use text mode for scripts, I think a similar encoding guessing method should be used as in python itself.

I'm attaching a complete build log (for multiple versions of Python). Below I'm repasting the backtrace:

Traceback (most recent call last):
  File "setup.py", line 35, in <module>
    'Topic :: System :: Installation/Setup'
  File "/usr/lib64/python3.1/distutils/core.py", line 149, in setup
    dist.run_commands()
  File "/usr/lib64/python3.1/distutils/dist.py", line 919, in run_commands
    self.run_command(cmd)
  File "/usr/lib64/python3.1/distutils/dist.py", line 938, in run_command
    cmd_obj.run()
  File "/usr/lib64/python3.1/distutils/command/build.py", line 128, in run
    self.run_command(cmd_name)
  File "/usr/lib64/python3.1/distutils/cmd.py", line 315, in run_command
    self.distribution.run_command(command)
  File "/usr/lib64/python3.1/distutils/dist.py", line 938, in run_command
    cmd_obj.run()
  File "/usr/lib64/python3.1/distutils/command/build_scripts.py", line 51, in run
    self.copy_scripts()
  File "/usr/lib64/python3.1/distutils/command/build_scripts.py", line 83, in copy_scripts
    first_line = f.readline()
  File "/usr/lib64/python3.1/encodings/ascii.py", line 26, in decode
    return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc5 in position 127: ordinal not in range(128)

[1] http://github.com/mgorny/smart-live-rebuild

Thanks for the report. This is a duplicate, you can add yourself to the nosy list on the superseder bug to track status.

Navigation Menu

Repository navigation

distutils fail to install unicode-encoded files with POSIX locale #54260

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

distutils fail to install unicode-encoded files with POSIX locale #54260

Description

Activity

mgorny commented on Oct 8, 2010

merwok commented on Oct 18, 2010

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions