Skip to content

Python fails silently on bad locale #46426

Closed
Closed
Python fails silently on bad locale#46426
Labels
interpreter-core(Objects, Python, Grammar, and Parser dirs)type-bugAn unexpected behavior, bug, or error

Description

@motteneder
motteneder
mannequin
opened on Feb 24, 2008
BPO 2173
Nosy @facundobatista, @mdickinson, @vstinner
Files
  • full_make_output.txt: Full make output when LANG=UTF-8
  • make.log: make output
  • make.log: make output
  • issue2173.patch
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = <Date 2008-12-11.18:13:31.688>
created_at = <Date 2008-02-24.10:26:29.374>
labels = ['interpreter-core', 'type-bug']
title = 'Python fails silently on bad locale'
updated_at = <Date 2008-12-11.18:13:31.687>
user = 'https://bugs.python.org/motteneder'

    bugs.python.org fields:

    activity = <Date 2008-12-11.18:13:31.687>
actor = 'mark.dickinson'
assignee = 'none'
closed = True
closed_date = <Date 2008-12-11.18:13:31.688>
closer = 'mark.dickinson'
components = ['Interpreter Core']
creation = <Date 2008-02-24.10:26:29.374>
creator = 'motteneder'
dependencies = []
files = ['9558', '9560', '9561', '12284']
hgrepos = []
issue_num = 2173
keywords = ['patch']
message_count = 11.0
messages = ['62892', '63039', '63042', '63044', '75726', '75813', '77318', '77321', '77325', '77616', '77623']
nosy_count = 7.0
nosy_names = ['facundobatista', 'mark.dickinson', 'vstinner', 'motteneder', 'wangchun', 'nelis', 'stuartcw']
pr_nums = []
priority = 'critical'
resolution = 'fixed'
stage = None
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue2173'
versions = ['Python 3.0', 'Python 3.1']

    Activity

    1. motteneder commented on Feb 24, 2008

      @motteneder
      mottenedermannequin
      on Feb 24, 2008
      MannequinAuthor

      Python seems to fail silently when LANG enviroment variable is set to a
      bad value. In Mac OS X Leopard it is set too UTF-8 for instance which
      does not seem to be valid. Python fails building the modules during make
      and the generated python.exe is unable to output anything. Setting the
      LANG variable to C or some other valid value fixes the problem. But I
      suppose python should fail more verbose in case of a bad locale,or maybe
      it should fallback to a standard locale.

    2. motteneder
      added
      interpreter-core(Objects, Python, Grammar, and Parser dirs)
      type-bugAn unexpected behavior, bug, or error
      on Feb 24, 2008

    3. facundobatista commented on Feb 26, 2008

      @facundobatista
      facundobatista
      on Feb 26, 2008
      Member

      Same here, Ubuntu on x86 32b.

      Doing first a "make clean", ./configure runs ok.

      But then, in the "make":

      ...
      File "/home/facundo/devel/reps/python/py3k/Lib/locale.py", line 479,
      in setlocale
      return _setlocale(category, locale)
      locale.Error: unsupported locale setting
      make: *** [sharedmods] Error 1

      Full make output is attached.

      The problem seems to be in the setlocale module. I compiled everything
      with LANG in es_AR.UTF-8, but then executing Python again with a bad locale:

      facundo@virtub:~/devel/reps/python/py3k$ LANG=UTF-8 ./python 
Python 3.0a2+ (py3k:61084, Feb 26 2008, 10:12:19) 
[GCC 4.1.3 20070929 (prerelease) (Ubuntu 4.1.2-16ubuntu2)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import locale
>>> locale.getpreferredencoding()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/facundo/devel/reps/python/py3k/Lib/locale.py", line 515,
in getpreferredencoding
    setlocale(LC_CTYPE, "")
  File "/home/facundo/devel/reps/python/py3k/Lib/locale.py", line 479,
in setlocale
    return _setlocale(category, locale)
locale.Error: unsupported locale setting
>>>

      Regards,

    4. motteneder commented on Feb 26, 2008

      @motteneder
      mottenedermannequin
      on Feb 26, 2008
      MannequinAuthor

      Interestingly it fails with a different error message on Mac. The Ubuntu
      one is at least helpful.
      I get this during make(configure also runs fine) :
      gcc -u _PyMac_Error -o python.exe \
      Modules/python.o \
      libpython3.0.a -ldl
      make: *** [sharedmods] Error 1

      The created python.exe also behaves very strange:
Tinuviel:py3k mc$ ./python.exe 
Python 3.0a2+ (py3k:61071, Feb 26 2008, 14:50:42) 
[GCC 4.0.1 (Apple Inc. build 5465)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> print("hello")
>>> exit()
>>> exit()
>>> dfklasjd
>>>

      Locales:
      Tinuviel:py3k mc$ locale
      LANG=
      LC_COLLATE="C"
      LC_CTYPE="UTF-8"
      LC_MESSAGES="C"
      LC_MONETARY="C"
      LC_NUMERIC="C"
      LC_TIME="C"
      LC_ALL=

      Solution: set LANG to "C" or some other valid value.

    5. motteneder commented on Feb 26, 2008

      @motteneder
      mottenedermannequin
      on Feb 26, 2008
      MannequinAuthor
      On my system I get different results. The error message is everything
but useful.
My Output:
gcc  -u _PyMac_Error -o python.exe \
			Modules/python.o \
			libpython3.0.a -ldl      
make: *** [sharedmods] Error 1
Tinuviel:py3k mc$ ./python.exe 
Python 3.0a2+ (py3k:61071, Feb 26 2008, 14:50:42) 
[GCC 4.0.1 (Apple Inc. build 5465)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> print("hello")
>>> exit()
>>> exit()
>>> dfklasjd
>>>

      Locales:
      LANG=
      LC_COLLATE="C"
      LC_CTYPE="UTF-8"
      LC_MESSAGES="C"
      LC_MONETARY="C"
      LC_NUMERIC="C"
      LC_TIME="C"
      LC_ALL=

    6. birkenfeld
      changed the title [-]Pyton fails silently on bad locale[/-] [+]Python fails silently on bad locale[/+] on Feb 26, 2008

    7. vstinner commented on Nov 11, 2008

      @vstinner
      vstinner
      on Nov 11, 2008
      Member

      Can you retry with Python 3.0rc2?

    8. wangchun commented on Nov 13, 2008

      @wangchun
      wangchunmannequin
      on Nov 13, 2008
      Mannequin

      This issue remains unsolved in the latest python 3.0rc2+ subversion
      repository as of 2008-11-13.

    9. mdickinson commented on Dec 8, 2008

      @mdickinson
      mdickinson
      on Dec 8, 2008
      Member

      See bpo-4585, which appears to be the same problem.

    10. mdickinson commented on Dec 8, 2008

      @mdickinson
      mdickinson
      on Dec 8, 2008
      Member

      I think I've traced the 'no output' problem back to the device_encoding
      function in Modules/posixmodule.c. The problem occurs when this function
      is called to try to get the encoding for stdout.

      On my machine, if I do:

      LC_CTYPE="UTF-8" ./python.exe

      then the nl_langinfo call in device_encoding returns an empty string. If
      I do

      LC_CTYPE="bogus" ./python.exe

      then it returns "US-ASCII"

      and if I do

      LC_CTYPE="en_US.UTF-8" ./python.exe

      then it returns "UTF-8".

      In the first case (where the encoding is set to ""), any subsequent
      attempts to send anything to stdout result in a LookupError with the
      message "unknown encoding". But of course, since this exception message
      is itself sent to stdout (or possibly stderr?) the same problem occurs
      again and we just don't see any output.

      I don't yet know why this causes the module build to fail, or why exit()
      doesn't work.

      I think we should try to get this fixed before 3.0.1. Any help would be
      welcomed.

    11. mdickinson commented on Dec 8, 2008

      @mdickinson
      mdickinson
      on Dec 8, 2008
      Member

      Could someone please try the attached patch and see if it solves the
      problem.

      I'm not sure whether Facundo's problem is the same issue or not.

    12. stuartcw commented on Dec 11, 2008

      @stuartcw
      stuartcwmannequin
      on Dec 11, 2008
      Mannequin

      I'm running on Mac OS X. I applied the patch bpo-2173.patch and it
      solved the "make: *** [sharedmods] Error 1" problem. A cursory test of
      /usr/local/bin/python3.0 was successful.

    13. mdickinson commented on Dec 11, 2008

      @mdickinson
      mdickinson
      on Dec 11, 2008
      Member

      Thanks, Stuart!

      I've committed the fix in r67703 and r67704, so it should appear in 3.0.1.
      This should fix the silent build failure, and the zero-output interpreter.

      The behaviour is still not ideal: a bad LC_CTYPE setting causes
      sys.stdout and sys.stdin to have their encodings set to 'ascii'. 'UTF-8'
      would probably be a better guess, on OS X. But this is a lesser problem.
      To fix it, we should try to understand why nl_langinfo is returning an
      empty string in the first place. But maybe only Apple knows the answer to
      that one. :)

    14. mdickinson
      closed this as completedon Dec 11, 2008
    15. ezio-melotti
      transferred this issue fromon Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

    Metadata

    Metadata

    Assignees

    No one assigned

      Labels

      interpreter-core(Objects, Python, Grammar, and Parser dirs)type-bugAn unexpected behavior, bug, or error

      Projects

      No projects

        Milestone

        No milestone

        Relationships

        None yet

        Development

        No branches or pull requests

        Issue actions