Skip to content

[FreeBSD] test_c_locale_coercion doesn't support new C.UTF-8 locale of FreeBSD CURRENT #79471

Closed
Closed
[FreeBSD] test_c_locale_coercion doesn't support new C.UTF-8 locale of FreeBSD CURRENT#79471
Labels
3.7 (EOL)end of life3.8 (EOL)end of lifeinterpreter-core(Objects, Python, Grammar, and Parser dirs)

Description

@vstinner
vstinner
opened on Nov 21, 2018
BPO 35290
Nosy @ncoghlan, @vstinner, @koobs
PRs
  • bpo-35290: Add debug info to test_c_locale_coercion #10631
  • bpo-34523: Fix C locale coercion on FreeBSD CURRENT #10672
  • [3.7] bpo-34523: Fix C locale coercion on FreeBSD CURRENT (GH-10672) #10673
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = <Date 2018-11-23.12:38:21.849>
created_at = <Date 2018-11-21.10:49:52.346>
labels = ['interpreter-core', '3.7', '3.8']
title = "[FreeBSD] test_c_locale_coercion doesn't support new C.UTF-8 locale of FreeBSD CURRENT"
updated_at = <Date 2018-11-23.12:38:21.848>
user = 'https://github.com/vstinner'

    bugs.python.org fields:

    activity = <Date 2018-11-23.12:38:21.848>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2018-11-23.12:38:21.849>
closer = 'vstinner'
components = ['Interpreter Core']
creation = <Date 2018-11-21.10:49:52.346>
creator = 'vstinner'
dependencies = []
files = []
hgrepos = []
issue_num = 35290
keywords = ['patch']
message_count = 13.0
messages = ['330189', '330190', '330238', '330241', '330243', '330254', '330258', '330303', '330306', '330308', '330312', '330318', '330319']
nosy_count = 3.0
nosy_names = ['ncoghlan', 'vstinner', 'koobs']
pr_nums = ['10631', '10672', '10673']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue35290'
versions = ['Python 3.7', 'Python 3.8']

    Activity

    1. vstinner commented on Nov 21, 2018

      @vstinner
      vstinner
      on Nov 21, 2018
      MemberAuthor

      It seems like the commit 02e6bf7 (bpo-28604) broke test_c_locale_coercion on FreeBSD CURRENT Shared 3.x:

      https://buildbot.python.org/all/#/builders/168/builds/247

      0:19:33 load avg: 7.69 [401/419/2] test_c_locale_coercion failed -- running: test_decimal (40 sec 626 ms), test_lib2to3 (2 min 3 sec)
      test_external_target_locale_configuration (test.test_c_locale_coercion.LocaleConfigurationTests) ... ok
      test_LC_ALL_set_to_C (test.test_c_locale_coercion.LocaleCoercionTests) ... ok
      test_PYTHONCOERCECLOCALE_not_set (test.test_c_locale_coercion.LocaleCoercionTests) ... test_PYTHONCOERCECLOCALE_not_zero (test.test_c_locale_coercion.LocaleCoercionTests) ... test_PYTHONCOERCECLOCALE_set_to_warn (test.test_c_locale_coercion.LocaleCoercionTests) ... test_PYTHONCOERCECLOCALE_set_to_zero (test.test_c_locale_coercion.LocaleCoercionTests) ... ok

      ======================================================================
      FAIL: test_PYTHONCOERCECLOCALE_not_set (test.test_c_locale_coercion.LocaleCoercionTests) (default_locale=True, PYTHONCOERCECLOCALE=None)
      ----------------------------------------------------------------------

      Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/test_c_locale_coercion.py", line 341, in _check_c_locale_coercion
    self._check_child_encoding_details(base_var_dict,
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/test_c_locale_coercion.py", line 231, in _check_child_encoding_details
    self.assertEqual(encoding_details, expected_details)
AssertionError: {'fsencoding': 'ascii', 'stdin_info': 'utf-8:surrogateesc[130 chars]: ''} != {'fsencoding': 'utf-8', 'stdin_info': 'utf-8:surrogateesc[130 chars]: ''}
- {'fsencoding': 'ascii',
?                 ^^^^^

      + {'fsencoding': 'utf-8',
      ? ^^^^^

      'lang': '',
      'lc_all': '',
      'lc_ctype': 'c.utf-8',
      'stderr_info': 'utf-8:backslashreplace',
      'stdin_info': 'utf-8:surrogateescape',
      'stdout_info': 'utf-8:surrogateescape'}

      ======================================================================
      FAIL: test_PYTHONCOERCECLOCALE_not_set (test.test_c_locale_coercion.LocaleCoercionTests) (env_var='LANG', nominal_locale='C', PYTHONCOERCECLOCALE=None)
      ----------------------------------------------------------------------

      Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/test_c_locale_coercion.py", line 356, in _check_c_locale_coercion
    self._check_child_encoding_details(var_dict,
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/test_c_locale_coercion.py", line 231, in _check_child_encoding_details
    self.assertEqual(encoding_details, expected_details)
AssertionError: {'fsencoding': 'ascii', 'stdin_info': 'utf-8:surrogateesc[131 chars]: ''} != {'fsencoding': 'utf-8', 'stdin_info': 'utf-8:surrogateesc[131 chars]: ''}
- {'fsencoding': 'ascii',
?                 ^^^^^

      + {'fsencoding': 'utf-8',
      ? ^^^^^

      'lang': 'c',
      'lc_all': '',
      'lc_ctype': 'c.utf-8',
      'stderr_info': 'utf-8:backslashreplace',
      'stdin_info': 'utf-8:surrogateescape',
      'stdout_info': 'utf-8:surrogateescape'}

      ======================================================================
      FAIL: test_PYTHONCOERCECLOCALE_not_set (test.test_c_locale_coercion.LocaleCoercionTests) (env_var='LC_CTYPE', nominal_locale='C', PYTHONCOERCECLOCALE=None)
      ----------------------------------------------------------------------

      Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/test_c_locale_coercion.py", line 356, in _check_c_locale_coercion
    self._check_child_encoding_details(var_dict,
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/test_c_locale_coercion.py", line 231, in _check_child_encoding_details
    self.assertEqual(encoding_details, expected_details)
AssertionError: {'fsencoding': 'ascii', 'stdin_info': 'utf-8:surrogateesc[130 chars]: ''} != {'fsencoding': 'utf-8', 'stdin_info': 'utf-8:surrogateesc[130 chars]: ''}
- {'fsencoding': 'ascii',
?                 ^^^^^

      + {'fsencoding': 'utf-8',
      ? ^^^^^

      'lang': '',
      'lc_all': '',
      'lc_ctype': 'c.utf-8',
      'stderr_info': 'utf-8:backslashreplace',
      'stdin_info': 'utf-8:surrogateescape',
      'stdout_info': 'utf-8:surrogateescape'}

      ======================================================================
      FAIL: test_PYTHONCOERCECLOCALE_not_set (test.test_c_locale_coercion.LocaleCoercionTests) (env_var='LANG', nominal_locale='invalid.ascii', PYTHONCOERCECLOCALE=None)
      ----------------------------------------------------------------------

      Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/test_c_locale_coercion.py", line 356, in _check_c_locale_coercion
    self._check_child_encoding_details(var_dict,
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/test_c_locale_coercion.py", line 231, in _check_child_encoding_details
    self.assertEqual(encoding_details, expected_details)
AssertionError: {'fsencoding': 'ascii', 'stdin_info': 'utf-8:surrogateesc[143 chars]: ''} != {'fsencoding': 'utf-8', 'stdin_info': 'utf-8:surrogateesc[143 chars]: ''}
- {'fsencoding': 'ascii',
?                 ^^^^^

      + {'fsencoding': 'utf-8',
      ? ^^^^^

      'lang': 'invalid.ascii',
      'lc_all': '',
      'lc_ctype': 'c.utf-8',
      'stderr_info': 'utf-8:backslashreplace',
      'stdin_info': 'utf-8:surrogateescape',
      'stdout_info': 'utf-8:surrogateescape'}

      ======================================================================
      FAIL: test_PYTHONCOERCECLOCALE_not_set (test.test_c_locale_coercion.LocaleCoercionTests) (env_var='LC_CTYPE', nominal_locale='invalid.ascii', PYTHONCOERCECLOCALE=None)
      ----------------------------------------------------------------------

      Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/test_c_locale_coercion.py", line 356, in _check_c_locale_coercion
    self._check_child_encoding_details(var_dict,
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/test_c_locale_coercion.py", line 231, in _check_child_encoding_details
    self.assertEqual(encoding_details, expected_details)
AssertionError: {'fsencoding': 'ascii', 'stdin_info': 'utf-8:surrogateesc[130 chars]: ''} != {'fsencoding': 'utf-8', 'stdin_info': 'utf-8:surrogateesc[130 chars]: ''}
- {'fsencoding': 'ascii',
?                 ^^^^^

      + {'fsencoding': 'utf-8',
      ? ^^^^^

      'lang': '',
      'lc_all': '',
      'lc_ctype': 'c.utf-8',
      'stderr_info': 'utf-8:backslashreplace',
      'stdin_info': 'utf-8:surrogateescape',
      'stdout_info': 'utf-8:surrogateescape'}

      ======================================================================
      FAIL: test_PYTHONCOERCECLOCALE_not_zero (test.test_c_locale_coercion.LocaleCoercionTests) (default_locale=True, PYTHONCOERCECLOCALE='')
      ----------------------------------------------------------------------

      Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/test_c_locale_coercion.py", line 341, in _check_c_locale_coercion
    self._check_child_encoding_details(base_var_dict,
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/test_c_locale_coercion.py", line 231, in _check_child_encoding_details
    self.assertEqual(encoding_details, expected_details)
AssertionError: {'fsencoding': 'ascii', 'stdin_info': 'utf-8:surrogateesc[130 chars]: ''} != {'fsencoding': 'utf-8', 'stdin_info': 'utf-8:surrogateesc[130 chars]: ''}
- {'fsencoding': 'ascii',
?                 ^^^^^

      + {'fsencoding': 'utf-8',
      ? ^^^^^

      'lang': '',
      'lc_all': '',
      'lc_ctype': 'c.utf-8',
      'stderr_info': 'utf-8:backslashreplace',
      'stdin_info': 'utf-8:surrogateescape',
      'stdout_info': 'utf-8:surrogateescape'}

      ======================================================================
      FAIL: test_PYTHONCOERCECLOCALE_not_zero (test.test_c_locale_coercion.LocaleCoercionTests) (env_var='LANG', nominal_locale='C', PYTHONCOERCECLOCALE='')
      ----------------------------------------------------------------------

      Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/test_c_locale_coercion.py", line 356, in _check_c_locale_coercion
    self._check_child_encoding_details(var_dict,
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/test_c_locale_coercion.py", line 231, in _check_child_encoding_details
    self.assertEqual(encoding_details, expected_details)
AssertionError: {'fsencoding': 'ascii', 'stdin_info': 'utf-8:surrogateesc[131 chars]: ''} != {'fsencoding': 'utf-8', 'stdin_info': 'utf-8:surrogateesc[131 chars]: ''}
- {'fsencoding': 'ascii',
?                 ^^^^^

      + {'fsencoding': 'utf-8',
      ? ^^^^^

      'lang': 'c',
      'lc_all': '',
      'lc_ctype': 'c.utf-8',
      'stderr_info': 'utf-8:backslashreplace',
      'stdin_info': 'utf-8:surrogateescape',
      'stdout_info': 'utf-8:surrogateescape'}

      ======================================================================
      FAIL: test_PYTHONCOERCECLOCALE_not_zero (test.test_c_locale_coercion.LocaleCoercionTests) (env_var='LC_CTYPE', nominal_locale='C', PYTHONCOERCECLOCALE='')
      ----------------------------------------------------------------------

      Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/test_c_locale_coercion.py", line 356, in _check_c_locale_coercion
    self._check_child_encoding_details(var_dict,
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/test_c_locale_coercion.py", line 231, in _check_child_encoding_details
    self.assertEqual(encoding_details, expected_details)
AssertionError: {'fsencoding': 'ascii', 'stdin_info': 'utf-8:surrogateesc[130 chars]: ''} != {'fsencoding': 'utf-8', 'stdin_info': 'utf-8:surrogateesc[130 chars]: ''}
- {'fsencoding': 'ascii',
?                 ^^^^^

      + {'fsencoding': 'utf-8',
      ? ^^^^^

      'lang': '',
      'lc_all': '',
      'lc_ctype': 'c.utf-8',
      'stderr_info': 'utf-8:backslashreplace',
      'stdin_info': 'utf-8:surrogateescape',
      'stdout_info': 'utf-8:surrogateescape'}

      ======================================================================
      FAIL: test_PYTHONCOERCECLOCALE_not_zero (test.test_c_locale_coercion.LocaleCoercionTests) (env_var='LANG', nominal_locale='invalid.ascii', PYTHONCOERCECLOCALE='')
      ----------------------------------------------------------------------

      Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/test_c_locale_coercion.py", line 356, in _check_c_locale_coercion
    self._check_child_encoding_details(var_dict,
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/test_c_locale_coercion.py", line 231, in _check_child_encoding_details
    self.assertEqual(encoding_details, expected_details)
AssertionError: {'fsencoding': 'ascii', 'stdin_info': 'utf-8:surrogateesc[143 chars]: ''} != {'fsencoding': 'utf-8', 'stdin_info': 'utf-8:surrogateesc[143 chars]: ''}
- {'fsencoding': 'ascii',
?                 ^^^^^

      + {'fsencoding': 'utf-8',
      ? ^^^^^

      'lang': 'invalid.ascii',
      'lc_all': '',
      'lc_ctype': 'c.utf-8',
      'stderr_info': 'utf-8:backslashreplace',
      'stdin_info': 'utf-8:surrogateescape',
      'stdout_info': 'utf-8:surrogateescape'}

      ======================================================================
      FAIL: test_PYTHONCOERCECLOCALE_not_zero (test.test_c_locale_coercion.LocaleCoercionTests) (env_var='LC_CTYPE', nominal_locale='invalid.ascii', PYTHONCOERCECLOCALE='')
      ----------------------------------------------------------------------

      Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/test_c_locale_coercion.py", line 356, in _check_c_locale_coercion
    self._check_child_encoding_details(var_dict,
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/test_c_locale_coercion.py", line 231, in _check_child_encoding_details
    self.assertEqual(encoding_details, expected_details)
AssertionError: {'fsencoding': 'ascii', 'stdin_info': 'utf-8:surrogateesc[130 chars]: ''} != {'fsencoding': 'utf-8', 'stdin_info': 'utf-8:surrogateesc[130 chars]: ''}
- {'fsencoding': 'ascii',
?                 ^^^^^

      + {'fsencoding': 'utf-8',
      ? ^^^^^

      'lang': '',
      'lc_all': '',
      'lc_ctype': 'c.utf-8',
      'stderr_info': 'utf-8:backslashreplace',
      'stdin_info': 'utf-8:surrogateescape',
      'stdout_info': 'utf-8:surrogateescape'}

      ======================================================================
      FAIL: test_PYTHONCOERCECLOCALE_not_zero (test.test_c_locale_coercion.LocaleCoercionTests) (default_locale=True, PYTHONCOERCECLOCALE='1')
      ----------------------------------------------------------------------

      Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/test_c_locale_coercion.py", line 341, in _check_c_locale_coercion
    self._check_child_encoding_details(base_var_dict,
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/test_c_locale_coercion.py", line 231, in _check_child_encoding_details
    self.assertEqual(encoding_details, expected_details)
AssertionError: {'fsencoding': 'ascii', 'stdin_info': 'utf-8:surrogateesc[130 chars]: ''} != {'fsencoding': 'utf-8', 'stdin_info': 'utf-8:surrogateesc[130 chars]: ''}
- {'fsencoding': 'ascii',
?                 ^^^^^

      + {'fsencoding': 'utf-8',
      ? ^^^^^

      'lang': '',
      'lc_all': '',
      'lc_ctype': 'c.utf-8',
      'stderr_info': 'utf-8:backslashreplace',
      'stdin_info': 'utf-8:surrogateescape',
      'stdout_info': 'utf-8:surrogateescape'}

      ======================================================================
      FAIL: test_PYTHONCOERCECLOCALE_not_zero (test.test_c_locale_coercion.LocaleCoercionTests) (env_var='LANG', nominal_locale='C', PYTHONCOERCECLOCALE='1')
      ----------------------------------------------------------------------

      Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/test_c_locale_coercion.py", line 356, in _check_c_locale_coercion
    self._check_child_encoding_details(var_dict,
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/test_c_locale_coercion.py", line 231, in _check_child_encoding_details
    self.assertEqual(encoding_details, expected_details)
AssertionError: {'fsencoding': 'ascii', 'stdin_info': 'utf-8:surrogateesc[131 chars]: ''} != {'fsencoding': 'utf-8', 'stdin_info': 'utf-8:surrogateesc[131 chars]: ''}
- {'fsencoding': 'ascii',
?                 ^^^^^

      + {'fsencoding': 'utf-8',
      ? ^^^^^

      'lang': 'c',
      'lc_all': '',
      'lc_ctype': 'c.utf-8',
      'stderr_info': 'utf-8:backslashreplace',
      'stdin_info': 'utf-8:surrogateescape',
      'stdout_info': 'utf-8:surrogateescape'}

      ======================================================================
      FAIL: test_PYTHONCOERCECLOCALE_not_zero (test.test_c_locale_coercion.LocaleCoercionTests) (env_var='LC_CTYPE', nominal_locale='C', PYTHONCOERCECLOCALE='1')
      ----------------------------------------------------------------------

      Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/test_c_locale_coercion.py", line 356, in _check_c_locale_coercion
    self._check_child_encoding_details(var_dict,
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/test_c_locale_coercion.py", line 231, in _check_child_encoding_details
    self.assertEqual(encoding_details, expected_details)
AssertionError: {'fsencoding': 'ascii', 'stdin_info': 'utf-8:surrogateesc[130 chars]: ''} != {'fsencoding': 'utf-8', 'stdin_info': 'utf-8:surrogateesc[130 chars]: ''}
- {'fsencoding': 'ascii',
?                 ^^^^^

      + {'fsencoding': 'utf-8',
      ? ^^^^^

      'lang': '',
      'lc_all': '',
      'lc_ctype': 'c.utf-8',
      'stderr_info': 'utf-8:backslashreplace',
      'stdin_info': 'utf-8:surrogateescape',
      'stdout_info': 'utf-8:surrogateescape'}

      ======================================================================
      FAIL: test_PYTHONCOERCECLOCALE_not_zero (test.test_c_locale_coercion.LocaleCoercionTests) (env_var='LANG', nominal_locale='invalid.ascii', PYTHONCOERCECLOCALE='1')
      ----------------------------------------------------------------------

      Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/test_c_locale_coercion.py", line 356, in _check_c_locale_coercion
    self._check_child_encoding_details(var_dict,
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/test_c_locale_coercion.py", line 231, in _check_child_encoding_details
    self.assertEqual(encoding_details, expected_details)
AssertionError: {'fsencoding': 'ascii', 'stdin_info': 'utf-8:surrogateesc[143 chars]: ''} != {'fsencoding': 'utf-8', 'stdin_info': 'utf-8:surrogateesc[143 chars]: ''}
- {'fsencoding': 'ascii',
?                 ^^^^^

      + {'fsencoding': 'utf-8',
      ? ^^^^^

      'lang': 'invalid.ascii',
      'lc_all': '',
      'lc_ctype': 'c.utf-8',
      'stderr_info': 'utf-8:backslashreplace',
      'stdin_info': 'utf-8:surrogateescape',
      'stdout_info': 'utf-8:surrogateescape'}

      ======================================================================
      FAIL: test_PYTHONCOERCECLOCALE_not_zero (test.test_c_locale_coercion.LocaleCoercionTests) (env_var='LC_CTYPE', nominal_locale='invalid.ascii', PYTHONCOERCECLOCALE='1')
      ----------------------------------------------------------------------

      Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/test_c_locale_coercion.py", line 356, in _check_c_locale_coercion
    self._check_child_encoding_details(var_dict,
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/test_c_locale_coercion.py", line 231, in _check_child_encoding_details
    self.assertEqual(encoding_details, expected_details)
AssertionError: {'fsencoding': 'ascii', 'stdin_info': 'utf-8:surrogateesc[130 chars]: ''} != {'fsencoding': 'utf-8', 'stdin_info': 'utf-8:surrogateesc[130 chars]: ''}
- {'fsencoding': 'ascii',
?                 ^^^^^

      + {'fsencoding': 'utf-8',
      ? ^^^^^

      'lang': '',
      'lc_all': '',
      'lc_ctype': 'c.utf-8',
      'stderr_info': 'utf-8:backslashreplace',
      'stdin_info': 'utf-8:surrogateescape',
      'stdout_info': 'utf-8:surrogateescape'}

      ======================================================================
      FAIL: test_PYTHONCOERCECLOCALE_not_zero (test.test_c_locale_coercion.LocaleCoercionTests) (default_locale=True, PYTHONCOERCECLOCALE='true')
      ----------------------------------------------------------------------

      Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/test_c_locale_coercion.py", line 341, in _check_c_locale_coercion
    self._check_child_encoding_details(base_var_dict,
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/test_c_locale_coercion.py", line 231, in _check_child_encoding_details
    self.assertEqual(encoding_details, expected_details)
AssertionError: {'fsencoding': 'ascii', 'stdin_info': 'utf-8:surrogateesc[130 chars]: ''} != {'fsencoding': 'utf-8', 'stdin_info': 'utf-8:surrogateesc[130 chars]: ''}
- {'fsencoding': 'ascii',
?                 ^^^^^

      + {'fsencoding': 'utf-8',
      ? ^^^^^

      'lang': '',
      'lc_all': '',
      'lc_ctype': 'c.utf-8',
      'stderr_info': 'utf-8:backslashreplace',
      'stdin_info': 'utf-8:surrogateescape',
      'stdout_info': 'utf-8:surrogateescape'}

      ======================================================================
      FAIL: test_PYTHONCOERCECLOCALE_not_zero (test.test_c_locale_coercion.LocaleCoercionTests) (env_var='LANG', nominal_locale='C', PYTHONCOERCECLOCALE='true')
      ----------------------------------------------------------------------

      Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/test_c_locale_coercion.py", line 356, in _check_c_locale_coercion
    self._check_child_encoding_details(var_dict,
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/test_c_locale_coercion.py", line 231, in _check_child_encoding_details
    self.assertEqual(encoding_details, expected_details)
AssertionError: {'fsencoding': 'ascii', 'stdin_info': 'utf-8:surrogateesc[131 chars]: ''} != {'fsencoding': 'utf-8', 'stdin_info': 'utf-8:surrogateesc[131 chars]: ''}
- {'fsencoding': 'ascii',
?                 ^^^^^

      + {'fsencoding': 'utf-8',
      ? ^^^^^

      'lang': 'c',
      'lc_all': '',
      'lc_ctype': 'c.utf-8',
      'stderr_info': 'utf-8:backslashreplace',
      'stdin_info': 'utf-8:surrogateescape',
      'stdout_info': 'utf-8:surrogateescape'}

      ======================================================================
      FAIL: test_PYTHONCOERCECLOCALE_not_zero (test.test_c_locale_coercion.LocaleCoercionTests) (env_var='LC_CTYPE', nominal_locale='C', PYTHONCOERCECLOCALE='true')
      ----------------------------------------------------------------------

      Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/test_c_locale_coercion.py", line 356, in _check_c_locale_coercion
    self._check_child_encoding_details(var_dict,
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/test_c_locale_coercion.py", line 231, in _check_child_encoding_details
    self.assertEqual(encoding_details, expected_details)
AssertionError: {'fsencoding': 'ascii', 'stdin_info': 'utf-8:surrogateesc[130 chars]: ''} != {'fsencoding': 'utf-8', 'stdin_info': 'utf-8:surrogateesc[130 chars]: ''}
- {'fsencoding': 'ascii',
?                 ^^^^^

      + {'fsencoding': 'utf-8',
      ? ^^^^^

      'lang': '',
      'lc_all': '',
      'lc_ctype': 'c.utf-8',
      'stderr_info': 'utf-8:backslashreplace',
      'stdin_info': 'utf-8:surrogateescape',
      'stdout_info': 'utf-8:surrogateescape'}

      ======================================================================
      FAIL: test_PYTHONCOERCECLOCALE_not_zero (test.test_c_locale_coercion.LocaleCoercionTests) (env_var='LANG', nominal_locale='invalid.ascii', PYTHONCOERCECLOCALE='true')
      ----------------------------------------------------------------------

      Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/test_c_locale_coercion.py", line 356, in _check_c_locale_coercion
    self._check_child_encoding_details(var_dict,
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/test_c_locale_coercion.py", line 231, in _check_child_encoding_details
    self.assertEqual(encoding_details, expected_details)
AssertionError: {'fsencoding': 'ascii', 'stdin_info': 'utf-8:surrogateesc[143 chars]: ''} != {'fsencoding': 'utf-8', 'stdin_info': 'utf-8:surrogateesc[143 chars]: ''}
- {'fsencoding': 'ascii',
?                 ^^^^^

      + {'fsencoding': 'utf-8',
      ? ^^^^^

      'lang': 'invalid.ascii',
      'lc_all': '',
      'lc_ctype': 'c.utf-8',
      'stderr_info': 'utf-8:backslashreplace',
      'stdin_info': 'utf-8:surrogateescape',
      'stdout_info': 'utf-8:surrogateescape'}

      ======================================================================
      FAIL: test_PYTHONCOERCECLOCALE_not_zero (test.test_c_locale_coercion.LocaleCoercionTests) (env_var='LC_CTYPE', nominal_locale='invalid.ascii', PYTHONCOERCECLOCALE='true')
      ----------------------------------------------------------------------

      Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/test_c_locale_coercion.py", line 356, in _check_c_locale_coercion
    self._check_child_encoding_details(var_dict,
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/test_c_locale_coercion.py", line 231, in _check_child_encoding_details
    self.assertEqual(encoding_details, expected_details)
AssertionError: {'fsencoding': 'ascii', 'stdin_info': 'utf-8:surrogateesc[130 chars]: ''} != {'fsencoding': 'utf-8', 'stdin_info': 'utf-8:surrogateesc[130 chars]: ''}
- {'fsencoding': 'ascii',
?                 ^^^^^

      + {'fsencoding': 'utf-8',
      ? ^^^^^

      'lang': '',
      'lc_all': '',
      'lc_ctype': 'c.utf-8',
      'stderr_info': 'utf-8:backslashreplace',
      'stdin_info': 'utf-8:surrogateescape',
      'stdout_info': 'utf-8:surrogateescape'}

      ======================================================================
      FAIL: test_PYTHONCOERCECLOCALE_not_zero (test.test_c_locale_coercion.LocaleCoercionTests) (default_locale=True, PYTHONCOERCECLOCALE='false')
      ----------------------------------------------------------------------

      Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/test_c_locale_coercion.py", line 341, in _check_c_locale_coercion
    self._check_child_encoding_details(base_var_dict,
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/test_c_locale_coercion.py", line 231, in _check_child_encoding_details
    self.assertEqual(encoding_details, expected_details)
AssertionError: {'fsencoding': 'ascii', 'stdin_info': 'utf-8:surrogateesc[130 chars]: ''} != {'fsencoding': 'utf-8', 'stdin_info': 'utf-8:surrogateesc[130 chars]: ''}
- {'fsencoding': 'ascii',
?                 ^^^^^

      + {'fsencoding': 'utf-8',
      ? ^^^^^

      'lang': '',
      'lc_all': '',
      'lc_ctype': 'c.utf-8',
      'stderr_info': 'utf-8:backslashreplace',
      'stdin_info': 'utf-8:surrogateescape',
      'stdout_info': 'utf-8:surrogateescape'}

      ======================================================================
      FAIL: test_PYTHONCOERCECLOCALE_not_zero (test.test_c_locale_coercion.LocaleCoercionTests) (env_var='LANG', nominal_locale='C', PYTHONCOERCECLOCALE='false')
      ----------------------------------------------------------------------

      Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/test_c_locale_coercion.py", line 356, in _check_c_locale_coercion
    self._check_child_encoding_details(var_dict,
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/test_c_locale_coercion.py", line 231, in _check_child_encoding_details
    self.assertEqual(encoding_details, expected_details)
AssertionError: {'fsencoding': 'ascii', 'stdin_info': 'utf-8:surrogateesc[131 chars]: ''} != {'fsencoding': 'utf-8', 'stdin_info': 'utf-8:surrogateesc[131 chars]: ''}
- {'fsencoding': 'ascii',
?                 ^^^^^

      + {'fsencoding': 'utf-8',
      ? ^^^^^

      'lang': 'c',
      'lc_all': '',
      'lc_ctype': 'c.utf-8',
      'stderr_info': 'utf-8:backslashreplace',
      'stdin_info': 'utf-8:surrogateescape',
      'stdout_info': 'utf-8:surrogateescape'}

      ======================================================================
      FAIL: test_PYTHONCOERCECLOCALE_not_zero (test.test_c_locale_coercion.LocaleCoercionTests) (env_var='LC_CTYPE', nominal_locale='C', PYTHONCOERCECLOCALE='false')
      ----------------------------------------------------------------------

      Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/test_c_locale_coercion.py", line 356, in _check_c_locale_coercion
    self._check_child_encoding_details(var_dict,
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/test_c_locale_coercion.py", line 231, in _check_child_encoding_details
    self.assertEqual(encoding_details, expected_details)
AssertionError: {'fsencoding': 'ascii', 'stdin_info': 'utf-8:surrogateesc[130 chars]: ''} != {'fsencoding': 'utf-8', 'stdin_info': 'utf-8:surrogateesc[130 chars]: ''}
- {'fsencoding': 'ascii',
?                 ^^^^^

      + {'fsencoding': 'utf-8',
      ? ^^^^^

      'lang': '',
      'lc_all': '',
      'lc_ctype': 'c.utf-8',
      'stderr_info': 'utf-8:backslashreplace',
      'stdin_info': 'utf-8:surrogateescape',
      'stdout_info': 'utf-8:surrogateescape'}

      ======================================================================
      FAIL: test_PYTHONCOERCECLOCALE_not_zero (test.test_c_locale_coercion.LocaleCoercionTests) (env_var='LANG', nominal_locale='invalid.ascii', PYTHONCOERCECLOCALE='false')
      ----------------------------------------------------------------------

      Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/test_c_locale_coercion.py", line 356, in _check_c_locale_coercion
    self._check_child_encoding_details(var_dict,
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/test_c_locale_coercion.py", line 231, in _check_child_encoding_details
    self.assertEqual(encoding_details, expected_details)
AssertionError: {'fsencoding': 'ascii', 'stdin_info': 'utf-8:surrogateesc[143 chars]: ''} != {'fsencoding': 'utf-8', 'stdin_info': 'utf-8:surrogateesc[143 chars]: ''}
- {'fsencoding': 'ascii',
?                 ^^^^^

      + {'fsencoding': 'utf-8',
      ? ^^^^^

      'lang': 'invalid.ascii',
      'lc_all': '',
      'lc_ctype': 'c.utf-8',
      'stderr_info': 'utf-8:backslashreplace',
      'stdin_info': 'utf-8:surrogateescape',
      'stdout_info': 'utf-8:surrogateescape'}

      ======================================================================
      FAIL: test_PYTHONCOERCECLOCALE_not_zero (test.test_c_locale_coercion.LocaleCoercionTests) (env_var='LC_CTYPE', nominal_locale='invalid.ascii', PYTHONCOERCECLOCALE='false')
      ----------------------------------------------------------------------

      Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/test_c_locale_coercion.py", line 356, in _check_c_locale_coercion
    self._check_child_encoding_details(var_dict,
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/test_c_locale_coercion.py", line 231, in _check_child_encoding_details
    self.assertEqual(encoding_details, expected_details)
AssertionError: {'fsencoding': 'ascii', 'stdin_info': 'utf-8:surrogateesc[130 chars]: ''} != {'fsencoding': 'utf-8', 'stdin_info': 'utf-8:surrogateesc[130 chars]: ''}
- {'fsencoding': 'ascii',
?                 ^^^^^

      + {'fsencoding': 'utf-8',
      ? ^^^^^

      'lang': '',
      'lc_all': '',
      'lc_ctype': 'c.utf-8',
      'stderr_info': 'utf-8:backslashreplace',
      'stdin_info': 'utf-8:surrogateescape',
      'stdout_info': 'utf-8:surrogateescape'}

      ======================================================================
      FAIL: test_PYTHONCOERCECLOCALE_set_to_warn (test.test_c_locale_coercion.LocaleCoercionTests) (default_locale=True, PYTHONCOERCECLOCALE='warn')
      ----------------------------------------------------------------------

      Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/test_c_locale_coercion.py", line 341, in _check_c_locale_coercion
    self._check_child_encoding_details(base_var_dict,
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/test_c_locale_coercion.py", line 231, in _check_child_encoding_details
    self.assertEqual(encoding_details, expected_details)
AssertionError: {'fsencoding': 'ascii', 'stdin_info': 'utf-8:surrogateesc[130 chars]: ''} != {'fsencoding': 'utf-8', 'stdin_info': 'utf-8:surrogateesc[130 chars]: ''}
- {'fsencoding': 'ascii',
?                 ^^^^^

      + {'fsencoding': 'utf-8',
      ? ^^^^^

      'lang': '',
      'lc_all': '',
      'lc_ctype': 'c.utf-8',
      'stderr_info': 'utf-8:backslashreplace',
      'stdin_info': 'utf-8:surrogateescape',
      'stdout_info': 'utf-8:surrogateescape'}

      ======================================================================
      FAIL: test_PYTHONCOERCECLOCALE_set_to_warn (test.test_c_locale_coercion.LocaleCoercionTests) (env_var='LANG', nominal_locale='C', PYTHONCOERCECLOCALE='warn')
      ----------------------------------------------------------------------

      Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/test_c_locale_coercion.py", line 356, in _check_c_locale_coercion
    self._check_child_encoding_details(var_dict,
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/test_c_locale_coercion.py", line 231, in _check_child_encoding_details
    self.assertEqual(encoding_details, expected_details)
AssertionError: {'fsencoding': 'ascii', 'stdin_info': 'utf-8:surrogateesc[131 chars]: ''} != {'fsencoding': 'utf-8', 'stdin_info': 'utf-8:surrogateesc[131 chars]: ''}
- {'fsencoding': 'ascii',
?                 ^^^^^

      + {'fsencoding': 'utf-8',
      ? ^^^^^

      'lang': 'c',
      'lc_all': '',
      'lc_ctype': 'c.utf-8',
      'stderr_info': 'utf-8:backslashreplace',
      'stdin_info': 'utf-8:surrogateescape',
      'stdout_info': 'utf-8:surrogateescape'}

      ======================================================================
      FAIL: test_PYTHONCOERCECLOCALE_set_to_warn (test.test_c_locale_coercion.LocaleCoercionTests) (env_var='LC_CTYPE', nominal_locale='C', PYTHONCOERCECLOCALE='warn')
      ----------------------------------------------------------------------

      Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/test_c_locale_coercion.py", line 356, in _check_c_locale_coercion
    self._check_child_encoding_details(var_dict,
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/test_c_locale_coercion.py", line 231, in _check_child_encoding_details
    self.assertEqual(encoding_details, expected_details)
AssertionError: {'fsencoding': 'ascii', 'stdin_info': 'utf-8:surrogateesc[130 chars]: ''} != {'fsencoding': 'utf-8', 'stdin_info': 'utf-8:surrogateesc[130 chars]: ''}
- {'fsencoding': 'ascii',
?                 ^^^^^

      + {'fsencoding': 'utf-8',
      ? ^^^^^

      'lang': '',
      'lc_all': '',
      'lc_ctype': 'c.utf-8',
      'stderr_info': 'utf-8:backslashreplace',
      'stdin_info': 'utf-8:surrogateescape',
      'stdout_info': 'utf-8:surrogateescape'}

      ======================================================================
      FAIL: test_PYTHONCOERCECLOCALE_set_to_warn (test.test_c_locale_coercion.LocaleCoercionTests) (env_var='LANG', nominal_locale='invalid.ascii', PYTHONCOERCECLOCALE='warn')
      ----------------------------------------------------------------------

      Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/test_c_locale_coercion.py", line 356, in _check_c_locale_coercion
    self._check_child_encoding_details(var_dict,
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/test_c_locale_coercion.py", line 231, in _check_child_encoding_details
    self.assertEqual(encoding_details, expected_details)
AssertionError: {'fsencoding': 'ascii', 'stdin_info': 'utf-8:surrogateesc[143 chars]: ''} != {'fsencoding': 'utf-8', 'stdin_info': 'utf-8:surrogateesc[143 chars]: ''}
- {'fsencoding': 'ascii',
?                 ^^^^^

      + {'fsencoding': 'utf-8',
      ? ^^^^^

      'lang': 'invalid.ascii',
      'lc_all': '',
      'lc_ctype': 'c.utf-8',
      'stderr_info': 'utf-8:backslashreplace',
      'stdin_info': 'utf-8:surrogateescape',
      'stdout_info': 'utf-8:surrogateescape'}

      ======================================================================
      FAIL: test_PYTHONCOERCECLOCALE_set_to_warn (test.test_c_locale_coercion.LocaleCoercionTests) (env_var='LC_CTYPE', nominal_locale='invalid.ascii', PYTHONCOERCECLOCALE='warn')
      ----------------------------------------------------------------------

      Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/test_c_locale_coercion.py", line 356, in _check_c_locale_coercion
    self._check_child_encoding_details(var_dict,
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/test_c_locale_coercion.py", line 231, in _check_child_encoding_details
    self.assertEqual(encoding_details, expected_details)
AssertionError: {'fsencoding': 'ascii', 'stdin_info': 'utf-8:surrogateesc[130 chars]: ''} != {'fsencoding': 'utf-8', 'stdin_info': 'utf-8:surrogateesc[130 chars]: ''}
- {'fsencoding': 'ascii',
?                 ^^^^^

      + {'fsencoding': 'utf-8',
      ? ^^^^^

      'lang': '',
      'lc_all': '',
      'lc_ctype': 'c.utf-8',
      'stderr_info': 'utf-8:backslashreplace',
      'stdin_info': 'utf-8:surrogateescape',
      'stdout_info': 'utf-8:surrogateescape'}

      ----------------------------------------------------------------------

      Ran 6 tests in 6.205s

      FAILED (failures=30)
      test test_c_locale_coercion failed

    2. vstinner
      added
      3.8 (EOL)end of life
      interpreter-core(Objects, Python, Grammar, and Parser dirs)
      on Nov 21, 2018

    3. vstinner commented on Nov 21, 2018

      @vstinner
      vstinner
      on Nov 21, 2018
      MemberAuthor

      New changeset 7c2d570 by Victor Stinner in branch 'master':
      bpo-35290: Add debug info to test_c_locale_coercion (GH-10631)
      7c2d570

    4. vstinner commented on Nov 22, 2018

      @vstinner
      vstinner
      on Nov 22, 2018
      MemberAuthor

      Result of the additional debug info, on the FreeBSD CURRENT buildbot:

      AVAILABLE_TARGETS = ['C.UTF-8']
EXPECTED_C_LOCALE_EQUIVALENTS = ['C', 'invalid.ascii']
EXPECTED_C_LOCALE_STREAM_ENCODING = 'ascii'
EXPECTED_C_LOCALE_FS_ENCODING = 'ascii'
EXPECT_COERCION_IN_DEFAULT_LOCALE = True
_C_UTF8_LOCALES = ('C.UTF-8', 'C.utf8', 'UTF-8')
_check_nl_langinfo_CODESET = True

    5. vstinner commented on Nov 22, 2018

      @vstinner
      vstinner
      on Nov 22, 2018
      MemberAuthor

      AVAILABLE_TARGETS = ['C.UTF-8']

      Aha, that's something new in FreeBSD. In FreeBSD 11.2, this locale is invalid. The locale has been added to FreeBSD CURRENT (future FreeBSD 12) at November 4:
      https://reviews.freebsd.org/D17833

      See discussion on freebsd-current:

      http://freebsd.1045724.x6.nabble.com/C-UTF-8-as-default-locale-td6292359.html

      So it's unrelated to my change, but it's a bug in test_c_locale_coercion on FreeBSD 12.

    6. vstinner commented on Nov 22, 2018

      @vstinner
      vstinner
      on Nov 22, 2018
      MemberAuthor

      More FreeBSD commits, "Teach man(1) about C.UTF-8":
      https://svnweb.freebsd.org/base?view=revision&revision=340128

      ---

      EXPECTED_C_LOCALE_STREAM_ENCODING = 'ascii'
EXPECTED_C_LOCALE_FS_ENCODING = 'ascii'

      I guess that the problem comes from that. But I'm not sure of what is the FS encoding for LC_ALL=C or LC_ALL=C.UTF-8.

    7. vstinner
      changed the title [-][FreeBSD] test_c_locale_coercion: fsencoding is ASCII instead of UTF-8 on FreeBSD CURRENT buildbot[/-] [+][FreeBSD] test_c_locale_coercion doesn't support new C.UTF-8 locale of FreeBSD CURRENT[/+] on Nov 22, 2018

    8. ncoghlan commented on Nov 22, 2018

      @ncoghlan
      ncoghlan
      on Nov 22, 2018
      Contributor

      Ah, interesting. https://bugs.python.org/issue30672 covered the fact that this test was already weird on FreeBSD, but the current status is that it was expecting BSD variants to act somewhat like Mac OS X, not like Linux.

      I'm wondering if we may need to make test_c_locale_coercion less opinionated, where for most platforms it's just a "don't segfault" test, and it's only for ones where we're certain about how we expect it to behave (i.e. Linux, Mac OS X, AIX, maybe FreeBSD) that we try to check the specifics.

    9. vstinner commented on Nov 22, 2018

      @vstinner
      vstinner
      on Nov 22, 2018
      MemberAuthor

      I'm wondering if we may need to make test_c_locale_coercion less opinionated, where for most platforms it's just a "don't segfault" test, and it's only for ones where we're certain about how we expect it to behave (i.e. Linux, Mac OS X, AIX, maybe FreeBSD) that we try to check the specifics.

      For test_embed, I tried to be smart to guess the encodings (ex: hardcode some encodings depending on the platform) and... I gave up. It's very hard when you consider Linux, Windows, macOS, FreeBSD, but also HP-UX and AIX. Each platform has weird cases about encodings... Now the test spawns a Python subprocess to get the encoding. See get_stdio_encoding() and get_filesystem_encoding() in Lib/test/test_embed.py.

    10. vstinner commented on Nov 23, 2018

      @vstinner
      vstinner
      on Nov 23, 2018
      MemberAuthor

      Ok, I found the bug: it is related to the commit 905f1ac of bpo-34523. Python now uses ASCII for the filesystem encoding if the internal "force ASCII" mode is enabled in Py_DecodeLocale/Py_EncodeLocale.

      FreeBSD CURRENT introduced a new C.UTF-8 locale which is used by C locale coercion (PEP-538).

      _PyCoreConfig_Read() now uses ASCII for the filesystem encoding if the "force ASCII" mode is enable (to ensure that encodings are consistent everywhere Python).

      Problem: The LC_CTYPE now uses UTF-8 (thanks to C locale coercion) whereas Python uses ASCII for the filesystem encoding since the "force ASCII" is still enabled.

      I wrote PR 10672 to reset the "force ASCII" mode. I tested manually my fix on the FreeBSD CURRENT buildbot.

    11. vstinner commented on Nov 23, 2018

      @vstinner
      vstinner
      on Nov 23, 2018
      MemberAuthor

      Oh, I also modified Python 3.7 to respect the "force ASCII" mode in bpo-34403, commit 21220bb. Hopefully, this change is not part of the latest Python 3.7.1 bugfix release.

      By the way, the bug only impacts FreeBSD CURRENT users (after Nov 4, after C.UTF-8 has been added) who have no locale set (LC_CTYPE is "C", ex: python3 running in an empty environment) and explicitly disable the UTF-8 mode... Well, I guess that very few users (I mean no one) are impacted :-)

    12. vstinner commented on Nov 23, 2018

      @vstinner
      vstinner
      on Nov 23, 2018
      MemberAuthor

      The bug is that sys.getfilesystemencoding() should be the locale encoding. On FreeBSD CURRENT, Python uses ASCII for the filesystem encoding, whereas the locale encoding is UTF-8.

      CURRENT-amd64% env -i ./python -X utf8=0 -c 'import locale, sys; print(sys.getfilesystemencoding(), locale.getpreferredencoding())'
      ascii UTF-8

    13. vstinner commented on Nov 23, 2018

      @vstinner
      vstinner
      on Nov 23, 2018
      MemberAuthor

      New changeset 353933e by Victor Stinner in branch 'master':
      bpo-34523: Fix C locale coercion on FreeBSD CURRENT (GH-10672)
      353933e

    14. vstinner commented on Nov 23, 2018

      @vstinner
      vstinner
      on Nov 23, 2018
      MemberAuthor

      New changeset f6e323c by Victor Stinner in branch '3.7':
      bpo-34523: Fix C locale coercion on FreeBSD CURRENT (GH-10672) (GH-10673)
      f6e323c

    15. vstinner commented on Nov 23, 2018

      @vstinner
      vstinner
      on Nov 23, 2018
      MemberAuthor

      I tested my 3.7 and master fixes: they fix test_c_locale_coercion on the FreeBSD CURRENT buildbot.

    16. vstinner
      added
      3.7 (EOL)end of life
      on Nov 23, 2018
    17. vstinner
      closed this as completedon Nov 23, 2018
    18. ezio-melotti
      transferred this issue fromon Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

    Metadata

    Metadata

    Assignees

    No one assigned

      Labels

      3.7 (EOL)end of life3.8 (EOL)end of lifeinterpreter-core(Objects, Python, Grammar, and Parser dirs)

      Projects

      No projects

        Milestone

        No milestone

        Relationships

        None yet

        Development

        No branches or pull requests

        Issue actions