Repository navigation
Handle "POSIX" in the legacy locale detection #76419
Description
| BPO | 32238 |
|---|---|
| Nosy | @ronaldoussoren, @ncoghlan, @vstinner, @jwilk, @ned-deily, @ezio-melotti, @koobs |
| Dependencies | |
| Superseder |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = None
closed_at = None
created_at = <Date 2017-12-07.00:28:35.555>
labels = ['OS-mac', 'interpreter-core', 'type-bug', '3.8', '3.7', 'expert-unicode']
title = 'Handle "POSIX" in the legacy locale detection'
updated_at = <Date 2019-10-11.21:32:55.917>
user = 'https://github.com/ncoghlan'bugs.python.org fields:
activity = <Date 2019-10-11.21:32:55.917>
actor = 'vstinner'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Interpreter Core', 'macOS', 'Unicode', 'FreeBSD']
creation = <Date 2017-12-07.00:28:35.555>
creator = 'ncoghlan'
dependencies = ['32002']
files = []
hgrepos = []
issue_num = 32238
keywords = []
message_count = 4.0
messages = ['307781', '307782', '307783', '354502']
nosy_count = 7.0
nosy_names = ['ronaldoussoren', 'ncoghlan', 'vstinner', 'jwilk', 'ned.deily', 'ezio.melotti', 'koobs']
pr_nums = []
priority = 'normal'
resolution = None
stage = 'test needed'
status = 'open'
superseder = '30672'
type = 'behavior'
url = 'https://bugs.python.org/issue32238'
versions = ['Python 3.7', 'Python 3.8']
Activity
ncoghlan commentedon Dec 7, 2017
Right now, the legacy locale detection introduced in PEP-538 doesn't trigger for "LANG=POSIX" and "LC_CTYPE=POSIX" on macOS and other *BSD systems.
This is because we're looking specifically for "C" as the response from "setlocale(LC_CTYPE, NULL)", which works on Linux (where glibc reports "C" if you configured "POSIX"), but not on *BSD systems (where POSIX and C behave the same way, but are still reported as distinct locales).
As per Jakub Wilk's comments at https://mail.python.org/pipermail/python-dev/2017-December/151105.html, this isn't right: we should allow either string to be returned from setlocale, and consider both of them as indicating a legacy locale to be coerced to an explicitly UTF-8 based one if possible.
ncoghlan commentedon Dec 7, 2017
Added a dependency on https://bugs.python.org/issue32002, as we should finish the test case refactoring proposed there before adjusting the
POSIXlocale handling on macOS and other *BSD systems.ncoghlan commentedon Dec 7, 2017
Oops, I forgot I already had an open issue for this discrepancy - I just hadn't decided how to resolve it yet.
Marking as a duplicate of https://bugs.python.org/issue30672
vstinner commentedon Oct 11, 2019
In Python 3.8, if the LC_CTYPE is "POSIX", the default stdio error handler is now "surrogateescape" instead of "strict", and the UTF-8 is now enabled. In short, LC_CTYPE="POSIX" now behaves as LC_CTYPE="C".
This change impacts at least FreeBSD. If I correctly, if there is no LC_ALL, LC_CTYPE or LANG environment variable on FreeBSD, the LC_CTYPE locale is "POSIX".
See bpo-34485, bpo-19977 and the "POSIX locale on FreeBSD" section of my article:
https://vstinner.github.io/python3-locales-encodings.html
ronaldoussoren commentedon Jan 18, 2024
For macOS the default LC_CTYPE is
UTF-8, that is:LC_TYPE=UTF-8in the output ofenv(and inos.environ()in a Python session)unset LC_CTYPE) and start Python I still seeLC_CTYPE=UTF-8in the environmentIs there anything left to do for this issue? In particular for macOS as I don't have BSD VM's at hand.
devdanzin commentedon Sep 21, 2025
If this issue is done, so is #74857.
vstinner commentedon Sep 22, 2025
I close this PR as a duplicate of #74857.