Repository navigation
gh-91156: Use locale.getencoding() instead of getpreferredencoding
#91732
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 3 commits
0bf6bfc
8cd88b9
3005858
951875a
38601b0
b9f088b
d8e8c26
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -37,7 +37,7 @@ Linux and the BSD variants of Unix. | |
|
|
||
| import locale | ||
| locale.setlocale(locale.LC_ALL, '') | ||
| code = locale.getpreferredencoding() | ||
| code = locale.getencoding() | ||
|
|
||
| Then use *code* as the encoding for :meth:`str.encode` calls. | ||
|
|
||
|
|
@@ -924,7 +924,7 @@ the following methods and attributes: | |
| Encoding used to encode method arguments (Unicode strings and characters). | ||
| The encoding attribute is inherited from the parent window when a subwindow | ||
| is created, for example with :meth:`window.subwin`. By default, the locale | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Would you mind to replace "locale encoding" with "current locale encoding"? Just to remind that it can be changed at runtime. For example, the encoding used by the readline module is the currrent encoding, the encoding is not stored anywere. Its C code uses PyUnicode_EncodeLocale() and PyUnicode_DecodeLocale(): current LC_CTYPE locale encoding. |
||
| encoding is used (see :func:`locale.getpreferredencoding`). | ||
| encoding is used (see :func:`locale.getencoding`). | ||
|
|
||
| .. versionadded:: 3.3 | ||
|
|
||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -363,7 +363,7 @@ class TestEnUSCollation(BaseLocalizedTest, TestCollation): | |
| locale_type = locale.LC_ALL | ||
|
|
||
| def setUp(self): | ||
| enc = codecs.lookup(locale.getpreferredencoding(False) or 'ascii').name | ||
| enc = codecs.lookup(locale.getencoding() or 'ascii').name | ||
| if enc not in ('utf-8', 'iso8859-1', 'cp1252'): | ||
| raise unittest.SkipTest('encoding not suitable') | ||
| if enc != 'iso8859-1' and (sys.platform == 'darwin' or is_android or | ||
|
|
@@ -533,6 +533,12 @@ def test_defaults_UTF8(self): | |
| if orig_getlocale is not None: | ||
| _locale._getdefaultlocale = orig_getlocale | ||
|
|
||
| def test_getencoding(self): | ||
| # Invoke getencoding to make sure it does not cause exceptions. | ||
| enc = locale.getencoding() | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I suggest to test the type: add Maybe also ensure that the string is not empty? add |
||
| # make sure it is valid | ||
| codecs.lookup(enc) | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. _PyUnicode_InitEncodings() fails it config.filesystem_encoding or config.stdio_encoding is not known by codecs.lookup(name). So this call should not fail. If tomorrow this test fails, I suggest to remove it and only check that the string is non-empty.
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. When UTF-8 mode is enabled, both of stdio encoding and filesystem encoding are UTF-8, not locale encoding. |
||
|
|
||
| def test_getpreferredencoding(self): | ||
| # Invoke getpreferredencoding to make sure it does not cause exceptions. | ||
| enc = locale.getpreferredencoding() | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -145,11 +145,6 @@ def test_guess_all_types(self): | |
| self.assertNotIn('.no-such-ext', all) | ||
|
|
||
| def test_encoding(self): | ||
| getpreferredencoding = locale.getpreferredencoding | ||
| self.addCleanup(setattr, locale, 'getpreferredencoding', | ||
| getpreferredencoding) | ||
| locale.getpreferredencoding = lambda: 'ascii' | ||
|
|
||
|
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This hack doesn't work for most cases.
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Please remove This code is correct. I don't think that the locale encoding is still used: MimeType.read() calls |
||
| filename = support.findfile("mime.types") | ||
| mimes = mimetypes.MimeTypes([filename]) | ||
| exts = mimes.guess_all_extensions('application/vnd.geocube+xml', | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Remark unrelated to your PR.
This is wrong: Python now always call
setlocale(LC_CTYPE, "")at startup.Calling
locale.setlocale(locale.LC_ALL, '')is no longer needed.Moreover, curses likely use mbstowcs() and wcstombs() functions, rather than nl_langinfo() (nl_langinfo(CODESET)?).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
OK. Let's remove this note.