Repository navigation
ctypes.dlopen() doesn't support surrogates #52641
Description
| BPO | 8394 |
|---|---|
| Nosy | @loewis, @theller, @amauryfa, @vstinner |
| Files |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = 'https://github.com/theller'
closed_at = <Date 2010-04-20.20:20:24.626>
created_at = <Date 2010-04-14.01:16:19.236>
labels = ['ctypes', 'type-bug', 'library', 'expert-unicode']
title = "ctypes.dlopen() doesn't support surrogates"
updated_at = <Date 2010-04-20.20:20:24.624>
user = 'https://github.com/vstinner'bugs.python.org fields:
activity = <Date 2010-04-20.20:20:24.624>
actor = 'loewis'
assignee = 'theller'
closed = True
closed_date = <Date 2010-04-20.20:20:24.626>
closer = 'loewis'
components = ['Library (Lib)', 'Unicode', 'ctypes']
creation = <Date 2010-04-14.01:16:19.236>
creator = 'vstinner'
dependencies = []
files = ['16938']
hgrepos = []
issue_num = 8394
keywords = ['patch']
message_count = 10.0
messages = ['103108', '103115', '103272', '103273', '103458', '103564', '103566', '103569', '103570', '103746']
nosy_count = 4.0
nosy_names = ['loewis', 'theller', 'amaury.forgeotdarc', 'vstinner']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'test needed'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue8394'
versions = ['Python 3.1', 'Python 3.2']
Activity
vstinner commentedon Apr 14, 2010
The PEP-383 introduces filename using surrogates. ctypes.dlopen() support them. ctypes.cdll.LoadLibrary('libc\uDCff.so.6') fails with:
UnicodeEncodeError: 'utf-8' codec can't encode character '\udcff'
in position 4: surrogates not allowed
Attached patch fixes this issue.
TODO: Remove the assert(PyBytes_Check(name2)). I don't know if PyUnicode_FSConverter() does always return a PyBytes object or not.
amauryfa commentedon Apr 14, 2010
PyUnicode_FSConverter returns bytes and bytearray objects unchanged; otherwise it always return bytes.
Your patch should handle the case when name2 is a bytearray.
vstinner commentedon Apr 15, 2010
amaury> Your patch should handle the case when name2 is a bytearray.
Ok, fixed. I also tested None: Python does segfault :-) New patch rejects None value.
vstinner commentedon Apr 15, 2010
(oops, my patch included unrelated changes about trailing spaces)
vstinner commentedon Apr 18, 2010
Fixed: r80159 (py3k), r80160 (3.1). I commited a different version of my patch to support None.
amauryfa commentedon Apr 19, 2010
It does not work on Windows:
>>> ctypes.CDLL(b'kernel32') Traceback (most recent call last): File "<stdin>", line 1, in <module> File "D:\afa\python\py3k-1\lib\ctypes\__init__.py", line 350, in __init__ self._handle = _dlopen(self._name, mode) TypeError: bad argument type for built-in operationvstinner commentedon Apr 19, 2010
load_library() uses LoadLibraryW() which use a WCHAR*. To support bytes, we can use LoadLibraryA() and TCHAR*.
vstinner commentedon Apr 19, 2010
I only fixed UNIX/BSD versions of subprocess/ctypes.dlopen() because it's not possible to open some files with an undecodable filename. On Windows, the file system and Python3 use Unicode, and so there is no such corner case.
On Windows, should we encourage people migrating from byte to character string? Or should we support byte string for backward compatibility or because there is really a corner case where it's not possible to open a file with an unicode string?
See also my msg103565 (Issue bpo-8393, subprocess),
amauryfa commentedon Apr 19, 2010
yes, except that TCHAR* depends on compilation settings (it resolves to wchar_t when UNICODE is #defined); simply use char*.
loewis commentedon Apr 20, 2010
Amaury, I'm closing this for the same reason I explained in msg103745