Repository navigation
Output .pyd name in error message of ImportError when DLL load fails #55063
Description
| BPO | 10854 |
|---|---|
| Nosy | @loewis, @birkenfeld, @amauryfa, @ncoghlan, @kristjanvalur, @briancurtin, @ericsnowcurrently, @eryksun, @iritkatriel |
| Dependencies | |
| Files |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = 'https://github.com/briancurtin'
closed_at = <Date 2020-11-17.01:22:59.623>
created_at = <Date 2011-01-07.15:48:43.761>
labels = ['extension-modules', 'type-feature', '3.9', '3.10']
title = 'Output .pyd name in error message of ImportError when DLL load fails'
updated_at = <Date 2020-11-17.01:22:59.619>
user = 'https://bugs.python.org/techtonik'bugs.python.org fields:
activity = <Date 2020-11-17.01:22:59.619>
actor = 'eryksun'
assignee = 'brian.curtin'
closed = True
closed_date = <Date 2020-11-17.01:22:59.623>
closer = 'eryksun'
components = ['Extension Modules']
creation = <Date 2011-01-07.15:48:43.761>
creator = 'techtonik'
dependencies = ['1559549']
files = ['22768', '26476']
hgrepos = []
issue_num = 10854
keywords = ['patch']
message_count = 25.0
messages = ['125649', '125651', '125652', '125653', '125663', '125665', '125668', '125682', '125727', '125765', '125768', '126279', '126281', '126283', '141205', '141209', '147439', '158399', '162376', '162616', '166124', '166143', '166179', '381184', '381202']
nosy_count = 13.0
nosy_names = ['loewis', 'georg.brandl', 'amaury.forgeotdarc', 'ncoghlan', 'kristjan.jonsson', 'techtonik', 'brian.curtin', 'cgohlke', 'python-dev', 'eric.snow', 'piotr.dobrogost', 'eryksun', 'iritkatriel']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue10854'
versions = ['Python 3.9', 'Python 3.10']
Activity
techtonik commentedon Jan 7, 2011
When an extension could not be loaded, because it requires some DLL that is missing, Python shows the following error message:
ImportError: DLL load failed: The specified module could not be found.
It will help tremendously in debugging extension problems if the missing DLL name is specified in error message.
amauryfa commentedon Jan 7, 2011
Is it even possible? Each time I tried, the only solutions involved an external program like Dependency Walker.
briancurtin commentedon Jan 7, 2011
I'm pretty sure we can't do this, if I understand your request.
Say you have techtonik.pyd as your extension and it depends on foobar.dll. If we try to load techtonik.pyd and this pyd can't find or successfully load foobar.dll, Python doesn't know about *this* DLL. It just knows that it can't load techtonik.pyd, so that's what we'd be able to output, which isn't that helpful.
briancurtin commentedon Jan 7, 2011
Agree with Amaury. depends has always been my solution to this type of problem.
techtonik commentedon Jan 7, 2011
Can you cross reference the part of code where this error is catched?
Why Python can't get information about the reason .DLL is not loaded?
Is it at least possible to add a hook point at the exact time the import fails to insert component that is able to inquiry system about this information (like depends)?
briancurtin commentedon Jan 7, 2011
See _PyImport_GetDynLoadFunc in Python/dynload_win.c -- that's where this is happening.
Windows does not provide it in the case you are speaking of. If I call LoadLibraryEx for techtonik.pyd, either it succeeds because everything is there, or it fails because something is not there. We get no information about dependency hierarchies failing to load.
amauryfa commentedon Jan 7, 2011
The code is in Python/dynload_win.c
To load an extension module mymodule.pyd, Python calls LoadLibrary('/path/to/mymodule.pyd'); when it returns NULL, the code calls GetLastError() (which returns 126 in this case) then FormatMessage to get an error message.
To get the full path to the .pyd, you can use imp.find_module(), but there is nothing else we can do I fear.
techtonik commentedon Jan 7, 2011
I see. But depends.exe dependency walker somehow finds the exact code that is failing, so there should be a way to do extra investigation in case of error.
py2exe has a custom .dll loader that bypasses LoadLibrary(Ex) calls from that I can see at http://py2exe.svn.sourceforge.net/viewvc/py2exe/trunk/py2exe/source/
Even if this code can't be used as a replacement for LoadLibrary calls - it can be very useful for troubleshooting problems in case of missing dependencies. It will also help when embedding python or developing extensions.
loewis commentedon Jan 7, 2011
Closing this as "won't fix". Python is not going to reimplement depends.exe.
techtonik commentedon Jan 8, 2011
Martin, shouldn't there be at least majority to reach consensus on this matter, or you're self-proclaimed BSDL II? =)
At least failed .pyd module name should be present in error message.
11 remaining items
python-dev commentedon Apr 16, 2012
New changeset f341b99bb370 by Brian Curtin in branch 'default':
Fix bpo-10854. Make use of the new path and name attributes on ImportError
http://hg.python.org/cpython/rev/f341b99bb370
kristjanvalur commentedon Jun 5, 2012
repr() or str() of ImportError doesn't include the 'name' or 'path' members. It would be useful to have them added to the repr if present.
kristjanvalur commentedon Jun 11, 2012
Brian, reopening this since the original issue isn't addressed: The "path" and "module" attributes aren't part of the error "repr"
kristjanvalur commentedon Jul 22, 2012
An example patch that aims to put the "name" and "path" in str() and repr() representations of ImportError.
Output when _socket.pyd is not found:
>>> import socket Traceback (most recent call last): File "<stdin>", line 1, in <module> File "D:\pydev\hg\cpython3\lib\socket.py", line 47, in <module> import _socket ImportError: No module named '_socket', _socket, >>> sys.last_value <ImportError "No module named '_socket'", '_socket', None> >>>amauryfa commentedon Jul 22, 2012
How it the new message better than "ImportError: No module named '_socket'"?
kristjanvalur commentedon Jul 22, 2012
Ok, the demo demo I added sucks, because I didn't demo the original problem (.pyd found but could not be loaded), but the point is, it displays the "name" and "path" attributes. They can be set, for example, if the module is found but cannot be loaded for some reason.
That's the point of this issue. The fix added a "name" and "path" attributes, but they were never actually _output_ when such an error occurred.
iritkatriel commentedon Nov 16, 2020
The patch adding 'name' and 'path' to the ImportError str/repr was not merged yet.
eryksun commentedon Nov 17, 2020
It's up to whatever code raises an ImportError to determine how the name and path should be included in the error message. In Windows, the DLL name was added to the exception message in 3.8. So I'm closing this issue again, with regard to its final scope.
With regard to its original scope, the reason for closing is "wont fix". There are no plans to extend the import error of an extension module in Windows to include the missing DLL dependency that caused the import to fail. There are tools such as the "DLL Diagnostics" package on PyPI that implement a proper trace by parsing loader snaps, and at the very least there are tools such as dumpbin.exe (or link.exe /dump) to manually list the direct dependencies of a DLL.