Repository navigation
Support more platforms in ctypes.find_library #142020
Description
Activity
AFAICT, we do check this. Where is the
ldconfigcall being used? The only call I could find was inctypesbut we have a try-catch already.AFAICT, we do check this. Where is the
ldconfigcall being used? The only call I could find was inctypesbut we have a try-catch already.The call is coming from pyudev:
Traceback (most recent call last): File "/tmp/samuel/Volatile/AppDir/bin/oversteer", line 53, in <module> sys.exit(app.run(sys.argv)) ~~~~~~~^^^^^^^^^^ File "/tmp/samuel/Volatile/AppDir/lib/python3.13/site-packages/oversteer/application.py", line 63, in run self.device_manager.start() ~~~~~~~~~~~~~~~~~~~~~~~~~^^ File "/tmp/samuel/Volatile/AppDir/lib/python3.13/site-packages/oversteer/device_manager.py", line 54, in start context = pyudev.Context() File "/tmp/samuel/Volatile/AppDir/lib/python3.13/site-packages/pyudev/core.py", line 59, in __init__ self._libudev = load_ctypes_library("udev", SIGNATURES, ERROR_CHECKERS) ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/tmp/samuel/Volatile/AppDir/lib/python3.13/site-packages/pyudev/_ctypeslib/utils.py", line 51, in load_ctypes_library raise ImportError("No library named %s" % name) ImportError: No library named udevA similar issue happened with another python app that used pyusb.
Maybe the check for
ldconfig -pstops any further lookup?In this case, i think it's an issue with
pyudevand not CPython itself.load_ctypes_libraryis a pyudev function andpyudev/_ctypeslib/utils.pyis not thectypesfrom CPython.Also, the issue doesn't seem related to ldconfig. It looks like the host machine is missing the
.solibrary forudev(that is the corresponding*-develor*-devpackage depending on the distro)More generally, what you could actually want is for
ctypesto still find your library I guess?ctypes.find_librarytries quite hard to find libs but it's not always possible to have a very generic solution for that, especially if it's for platforms that we don't support officially as per PEP-11. I would suggest to report this issue topyudevinstead and ask them if it's possible to try harder for finding libraries (it looks like they are doing their own machinery for that).In this case, i think it's an issue with pyudev and not CPython itself. load_ctypes_library is a pyudev function and pyudev/_ctypeslib/utils.py is not the ctypes from CPython.
Isn't pyudev using
find_libraryfrom here though?Also, the issue doesn't seem related to ldconfig. It looks like the host machine is missing the
.solibrary forudev(that is the corresponding*-develor*-devpackage depending on the distro)It is related to ldconfig, after we added it and made sure to redirect all calls to
/sbin/ldconfigto the bundled one that supports-pthe issue was fixed.And as previously mentioned this is weird that it even works, because that bundled
ldconfigis outputting the libraries of the host, not the bundled ones, yet it eventually finds the bundled ones without problem.In other words this is failing because
/sbin/ldconfig -pfails, not because it cannot find the library.Isn't pyudev using find_library from here though?
I don't know. What I can tell you is that the call to
ldconfigis protected by a try-catch. The exception being raised is not raised from CPython but frompyudev.Ref:
Lines 297 to 310 in 3bbd669
The exception being raised is here:
IOW,
ctypes.find_libraryreturns None whichpyudevconsiders to indicate that the library is missing which is the case I guess. I would suggest reporting this topyudev. It's not an issue in CPython (unless you want us to allow to specify the path to ldconfig?)I thought the problem was from here because ctypes is the only thing that contains the
/sbin/ldconfingand it attempts to execute/sbin/ldconfig -p
This problem gets fixed by bundling an
ldconfigthat supports-p, even though its output is totally useless here, this fixes the issue I'm having.So I guess the exception stops pyudev from trying to keep looking up?
(this problem also affects pyusb btw)
Here is an
LD_DEBUG=libsoutput of the working appimage with a bundledldconfig: libs.logDoesn't attempt any of the libraries reported by the bundled
ldconfig -p. (which is good, it is all working fine btw)So my current theory is that pyudev fails because
find_libraryerrors? stops? whenldconfig -pis not possible.21 remaining items
I think we've established that the current implementation of
find_libraryis not reliable. The question is: is there a reliable method of doing what you need? If so, we can update the existing function, or add a new one if that would be an incompatible change.Alright, the ideal solution is finding a way to
dlopena dummy library and check what directories were looked at, this is the perfect solution since this will 100% reliably list all possible library locations.If that is not possible, the other solution is to just not hardcode
ldconfigtosbinand just usePATH, you will still ned to check for/sbin/ldconfigsince some distros don't expose/sbininPATHby default.note that right now if I attempt to patch this to use the
ldconfiginPATH, the variable seems to be reset since it only looks in/binand/usr/binon my end.Another is having a variable like
PYTHON_LIBRARY_PATHthat we can list the directories to be looked at, however I'm a bit worried this will cause issues since it can leak to other python instances on the system similar the issues when usingLD_LIBRARY_PATHReacted by Filipe Laíns- added a commit that references this issue
on Dec 15, 2025 I'm proposing to soft-deprecate
ctypes.util.find_library: https://discuss.python.org/t/106232Reacted by mara004I'm proposing to soft-deprecate
ctypes.util.find_library: https://discuss.python.org/t/106232I don't think this is needed.
I while back we noticed with python it is no longer needed to
PYTHONHOMEanymore, python figures out the location of the pythondir on its own, see for example the output of an appimage that has python bundled without settingPYTHONHOME:./puddletag.AppImage python -c "import sys,os; print('exe=', sys.executable); print('prefix=', sys.prefix); print('paths='); print('\n'.join(sys.path))" 222ms exe= /tmp/samuel/.mount_puddlremp6111792694178396282/bin/python prefix= /tmp/samuel/.mount_puddlremp6111792694178396282 paths= /tmp/samuel/.mount_puddlremp6111792694178396282/lib/python314.zip /tmp/samuel/.mount_puddlremp6111792694178396282/lib/python3.14 /tmp/samuel/.mount_puddlremp6111792694178396282/lib/python3.14/lib-dynload /tmp/samuel/.mount_puddlremp6111792694178396282/lib/python3.14/site-packagesThis logic can be used to find the libraries as well, basically all the
*.sofiles in the directory where the python dir is located.This logic can be used to find the libraries as well, basically all the *.so files in the directory where the python dir is located.
Yes. Many approaches are better than
find_library's logic of runningldconfig/gcc/objdump/ld.find_libraryis now soft-deprecated. It cannot be made to work correctly and consistently on all platforms, and any change runs a chance to break whatever workarounds people already have in place.
I'm prioritizing stability over new features, and encourage solving this outside ctypes itself.
Bug report
Bug description:
I make appimages that bundle glibc and as result are able to work on any linux system (even in musl libc systems).
I had this issue which wasn't fully fixed until I manage to find out that the problem was cpython running
/sbin/ldconfig -pwhich caused everything to fail.Interestingly enough the output of
ldconfigis not needed here, since after doing some patching to bundleldconfigthe issue was fixed, thatldconfigis just outputting the libraries of the alpine system and none of the bundled libraries yet the application finds and uses the bundled libraries without issue.so this should test if
ldconfig -pis supported before attempting to use it.CPython versions tested on:
3.13
Operating systems tested on:
Linux
Linked PRs