Repository navigation
Pyinstaller 5.x quits freezing when importing gi.pygtkcompat #6897
Description
Hi, I found issue of pyinstaller 5.x quits freezing when 'import gi.pygtkcompat' encountered in a source-file('twisted' in this case). This happens because that module has various gstreamer import references in one of it's functions, and hence stops freezing with error of missing gst typelibs/namespace as not having gstreamer built.
Can be easilly reproduced with 'pyinstaller gst.py', and a 'gst.py' of just 'import gi.pygtkcompat'. BTW, also happens with 'import pygtkcompat'.
Win10, python 3.9.13(also tested 3.10.5), latest pygobject/gtk3 built from gvsbuild and added to %PATH%.
This was also issue in pyinstaller v4.x, showing the error on the cmd-output but just didn't quit the freeze - I believe you changed pyinstaller v5.x to be more effective against quiting upon child-process errors, so guessing that's why.
Current workaround is with pyinstaller 5.x using an 'excludes' directive in spec of:
excludes=['gi.repository.Gst','gi.repository.GstAudio','gi.repository.GstVideo','gi.repository.GstBase','gi.repository.GstPbutils'],
Which works, but was wondering if this is something need more looking into from your side, if possible, thanks - or if we just need to add such meassures ourself because of the way this works in general?
Thanks alot in advance!
C:\deluge2\deluge-build>pyinstaller gst.py
546 INFO: PyInstaller: 5.1
546 INFO: Python: 3.9.13
562 INFO: Platform: Windows-10-10.0.19044-SP0
562 INFO: wrote C:\deluge2\deluge-build\gst.spec
578 INFO: UPX is not available.
593 INFO: Extending PYTHONPATH with paths
['C:\\deluge2\\deluge-build']
pygame 2.1.2 (SDL 2.0.18, Python 3.9.13)
Hello from the pygame community. https://www.pygame.org/contribute.html
3108 INFO: checking Analysis
3125 INFO: Building because C:\deluge2\python\Lib\site-packages\PyInstaller\hooks\rthooks\pyi_rth_gstreamer.py changed
3140 INFO: Initializing module dependency graph...
3140 INFO: Caching module graph hooks...
3264 INFO: Analyzing base_library.zip ...
9640 INFO: Processing pre-find module path hook distutils from 'C:\\deluge2\\python\\lib\\site-packages\\PyInstaller\\hooks\\pre_find_module_path\\hook-distutils.py'.
9640 INFO: distutils: retargeting to non-venv dir 'C:\\deluge2\\python\\lib'
17296 INFO: Caching module dependency graph...
17640 INFO: running Analysis Analysis-00.toc
17687 INFO: Adding Microsoft.Windows.Common-Controls to dependent assemblies of final executable
required by C:\deluge2\python\python.exe
17861 INFO: Analyzing C:\deluge2\deluge-build\gst.py
18328 INFO: Processing pre-safe import module hook gi.repository.GObject from 'C:\\deluge2\\python\\lib\\site-packages\\PyInstaller\\hooks\\pre_safe_import_module\\hook-gi.repository.GObject.py'.
18343 INFO: Processing pre-safe import module hook gi.repository.GLib from 'C:\\deluge2\\python\\lib\\site-packages\\PyInstaller\\hooks\\pre_safe_import_module\\hook-gi.repository.GLib.py'.
18343 INFO: Processing pre-safe import module hook gi.repository.Gio from 'C:\\deluge2\\python\\lib\\site-packages\\PyInstaller\\hooks\\pre_safe_import_module\\hook-gi.repository.Gio.py'.
18343 INFO: Processing pre-safe import module hook gi.repository.Atk from 'C:\\deluge2\\python\\lib\\site-packages\\PyInstaller\\hooks\\pre_safe_import_module\\hook-gi.repository.Atk.py'.
18359 INFO: Processing pre-safe import module hook gi.repository.Pango from 'C:\\deluge2\\python\\lib\\site-packages\\PyInstaller\\hooks\\pre_safe_import_module\\hook-gi.repository.Pango.py'.
18359 INFO: Processing pre-safe import module hook gi.repository.PangoCairo from 'C:\\deluge2\\python\\lib\\site-packages\\PyInstaller\\hooks\\pre_safe_import_module\\hook-gi.repository.PangoCairo.py'.
18359 INFO: Processing pre-safe import module hook gi.repository.Gdk from 'C:\\deluge2\\python\\lib\\site-packages\\PyInstaller\\hooks\\pre_safe_import_module\\hook-gi.repository.Gdk.py'.
18359 INFO: Processing pre-safe import module hook gi.repository.GdkPixbuf from 'C:\\deluge2\\python\\lib\\site-packages\\PyInstaller\\hooks\\pre_safe_import_module\\hook-gi.repository.GdkPixbuf.py'.
18359 INFO: Processing pre-safe import module hook gi.repository.Gtk from 'C:\\deluge2\\python\\lib\\site-packages\\PyInstaller\\hooks\\pre_safe_import_module\\hook-gi.repository.Gtk.py'.
18423 INFO: Processing pre-safe import module hook gi.repository.Gst from 'C:\\deluge2\\python\\lib\\site-packages\\PyInstaller\\hooks\\pre_safe_import_module\\hook-gi.repository.Gst.py'.
18423 INFO: Processing pre-safe import module hook gi.repository.GstAudio from 'C:\\deluge2\\python\\lib\\site-packages\\PyInstaller\\hooks\\pre_safe_import_module\\hook-gi.repository.GstAudio.py'.
18423 INFO: Processing pre-safe import module hook gi.repository.GstVideo from 'C:\\deluge2\\python\\lib\\site-packages\\PyInstaller\\hooks\\pre_safe_import_module\\hook-gi.repository.GstVideo.py'.
18438 INFO: Processing pre-safe import module hook gi.repository.GstBase from 'C:\\deluge2\\python\\lib\\site-packages\\PyInstaller\\hooks\\pre_safe_import_module\\hook-gi.repository.GstBase.py'.
18438 INFO: Processing pre-safe import module hook gi.repository.GstPbutils from 'C:\\deluge2\\python\\lib\\site-packages\\PyInstaller\\hooks\\pre_safe_import_module\\hook-gi.repository.GstPbutils.py'.
18438 INFO: Processing module hooks...
18438 INFO: Loading module hook 'hook-difflib.py' from 'C:\\deluge2\\python\\lib\\site-packages\\PyInstaller\\hooks'...
18452 INFO: Loading module hook 'hook-distutils.py' from 'C:\\deluge2\\python\\lib\\site-packages\\PyInstaller\\hooks'...
18452 INFO: Loading module hook 'hook-encodings.py' from 'C:\\deluge2\\python\\lib\\site-packages\\PyInstaller\\hooks'...
18672 INFO: Loading module hook 'hook-gi.py' from 'C:\\deluge2\\python\\lib\\site-packages\\PyInstaller\\hooks'...
18718 INFO: Loading module hook 'hook-gi.repository.Atk.py' from 'C:\\deluge2\\python\\lib\\site-packages\\PyInstaller\\hooks'...
20359 INFO: Loading module hook 'hook-gi.repository.Gdk.py' from 'C:\\deluge2\\python\\lib\\site-packages\\PyInstaller\\hooks'...
21391 INFO: Processing pre-safe import module hook gi.repository.cairo from 'C:\\deluge2\\python\\lib\\site-packages\\PyInstaller\\hooks\\pre_safe_import_module\\hook-gi.repository.cairo.py'.
21406 INFO: Loading module hook 'hook-gi.repository.GdkPixbuf.py' from 'C:\\deluge2\\python\\lib\\site-packages\\PyInstaller\\hooks'...
24452 INFO: Processing pre-safe import module hook gi.repository.GModule from 'C:\\deluge2\\python\\lib\\site-packages\\PyInstaller\\hooks\\pre_safe_import_module\\hook-gi.repository.GModule.py'.
24452 INFO: Loading module hook 'hook-gi.repository.Gio.py' from 'C:\\deluge2\\python\\lib\\site-packages\\PyInstaller\\hooks'...
26593 INFO: Loading module hook 'hook-gi.repository.GLib.py' from 'C:\\deluge2\\python\\lib\\site-packages\\PyInstaller\\hooks'...
29110 INFO: Loading module hook 'hook-gi.repository.GModule.py' from 'C:\\deluge2\\python\\lib\\site-packages\\PyInstaller\\hooks'...
30046 INFO: Loading module hook 'hook-gi.repository.GObject.py' from 'C:\\deluge2\\python\\lib\\site-packages\\PyInstaller\\hooks'...
31030 WARNING: Hidden import "gi._gobject" not found!
31030 INFO: Loading module hook 'hook-gi.repository.Gst.py' from 'C:\\deluge2\\python\\lib\\site-packages\\PyInstaller\\hooks'...
Traceback (most recent call last):
File "C:\deluge2\python\lib\runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
File "C:\deluge2\python\lib\runpy.py", line 87, in _run_code
exec(code, run_globals)
File "C:\deluge2\python\Scripts\pyinstaller.exe\__main__.py", line 7, in <module>
File "C:\deluge2\python\lib\site-packages\PyInstaller\__main__.py", line 178, in run
run_build(pyi_config, spec_file, **vars(args))
File "C:\deluge2\python\lib\site-packages\PyInstaller\__main__.py", line 59, in run_build
PyInstaller.building.build_main.main(pyi_config, spec_file, **kwargs)
File "C:\deluge2\python\lib\site-packages\PyInstaller\building\build_main.py", line 842, in main
build(specfile, distpath, workpath, clean_build)
File "C:\deluge2\python\lib\site-packages\PyInstaller\building\build_main.py", line 764, in build
exec(code, spec_namespace)
File "C:\deluge2\deluge-build\gst.spec", line 7, in <module>
a = Analysis(
File "C:\deluge2\python\lib\site-packages\PyInstaller\building\build_main.py", line 319, in __init__
self.__postinit__()
File "C:\deluge2\python\lib\site-packages\PyInstaller\building\datastruct.py", line 173, in __postinit__
self.assemble()
File "C:\deluge2\python\lib\site-packages\PyInstaller\building\build_main.py", line 487, in assemble
self.graph.process_post_graph_hooks(self)
File "C:\deluge2\python\lib\site-packages\PyInstaller\depend\analysis.py", line 326, in process_post_graph_hooks
module_hook.post_graph(analysis)
File "C:\deluge2\python\lib\site-packages\PyInstaller\depend\imphook.py", line 398, in post_graph
self._load_hook_module()
File "C:\deluge2\python\lib\site-packages\PyInstaller\depend\imphook.py", line 361, in _load_hook_module
self._hook_module = importlib_load_source(self.hook_module_name, self.hook_filename)
File "C:\deluge2\python\lib\site-packages\PyInstaller\compat.py", line 620, in importlib_load_source
return mod_loader.load_module()
File "<frozen importlib._bootstrap_external>", line 529, in _check_name_wrapper
File "<frozen importlib._bootstrap_external>", line 1029, in load_module
File "<frozen importlib._bootstrap_external>", line 854, in load_module
File "<frozen importlib._bootstrap>", line 274, in _load_module_shim
File "<frozen importlib._bootstrap>", line 711, in _load
File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 850, in exec_module
File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
File "C:\deluge2\python\lib\site-packages\PyInstaller\hooks\hook-gi.repository.Gst.py", line 30, in <module>
binaries, datas, hiddenimports = get_gi_typelibs('Gst', '1.0')
File "C:\deluge2\python\lib\site-packages\PyInstaller\utils\hooks\gi.py", line 71, in get_gi_typelibs
typelibs_data = _gi_typelibs(module, version)
File "C:\deluge2\python\lib\site-packages\PyInstaller\isolated\_parent.py", line 269, in wrapped
return call(function, *args, **kwargs)
File "C:\deluge2\python\lib\site-packages\PyInstaller\isolated\_parent.py", line 238, in call
return isolated.call(function, *args, **kwargs)
File "C:\deluge2\python\lib\site-packages\PyInstaller\isolated\_parent.py", line 176, in call
raise RuntimeError(f"Child process call to {function.__name__}() failed with:\n" + output)
RuntimeError: Child process call to _gi_typelibs() failed with:
File "C:\deluge2\python\lib\site-packages\PyInstaller\isolated\_child.py", line 63, in run_next_command
output = function(*args, **kwargs)
File "C:\deluge2\python\lib\site-packages\PyInstaller\utils\hooks\gi.py", line 63, in _gi_typelibs
repo.require(module, version, GIRepository.RepositoryLoadFlags.IREPOSITORY_LOAD_FLAG_LAZY)
gi.repository.GLib.GError: g-irepository-error-quark: Typelib file for namespace 'Gst', version '1.0' not found (0)
Activity
mhertz commentedon Jun 16, 2022
@rokm I really appreciate you looking into this, thank you so much! :)
Unfortunetly #6898 still fails, but thanks again for it!
Let me know if can help in some way - probably only to test things as not very good at this honestly, but regardless, just throwing out this just in case :)
Thanks!.
rokm commentedon Jun 16, 2022
Ah, so there are other places that need the exception to be caught. Will revise the PR accordingly...
mhertz commentedon Jun 17, 2022
Again really appreciate it! :)
As per the last remaining issue from 'PyInstaller\hooks\hook-gi.repository.Gst.py', then I added exception for 'plugin_path()' and return empty, just for testing meanwhile(and from your queue :) ), and that works, but not sure if it needs fixed more "universally" in the parent/child isolation or however that goes, but anyways just wanted to say many thanks again for looking into this.
bwoodsend commentedon Jun 17, 2022
We used to have an isolation system which didn't detect exceptions in subprocesses and lead to totally obscure errors like #5504. Aborting a build if a subprocess fails is definitely an intentional default - even if it has falsely derailed a few builds.
mhertz commentedon Jun 17, 2022
Thanks for elaborating, and I'm sorry for the not very bright comment regarding that :)
rokm commentedon Jun 17, 2022
I think the
gihooks need to be reworked a bit; since we declaregi.repository.<Module>as runtime modules, their hooks are always ran, so their contents should be protected by module availability check to avoid unnecessary warnings (or in this case, errors) about missing modules.But yes, adding exception handling around
plugin_path()inPyInstaller\hooks\hook-gi.repository.Gst.pyis an immediate/direct fix for the issue at hand.mhertz commentedon Jun 18, 2022
Really much appreciate y'all :)
mhertz commentedon Jun 18, 2022
At the risk of sounding repetetive, then sincerely thank you so much rokm, really appreciate all the nice work you did on this and the kind help you offered me! #6901 fixes my issue perfectly, while sounding like a great improvement to the project, so again thank you so much for all the work and help you layed into this, really appreciated immensely.