Skip to content

pyinstaller executable can't find package info #1187

Closed
pyinstaller/pyinstaller
#2637
Closed
pyinstaller executable can't find package info#1187
pyinstaller/pyinstaller
#2637
Assignees
dhermes
Labels
packaging

Description

@mtwu-aviatrix
mtwu-aviatrix
opened on Oct 16, 2015

The pyinstaller complains the error with the following line:

from gcloud.pubsub.client import Client

The module is installed correctly in /usr/local/lib/python2.7/dist-packages. I run setup.py (build/ install) in addition pip install --upgrade cloud too.

Traceback (most recent call last):
  File "<string>", line 267, in <module>
  File "/usr/local/lib/python2.7/dist-packages/PyInstaller-2.1.1dev_-py2.7.egg/PyInstaller/loader/pyi_importers.py", line 271, in load_module
    exec(bytecode, module.__dict__)
  File "/home/mtwu/gce/cloudn/cloudx-local/gce/gce_pubsub.py", line 6, in <module>
  File "/usr/local/lib/python2.7/dist-packages/PyInstaller-2.1.1dev_-py2.7.egg/PyInstaller/loader/pyi_importers.py", line 271, in load_module
    exec(bytecode, module.__dict__)
  File "/home/mtwu/gce/cloudn/install/build/cloudx_cli/out00-PYZ.pyz/gcloud", line 19, in <module>
  File "/home/mtwu/gce/cloudn/install/build/cloudx_cli/out00-PYZ.pyz/pkg_resources", line 551, in get_distribution
The following import will cause the following pyinstaller error.
from gcloud.pubsub.client import Client
  File "/home/mtwu/gce/cloudn/install/build/cloudx_cli/out00-PYZ.pyz/pkg_resources", line 431, in get_provider
  File "/home/mtwu/gce/cloudn/install/build/cloudx_cli/out00-PYZ.pyz/pkg_resources", line 952, in require
  File "/home/mtwu/gce/cloudn/install/build/cloudx_cli/out00-PYZ.pyz/pkg_resources", line 839, in resolve
pkg_resources.DistributionNotFound: The 'gcloud' distribution was not found and is required by the application
mtwu@ubuntu:/usr/local/lib/python2.7/dist-packages$ ls -l gcloud 
total 360
drwxr-sr-x 2 root staff  4096 Oct 16 09:13 bigquery
drwxr-sr-x 3 root staff  4096 Oct 16 09:13 bigtable
-rw-r--r-- 1 root staff  7564 Oct 16 09:13 client.py
-rw-r--r-- 1 root staff  8038 Oct 16 09:13 client.pyc
-rw-r--r-- 1 root staff 12949 Oct 16 09:13 connection.py
-rw-r--r-- 1 root staff 12304 Oct 16 09:13 connection.pyc
-rw-r--r-- 1 root staff 14075 Oct 16 09:13 credentials.py
-rw-r--r-- 1 root staff 13661 Oct 16 09:13 credentials.pyc
drwxr-sr-x 3 root staff  4096 Oct 16 09:13 datastore
-rw-r--r-- 1 root staff  4036 Oct 16 09:13 demo.py
-rw-r--r-- 1 root staff  4010 Oct 16 09:13 demo.pyc
drwxr-sr-x 2 root staff  4096 Oct 16 09:13 dns
-rw-r--r-- 1 root staff  1418 Oct 16 09:13 environment_vars.py
-rw-r--r-- 1 root staff   647 Oct 16 09:13 environment_vars.pyc
-rw-r--r-- 1 root staff  5819 Oct 16 09:13 exceptions.py
-rw-r--r-- 1 root staff  9026 Oct 16 09:13 exceptions.pyc
-rw-r--r-- 1 root staff  8048 Oct 16 09:13 _helpers.py
-rw-r--r-- 1 root staff 10032 Oct 16 09:13 _helpers.pyc
-rw-r--r-- 1 root staff   736 Oct 16 09:13 __init__.py
-rw-r--r-- 1 root staff   333 Oct 16 09:13 __init__.pyc
-rw-r--r-- 1 root staff  4957 Oct 16 09:13 iterator.py
-rw-r--r-- 1 root staff  5210 Oct 16 09:13 iterator.pyc
drwxr-sr-x 2 root staff  4096 Oct 16 09:13 pubsub
drwxr-sr-x 2 root staff  4096 Oct 16 09:13 resource_manager
drwxr-sr-x 2 root staff  4096 Oct 16 09:13 search
drwxr-sr-x 3 root staff  4096 Oct 16 09:13 storage
-rw-r--r-- 1 root staff  7021 Oct 16 09:13 test_client.py
-rw-r--r-- 1 root staff  9101 Oct 16 09:13 test_client.pyc
-rw-r--r-- 1 root staff 13090 Oct 16 09:13 test_connection.py
-rw-r--r-- 1 root staff 15451 Oct 16 09:13 test_connection.pyc
-rw-r--r-- 1 root staff 22622 Oct 16 09:13 test_credentials.py
-rw-r--r-- 1 root staff 28116 Oct 16 09:13 test_credentials.pyc
-rw-r--r-- 1 root staff  2844 Oct 16 09:13 test_exceptions.py
-rw-r--r-- 1 root staff  3664 Oct 16 09:13 test_exceptions.pyc
-rw-r--r-- 1 root staff 12177 Oct 16 09:13 test__helpers.py
-rw-r--r-- 1 root staff 19874 Oct 16 09:13 test__helpers.pyc
-rw-r--r-- 1 root staff  1143 Oct 16 09:13 _testing.py
-rw-r--r-- 1 root staff  1212 Oct 16 09:13 _testing.pyc
-rw-r--r-- 1 root staff  6781 Oct 16 09:13 test_iterator.py
-rw-r--r-- 1 root staff  8076 Oct 16 09:13 test_iterator.pyc

Activity

  1. dhermes commented on Oct 18, 2015

    @dhermes
    dhermes
    on Oct 18, 2015
    Contributor

    I'm not familiar with PyInstaller (I presume it is http://www.pyinstaller.org/).

    Can you elaborate and maybe explain how you expect this to work?

    The problem you describe seems to be a bug in part of PyInstaller. It is a chicken-and-egg problem. The get_distribution() check occurs only after the library is installed. It is used to set the __version__.

  2. mtwu-aviatrix commented on Oct 21, 2015

    @mtwu-aviatrix
    mtwu-aviatrix
    on Oct 21, 2015
    Author

    We have installed quite a few new modules from github lately. This is the first time we bump into the executable build issue. We do expect the module can work with pyinstaller after it is built and installed. I can run the client in the python native mode. We have reinstalled the module and pyinstaller few times without any luck.

  3. mtwu-aviatrix commented on Oct 21, 2015

    @mtwu-aviatrix
    mtwu-aviatrix
    on Oct 21, 2015
    Author

    This issue should be related to the version compatibility between pyinstaller and setuptools. Please advise the correct versions used in your environment which the executable can be built from pyinstaller.

  4. dhermes commented on Oct 23, 2015

    @dhermes
    dhermes
    on Oct 23, 2015
    Contributor

    We don't have any version requirements on setuptools. The latest version should work just fine.

  5. mtwu-aviatrix commented on Oct 27, 2015

    @mtwu-aviatrix
    mtwu-aviatrix
    on Oct 27, 2015
    Author

    Your code has to consider the run-time environment when it is bundled to an executable. In this case, pkg_resources can't work in the bundled executable though it works in native python mode. The following workaround diffs make the executable work.

    diff --git a/gcloud/__init__.py b/gcloud/__init__.py
index 8c1c7b4..e70010a 100644
--- a/gcloud/__init__.py
+++ b/gcloud/__init__.py
@@ -16,4 +16,5 @@

 from pkg_resources import get_distribution

-__version__ = get_distribution('gcloud').version
+#__version__ = get_distribution('gcloud').version
+__version__ = '0.8.0'
diff --git a/gcloud/connection.py b/gcloud/connection.py
index b732113..7650625 100644
--- a/gcloud/connection.py
+++ b/gcloud/connection.py
@@ -63,7 +63,8 @@ class Connection(object):
     :param http: An optional HTTP object to make requests.
     """

-    USER_AGENT = "gcloud-python/{0}".format(get_distribution('gcloud').version)
+#    USER_AGENT = "gcloud-python/{0}".format(get_distribution('gcloud').version)
+    USER_AGENT = "gcloud-python/{0}".format('0.8.0')
     """The user agent for gcloud-python requests."""

     SCOPE = None

  6. dhermes commented on Oct 27, 2015

    @dhermes
    dhermes
    on Oct 27, 2015
    Contributor

    If we were to hardcode it we'd replace get_distribution('gcloud').version with __version__ in gcloud/__init__.py.

    However, we intentionally haven't hardcoded it so we can rely on setuptools (a great part of the Python ecosystem) to get the version from the package info.

    It seems strange to me that a bundler would ignore setuptools, a core part of Python packaging. When was the bundler last updated? setuptools has been the de facto way to distribute packages for years now (@tseaver probably has a better sense of this than I do).

  7. mtwu-aviatrix commented on Oct 27, 2015

    @mtwu-aviatrix
    mtwu-aviatrix
    on Oct 27, 2015
    Author

    I am not sure if the setuptools can help much in this case. An executable has no visibility of the package info. At the setup time, it is a different story. I have used pyinstaller 2.x and 3.0 and they all ended up the same run-time error.

  8. dhermes commented on Oct 29, 2015

    @dhermes
    dhermes
    on Oct 29, 2015
    Contributor

    Yes, I'm saying a binary installer worth it's salt should work seemlessly with setuptools, even while running the packaged executable.

    We may just do a try ... except. It seems pkg_resources imports just fine in your packaged environment but none of the package info. ISTM that pkg_resources just searches sys.path / PYTHONPATH until it finds the distribution

    >>> import pprint
>>> from pkg_resources import get_distribution
>>> distro = get_distribution('gcloud')
>>> pprint.pprint(distro.__dict__)
{'_key': 'gcloud',
 '_provider': <pkg_resources.PathMetadata instance at 0x7f8961021200>,
 '_version': '0.7.1',
 'location': '/usr/local/lib/python2.7/dist-packages',
 'platform': None,
 'precedence': -1,
 'project_name': 'gcloud',
 'py_version': None}
>>> pprint.pprint(distro._provider.__dict__)
{'egg_info': '/usr/local/lib/python2.7/dist-packages/gcloud-0.7.1.dist-info',
 'module_path': '/usr/local/lib/python2.7/dist-packages'}

  9. mtwu-aviatrix commented on Oct 29, 2015

    @mtwu-aviatrix
    mtwu-aviatrix
    on Oct 29, 2015
    Author

    Have you tried to bundle gcloud to an executable file and run the file?

    In the native python interpreter, it always works. Once it is in an executable, can the external path search work?

  10. dhermes commented on Oct 29, 2015

    @dhermes
    dhermes
    on Oct 29, 2015
    Contributor

    No I haven't tried it. I've not had a use for one of these installers and am trying to understand how they work, who uses them and where they use them.

  11. mtwu-aviatrix commented on Oct 29, 2015

    @mtwu-aviatrix
    mtwu-aviatrix
    on Oct 29, 2015
    Author

    We build an executable file to support our solution.

  12. dhermes
    added
    packaging
    on Dec 31, 2015
  13. dhermes
    changed the title [-]pkg_resources.DistributionNotFound: The 'gcloud' distribution was not found[/-] [+]pyinstaller executable can't find package info[/+] on Dec 31, 2015

  14. dhermes commented on Dec 31, 2015

    @dhermes
    dhermes
    on Dec 31, 2015
    Contributor

    @mtwu-aviatrix I am trying to reproduce this error with PyInstaller and am not having luck.

    I made a bogus file (bogus.py):

    from gcloud import datastore
print(datastore)

    and then ran

    $ pyinstaller bogus.py

    without any issues. What am I missing?

  15. dhermes commented on Jan 14, 2016

    @dhermes
    dhermes
    on Jan 14, 2016
    Contributor

    @mtwu-aviatrix Closing this out since I can't reproduce. Feel free to reopen if it's still an issue.

  16. 16 remaining items

  17. bloxofcodes commented on Aug 2, 2018

    @bloxofcodes
    bloxofcodes
    on Aug 2, 2018

    Hi doing datas += copy_metadata('google-api-core') doesnt solve my error about gcloud...what particular steps you do to make it work?

  18. Vonisoa commented on Aug 8, 2018

    @Vonisoa
    Vonisoa
    on Aug 8, 2018

    @bloxofcodes Maybe you should look at C:\Python27\Lib\site-packages\PyInstaller\hooks or somewhere else according to your OS. And find if there is a hook for hook-.py, so in your case hook-google.cloud.py. If it exists but pyinstaller can't find it, you should import it in your code like this:
    import google.cloud,
    but if it doesn't exists, create this hook in the hook directory of Pyinstaller and add this inside:
    from PyInstaller.utils.hooks import copy_metadata
    datas = copy_metadata('google-cloud-core')

  19. Vonisoa commented on Aug 8, 2018

    @Vonisoa
    Vonisoa
    on Aug 8, 2018

    @srkn , thanks for your advice, I will try this and give feedback!

  20. Rahuld753 commented on Apr 18, 2019

    @Rahuld753
    Rahuld753
    on Apr 18, 2019

    You will need to create a pyinstaller hook for packages that depend on setuptools' get_distribution() to obtain metadata information about the package.

    Create a hook file in some hooks directory. The file name should be hook-<modulename>.py for pyinstaller to find it properly (i.e. hook-gcloud.py).

    The content will be:

    from PyInstaller.utils.hooks import copy_metadata
datas = copy_metadata('gcloud')

    The hooks directory should be set in the Analysis phase of your pyinstaller .spec file:

    a = Analysis(['bogus.py'],
             pathex=[],
             binaries=[],
             hiddenimports=[],
             hookspath="/path/to/your/hooks/dir",
             runtime_hooks=[],
             excludes=[],
             win_no_prefer_redirects=False,
             win_private_assemblies=False,
             cipher=block_cipher)

    I haven't tested the above, but this is what worked for me for another python package which is doing something similar (APScheduler).

    If and when you get this working, consider contributing the hook back to pyinstaller project.

  21. murtuzamdahod commented on May 21, 2019

    @murtuzamdahod
    murtuzamdahod
    on May 21, 2019

    @Shoshin23 I solved this problem by editing pyinstaller original hook-google.cloud.py file. I simply added this line, you can change it according your needs
    datas += copy_metadata('google-api-core')

    Thanks alot this worked for me :)

  22. murtuzamdahod commented on May 21, 2019

    @murtuzamdahod
    murtuzamdahod
    on May 21, 2019

    @srkn , thanks for your answer, it solve my problem. Now I have another trouble when running the .exe

    Exception in 'grpc._cython.cygrpc.ssl_roots_override_callback' ignored E0731 16:52:01.684000000 13212 src/core/lib/security/security_connector/security _connector.cc:1173] assertion failed: pem_root_certs != nullptr
    I don't know what can produce this error!!!

    first of all find the root.pem file from gcrp package then copy the file to your cwd and build using --add-data "root.pem:/grpc/_cython/_credentials/"

  23. JustinBeckwith
    assigned
    dhermes
    on Feb 1, 2021
  24. parthea
    added 3 commits that reference this issue on Nov 24, 2025

    fix: disambiguate method names that are reserved in transport classes (…

    0744ed5

    chore(main): release 0.63.1 (#1189)

    6e54ace

    chore(python): remove noxfile.py from templates (#1187)

    b33b52c
  25. parthea
    added 4 commits that reference this issue on Mar 2, 2026

    feat: add data model for client side metrics (#1187)

    db8d528

    fix(deps): restore dependency on python-dateutil (#1187)

    9a70b63

    chore(main): release 3.0.1 (#1189)

    ba8af86

    feat: Add support for Python 3.12 (#1187)

    d09b381
  26. parthea
    added a commit that references this issue on Apr 2, 2026

    chore: librarian release pull request: 20260402T161941Z (#16539)

    79a0651
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

packaging

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions