Repository navigation
Python 3.12-dev & Windows: zipimporter has no attribute find_module #11688
Description
Description
The vendored version of pkg_resources in pip fails with Python 3.12-dev on Windows.
Expected behavior
No response
pip version
22.3.1
Python version
3.12.0-alpha.3 (x64)
OS
Windows 2022 & 2019
How to Reproduce
Tl;Dr, you just have to run in Windows with Python 3.12:
$ pip download --no-binary=:all: --no-deps --dest ./local-download/ pygments==2.14.0
The long version is that I am emulating the CLI above in my code, by mimicking some of pip's internals.
That way I can download the source distribution of the Pygments version I depends on, and inspect its test data files which are not distributed as part of the wheel.
This construction is working perfectly on the full matrix of:
- Distributions:
ubuntu-22.04ubuntu-20.04macos-12macos-11windows-2022windows-2019
- Python versions:
3.73.83.93.103.113.12-dev
But not on the combination of 3.12-dev with either windows-2022 or windows-2019.
See the result of this matrix execution here, and the exact trace here.
Output
C:\(...)\pip\_internal\cli\req_command.py:247: in wrapper
return func(self, options, args)
C:\(...)\pip\_internal\commands\download.py:93: in run
session = self.get_default_session(options)
C:\(...)\pip\_internal\cli\req_command.py:98: in get_default_session
self._session = self.enter_context(self._build_session(options))
C:\(...)\pip\_internal\cli\req_command.py:125: in _build_session
session = PipSession(
C:\(...)\pip\_internal\network\session.py:343: in __init__
self.headers["User-Agent"] = user_agent()
C:\(...)\pip\_internal\network\session.py:175: in user_agent
setuptools_dist = get_default_environment().get_distribution("setuptools")
C:\(...)\pip\_internal\metadata\importlib\_envs.py:188: in get_distribution
return next(matches, None)
C:\(...)\pip\_internal\metadata\importlib\_envs.py:183: in <genexpr>
matches = (
C:\(...)\pip\_internal\metadata\base.py:612: in iter_all_distributions
for dist in self._iter_distributions():
C:\(...)\pip\_internal\metadata\importlib\_envs.py:176: in _iter_distributions
for dist in finder.find_eggs(location):
C:\(...)\pip\_internal\metadata\importlib\_envs.py:144: in find_eggs
yield from self._find_eggs_in_dir(location)
C:\(...)\pip\_internal\metadata\importlib\_envs.py:111: in _find_eggs_in_dir
from pip._vendor.pkg_resources import find_distributions
C:\(...)\pip\_vendor\pkg_resources\__init__.py:3251: in <module>
@_call_aside
C:\(...)\pip\_vendor\pkg_resources\__init__.py:3235: in _call_aside
f(*args, **kwargs)
C:\(...)\pip\_vendor\pkg_resources\__init__.py:3277: in _initialize_master_working_set
tuple(
C:\(...)\pip\_vendor\pkg_resources\__init__.py:3278: in <genexpr>
dist.activate(replace=False)
C:\(...)\pip\_vendor\pkg_resources\__init__.py:2785: in activate
declare_namespace(pkg)
C:\(...)\pip\_vendor\pkg_resources\__init__.py:2284: in declare_namespace
_handle_ns(packageName, path_item)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
def _handle_ns(packageName, path_item):
"""Ensure that named package includes a subpath of path_item (if needed)"""
importer = get_importer(path_item)
if importer is None:
return None
# capture warnings due to #1111
with warnings.catch_warnings():
warnings.simplefilter("ignore")
> loader = importer.find_module(packageName)
E AttributeError: 'zipimporter' object has no attribute 'find_module'. Did you mean: 'load_module'?
C:\(...)\pip\_vendor\pkg_resources\__init__.py:2201: AttributeErrorCode of Conduct
- I agree to follow the PSF Code of Conduct.
Activity
Allow tests to fail on Windows with Python 3.12-dev.
kdeldycke commentedon Jan 2, 2023
See the general deprecation issues of
importlibAPIs for Python 3.12:find_loaderandget_loaderinpkgutilpython/cpython#98520pfmoore commentedon Jan 2, 2023
Thanks for reporting this. Normally we'd pick up the fix when we update our vendored copy of pkg_resources. But we pin to a very old version of setuptools, so that won't happen automatically - this pin was added in #9170 but there's no information I can find as to why we don't keep pkg_resources up to date. @pradyunsg do you remember what the issue was there?
So we need to do something at our end to address this, once the setuptools fix is landed.
uranusjr commentedon Jan 2, 2023
On 3.12 we shouldn’t use pkg_resources, but importlib.metadata. So I’m guessing something made a distribution-wise override (by vendor-patching or environment variable). See also #11661.
pip/src/pip/_internal/metadata/__init__.py
Lines 29 to 49 in ba38c33
uranusjr commentedon Jan 2, 2023
Duplicate of #11501
kdeldycke commentedon Jan 3, 2023
Thanks @uranusjr for confirming the vendored
pkg_resourcesis indeed imposed by thesetuptoolsdependency. I stumble upon #11501 looking at the issue backlog but wasn't certain it was the right place to add my own observations. I should have mentioned it. Thanks again for making things clear! :)uranusjr commentedon Jan 3, 2023
@kdeldycke It would be a good idea to investigate why your copy of pip is trying to import
pkg_resourceson 3.12. It shouldn’t unless someone tinkered with the installation.