Repository navigation
pip's vendored pkg_resources should stop using pkgutil.ImpImporter #11501
Description
Description
pkgutil.ImpImporter has been deprecated since Python 3.3. It is slated to be removed in Python 3.12 but until pip no longer uses it, we probably can't remove this long deprecated class.
Expected behavior
No response
pip version
22.2.2
Python version
3.12
OS
macOS
How to Reproduce
- Grab the CPython branch:
gh pr checkout 98059 - configure && make test
test_venvfails becausepipuses this API- See https://github.com/python/cpython/actions/runs/3215689824/jobs/5256965254
Output
No response
Code of Conduct
- I agree to follow the PSF Code of Conduct.
Activity
More cleanup
warsaw commentedon Oct 9, 2022
pipalso needs to remove its reference toimporter.find_module()as that API is also slated for removal.pradyunsg commentedon Oct 10, 2022
Thanks for filing this @warsaw!
This module is under
_vendor, so... it comes from a different project that pip uses. Looking into this, they're still referenced in upstream setuptools:I believe this needs to get fixed there, and we'll pick up the fix once it's made there. :)
pradyunsg commentedon Oct 10, 2022
@warsaw Could you file an issue against https://github.com/pypa/setuptools for this?
pradyunsg commentedon Oct 10, 2022
/cc @jaraco for visibility, since setuptools "owns" pkg_resources and would need to make the relevant changes on this front.
pradyunsg commentedon Oct 10, 2022
/cc @pfmoore for awareness as the current RM -- to be clear, I don't know this affects any of the plans for the current release. I'm flagging this for awareness since there is, maybe, a chance that there's a setuptools release after we make this time's round of updates and before the release, in which case we might want to run another vendoring update. :)
warsaw commentedon Oct 10, 2022
Thanks @pradyunsg; I'll do this, although I'm not sure how to figure out which version of
setuptoolspipis currently vendoring. I'll just file this against the latest, i.e. 65.4.1warsaw commentedon Oct 10, 2022
See the referenced
setuptoolsissue above. And thanks for working on this!pradyunsg commentedon Oct 10, 2022
So... this issue made me realise that I'd broken our update logic in a subtle manner, which means we're on a ~2 year old version.
pip/src/pip/_vendor/vendor.txt
Line 19 in 8f381cd
#11500 (comment) is where I noticed that. I'll be trying to bump to the latest setuptools version in the next pip release, as a part of #11502 (or a follow up); assuming it's tractable in the limited free time that I have. :)
uranusjr commentedon Oct 10, 2022
Since we are already moving from pkg_resources to importlib.metadata, we can make Python 3.12 our cutoff to switch entirely to the latter backend. This would mean that we can simply keep pkg_resources as-is since those usages will only be accessed on earlier Python versions.
23 remaining items
uranusjr commentedon May 25, 2023
This looks like a setuptools issue. pip does not import setuptools unless the package tells it to, so this is something you should report to either setuptools or the packages using it and triggering the error.
abravalheri commentedon May 25, 2023
I suspect that
setuptoolshas already fixed problems withpkgutil.ImpImporterin v66.1.0.To make sure the package builds on Python 3.12, the best would be to use a relatively recent version of setuptools. This is probably specified in the
pyproject.tomlfile of the projects you are trying to build.bersbersbers commentedon May 25, 2023
Ah, this seems to be the key thing, because I have setuptools 67.8.0 installed but that is irrelevant.
So this explains why
numpydoes not install:https://github.com/numpy/numpy/blob/cc0abd768575d7f9e862de0b4912af27f6e9690d/pyproject.toml#LL7C6-L7C16
But this does not explain why
matplotlibdoes not install, does it?https://github.com/matplotlib/matplotlib/blob/7b821ce6b4a4474b1013435954c05c63574e3814/pyproject.toml#L7
abravalheri commentedon May 25, 2023
I am not sure how the dependency resolution algorithm works during build time, but matplotlib seems to depend on numpy, there might be a chance the same environment is being used to build both and then the same version of setuptools would be used?
@uranusjr might be more familiar with the dependency resolution at this stage...
bersbersbers commentedon May 25, 2023
You are right - even at build time:
https://github.com/matplotlib/matplotlib/blob/7b821ce6b4a4474b1013435954c05c63574e3814/pyproject.toml#LL5C29-L5C29
Okay, that explains that as well. I guess I will open an issue with numpy then.
pradyunsg commentedon May 30, 2023
With #11997 landed, is there anything more needed here
other than a pip release(and it's in 23.1.2)?Increase setuptools version above 66.1 see pypa/pip#11501
moving goalposts
py312: Tell tox/virtualenv to upgrade pip
Increase setuptools version above 66.1 see pypa/pip#11501
Increase setuptools version above 66.1 see pypa/pip#11501
uranusjr commentedon Jul 18, 2023
We already fixed this in pip 23.2 actually (since the bundled
pkg_resourceshas been upgraded to 67.7.1). The ensurepip upgrade will happen soon.(Edit: the bundled pip in CPython has since been updated to 23.2.1 on 2023-07-25. See python/cpython@f443b54)