Repository navigation
Raise a warning when pip falls back to the "legacy" direct install code #8102
Description
What's the problem this feature will solve?
The legacy code (which calls setup.py install directly) relies on setuptools to generate entry point wrappers. The wrappers setuptools generates require pkg_resources at runtime, and are reported as being very slow (see pypa/setuptools#510 (comment) and the discussion leading to it).
Having pip warn in this case would be helpful in diagnosing the subtle behaviour change when wheel isn't present.
Describe the solution you'd like
When pip uses the setup.py install legacy code path, issue a warning pointing out that because the wheel library isn't present, pip is running setup.py install directly, so old-style script wrappers will be generated.
Alternative Solutions
The solution is to install wheel, but this is not obvious from the information pip currently provides.
Additional context
The warning has the potential to be "noisy", warning users when they don't care about the issue it's describing. But the legacy code path is just that - legacy - and ultimately we'd like to remove it. Warning people that they are inadvertently using a legacy code path prepares them for that possibility.
Conversely, the requirement for wheel to be installed is not well-managed currently (the stdlib venv doesn't install wheel by default, people don't typically realise that they should install wheel when just doing pip install ., etc. The proposed warning will give the requirement more visibility, which might help trigger a more useful discussion on how to manage the requirement for wheel.
Longer term the solution will be to switch fully to PEP 517 processing, but that is likely to be some time off yet.
Activity
marcelm commentedon Apr 21, 2020
The answer may be obvious, but I’ve often wondered: As far as I understand, quite a few libraries are already vendored with pip, why isn’t wheel simply one of them?
pfmoore commentedon Apr 21, 2020
Basically, because pip doesn't itself use
wheel, it's the build subprocess that we call that needs wheel. Vendoring doesn't help in that situation. (Well, it may be that we could make it work somehow, but this issue is obsoleted bypyproject.toml, so there's a better solution for new projects).pradyunsg commentedon Apr 21, 2020
This looks similar to #7709, which got fixed in #7768, which is in pip 20.1b1.
sbidoul commentedon Apr 21, 2020
+1 for this.
Do we want to start the deprecation process now (and collect feedback in a deprecation issue), or simply raise awareness with louder messages?
We could raise the level from info to warning in
_should_build, and/or allude to a future deprecation:pip/src/pip/_internal/wheel_builder.py
Lines 72 to 78 in 814c54f
pip will also fallback to legacy install when wheel building failed for other reasons, so we may want to warn there also, and encourage people to make sure that
setup.py bdist_wheelworks for their packages:pip/src/pip/_internal/commands/install.py
Lines 368 to 370 in 814c54f
pradyunsg commentedon Apr 21, 2020
I'm on board for this, though we'd need to decide on how long this should be.
pfmoore commentedon Apr 21, 2020
+1 from me. Part of me wants to say 21.0, to match the Python 2 desupport, because that makes it a single breakpoint for users (the "desupport everything" release 🙂).
The converse argument is that users won't want to have to deal with 2 different issues in the same timeframe, but my instinct is that the affected groups of users are likely pretty disjoint (and/or the same solution, pin or upgrade your project, fixes both issues at the same time).
pradyunsg commentedon Apr 21, 2020
I'm glad I didn't say this first. :)
I'll flag that the sheer amount of traffic on PEP 517 and projects that can't install via wheels (~46k views, most viewed) tells me that there are lots of users who use the direct "setup.py install" code path.
The next topic in terms of number of views is something that, IIRC, made it to the front page of Hackernews, had to be closed by moderators, and THAT has just about half the number of views: https://discuss.python.org/top/all.
Thinking about this more, I'm uncomfortable suggesting "let's warn/deprecate now" without us spending some time checking if we can do something better in overall "PEP 517/PEP 518/build isolation" situation, which is strongly coupled to this IMO.
I really don't have a good feel for what is a reasonable approach/strategy for eventually transitioning off the legacy build/install logic would be, and it feels like we should talk about the broader problem first before discussing the specifics here.
sbidoul commentedon Apr 21, 2020
OTOH, our deprecation mechanism, combined with a friendly message, is a nice way to ask concerned users to report feedback in a GitHub issue. Based on that feedback we can still decide not to deprecate at the planned date or otherwise adjust the strategy.
Also, in my understanding, deprecating the
setup.py installcode path is "softer" that enforcing build isolation.pfmoore commentedon Apr 21, 2020
Wait - what I thought we were discussing the deprecation of was falling through to
setup.py install. All that would involve would saying something like "Cannot build wheel for legacy install, please install the wheel project or switch topyproject.toml". And converting that to a hard error and ripping out the legacysetup.py installcode 🙂 once we get to 20.1.Forcing everything to use PEP 517/518/build isolation is a much bigger issue, and I agree that for that, we need to disentangle some of the complexities around isolation,
--no-binary, etc.67 remaining items
pradyunsg commentedon Sep 17, 2022
I don't have a good sense of how much churn this would cause for users.
The intuition I have for adding
no-deprecatedflags is basically whether we expect to get users who will show up with a pitchfork and demand that we give them that functionality back.pfmoore commentedon Sep 17, 2022
My intuition says that we'll get people complaining, but they will be the sort of people who aren't looking for a transition mechanism, but rather for a "make it work again" flag. So they'll simply complain again when the
--use-deprecatedflag is removed.So I'm fine with not adding a
--use-deprecatedoption. Addingpyproject.tomlis straightforward, and if users can't do that, and can't fix the underlying issue that makes it impossible for them to do that within the deprecation period, and have ignored everything that's been said in various channels about looking at moving topyproject.tomlfor ages now, then I'm happy saying that they should build and publish wheels manually as a workaround, or advise their users to pin pip until they work out a better fix.If during the deprecation period we get anyone raising a serious and genuinely intractable issue, we can review and if necessary extend the deprecation period. If they don't engage with us until it's too late, well, I guess it's too late 🤷
sbidoul commentedon Sep 18, 2022
Agreed.
I think users who may be most impacted are application authors who are (perhaps unknowingly) on the setup.py install code path and rely on (unmaintained) libraries which install successfully but don't support
bdist_wheelfor some reason.They might be using setup.py install simply because they created a virtual environment with
python -m venvand don't havewheelinstalled.pradyunsg commentedon Sep 18, 2022
Hmm... I wonder if we should extend the isolation enabling logic, from #10717 to include wheel -- i.e. use isolated pyproject.toml builds if the environment does not have both setuptools and wheel installed. That's one way to avoid falling back to
setup.py installin plainpython -m venvenvironments -- before we actually make the changes in Python 3.12 for python/cpython#95299.pradyunsg commentedon Oct 7, 2022
Actually, here's a shower thought...
It might make sense to entirely disable both
setup.py install(this issue) andsetup.py develop(#11457) on Python 3.12+. There's currently no users who are on that version -- such a change wouldn't be disruptive and, instead, it would be a minor hassle for performing the 3.12 upgrade. To be clear, I don't think we should do this forsetup.py bdist_wheeljust yet, since there's a few more pieces to change around that (#9175).This could render some packages unusable on Python 3.12+, but I reckon that would be acceptable -- if they are actively maintained, the maintainers should be able to update the build system to behave correctly on newer versions of Python.
How do people feel about this, and do you reckon this is a good idea?
sbidoul commentedon Oct 7, 2022
@pradyunsg does that imply keeping support for setup.py install and setup.py develop until we drop support for Python 3.11?
pradyunsg commentedon Oct 7, 2022
No.
I'm thinking of it as putting a maximum limit of when it will definitely be dead -- when 3.11 support is dropped in ~5-6 years.
We could still drop support for these mechanisms on older versions of Python before then, but it certainly won't be a problem after 6 years.
sbidoul commentedon Oct 7, 2022
In any case, we have currently merged
gone_in=23.1for setup.py install :)pradyunsg commentedon Oct 7, 2022
Assuming that works out, this only applies to the setup.py develop case then. :)
Let's move the discussion there.
Make version name compliant with PEP440
pradyunsg commentedon Mar 1, 2023
@sbidoul Would you have time/energy to file follow ups removing the relevant logic in this case? I’m down to do so, if you don’t have the bandwidth.
sbidoul commentedon Mar 1, 2023
Hi @pradyunsg, yes I'm happy to do it. I'm kind of planning to work on this around mid/end of March.
pradyunsg commentedon Mar 1, 2023
Awesome! I'm looking forward to seeing this happen! ❤️