Skip to content

Make setuptools optional at runtime. - #8476

Merged
sklam merged 5 commits into
numba:mainfrom
stuartarchibald:wip/setuptools_optional_runtime
Mar 23, 2023
Merged

sklam merged 5 commits into
numba:mainfrom
stuartarchibald:wip/setuptools_optional_runtime

Conversation

@stuartarchibald

Copy link
Copy Markdown
Contributor

This is a mechanical change trying to move stdlib distutils use onto setuptools.distutils.

This is a mechanical change trying to move stdlib distutils use
onto setuptools.distutils.
@gmarkall gmarkall added 3 - Ready for Review Pending BuildFarm For PRs that have been reviewed but pending a push through our buildfarm 2 - In Progress and removed 2 - In Progress 3 - Ready for Review Pending BuildFarm For PRs that have been reviewed but pending a push through our buildfarm labels Oct 3, 2022
Comment thread numba/pycc/cc.py Outdated
Comment on lines +1 to +3
from setuptools import distutils as dutils
dir_util = dutils.dir_util
log = dutils.log

@flying-sheep flying-sheep Oct 11, 2022 •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why not like this?

Suggested change
from setuptools import distutils as dutils
dir_util = dutils.dir_util
log = dutils.log
from setuptools.distutils import dir_util, log

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for taking a look. The issue is:

>>> from setuptools.distutils import dir_util, log
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'setuptools.distutils'

@flying-sheep flying-sheep Oct 12, 2022 •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see, it’s a not really a submodule, it’s a re-export.

But since setuptools just directly imports distutils, what’s the advantage of importing it from there instead of just doing import distutils?

If the intention is to to not import distutils because it’s deprecated, you need to actually remove the import instead of importing the same thing using a more roundabout path. Should be easy enough, there doesn’t seem to be any good reason to import either symbol from it: log is just a logging.Logger (you can just use another logger I guess) and dir_util.mkpath(...) provides the same functionality as pathlib.Path(...).mkdir(parents=True, exists_ok=True)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are good points, thanks for raising them.

I see, it’s a not really a submodule, it’s a re-export.

But since setuptools just directly imports distutils, what’s the advantage of importing it from there instead of just doing import distutils?

I was looking to use the setuptools vendored distutils but think I have done this incorrectly. It's probably setuptools._distutils.

If the intention is to to not import distutils because it’s deprecated, you need to actually remove the import instead of importing the same thing using a more roundabout path. Should be easy enough, there doesn’t seem to be any good reason to import either symbol from it: log is just a logging.Logger (you can just use another logger I guess) and dir_util.mkpath(...) provides the same functionality as pathlib.Path(...).mkdir(parents=True, exists_ok=True)

This patch is trying to do the absolute minimum necessary to make pycc not use distutils and instead use setuptools's internal copy, and then to make sure that setuptools an optional runtime dependency. Given pycc is scheduled for deprecation in the next Numba release (xref: #8509), I'm not sure that it is worth spending time on finding a full replacement for distutils, then writing and testing it, though can probably make the changes RE: log and mkpath as noted.

@github-actions

Copy link
Copy Markdown

This pull request is marked as stale as it has had no activity in the past 3 months. Please respond to this comment if you're still interested in working on this. Many thanks!

@github-actions github-actions Bot added the stale Marker label for stale issues. label Feb 16, 2023
@flying-sheep

flying-sheep commented Feb 16, 2023 •

Copy link
Copy Markdown
Contributor

Feels like this was simply forgotten by the maintainers. @stuartarchibald is it ready?

@github-actions github-actions Bot removed the stale Marker label for stale issues. label Feb 17, 2023
@stuartarchibald

Copy link
Copy Markdown
Contributor Author

Feels like this was simply forgotten by the maintainers. @stuartarchibald is it ready?

@flying-sheep thanks for raising this. It's on the "housekeeping" list for Numba 0.57 (#8762). It's a relatively isolated change and perhaps a lower risk item so hasn't been a high priority whilst other more concerning things have been de-risked e.g. Python 3.11 support. Will hopefully get to this soon!

@jakirkham

Copy link
Copy Markdown
Contributor

After having experienced some pain in conda-forge around Setuptools 66, would just like to voice my support for this and any other change that makes us less reliant on Setuptools

@sklam

sklam commented Mar 8, 2023

Copy link
Copy Markdown
Member

I've manually tested this on py3.11 branch. Looks ok.

@numba numba deleted a comment from azure-pipelines Bot Mar 21, 2023
@sklam

sklam commented Mar 21, 2023

Copy link
Copy Markdown
Member

This patch now conflicts with main due to the py3.11 branch merge.

Resolved conflicts in:
	numba/pycc/__init__.py
	numba/pycc/platform.py
	numba/tests/support.py
	numba/tests/test_pycc.py
@stuartarchibald

Copy link
Copy Markdown
Contributor Author

This patch now conflicts with main due to the py3.11 branch merge.

ce5aa59 resolves.

@sklam sklam left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just one style issue

Comment thread numba/pycc/cc.py Outdated
Comment on lines +2 to +3
dir_util = dutils.dir_util
log = dutils.log

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion: move the assignments below all import for flake8

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 4c3948b.

@stuartarchibald
stuartarchibald marked this pull request as ready for review March 23, 2023 09:59
@stuartarchibald

Copy link
Copy Markdown
Contributor Author

I've marked this as ready for review, this is with view of getting the code changes merged. Will open another PR that's built on top of this that handles packaging, CI, docs etc.

@sklam sklam left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code changes look good. As noted in previous comment by OP, non-code changes will go in a different PR.

@sklam sklam added 5 - Ready to merge Review and testing done, is ready to merge and removed 3 - Ready for Review labels Mar 23, 2023
@sklam sklam added this to the Numba 0.57 RC milestone Mar 23, 2023
@stuartarchibald stuartarchibald added the Effort - medium Medium size effort needed label Mar 23, 2023
@sklam
sklam merged commit eac7235 into numba:main Mar 23, 2023
@jakirkham

Copy link
Copy Markdown
Contributor

Thanks all! 🙏

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

5 - Ready to merge Review and testing done, is ready to merge Effort - medium Medium size effort needed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants