Skip to content

Sync with distutils @ e8eb87855 - #5292

Merged
jaraco merged 145 commits into
mainfrom
distutils-e8eb87855
Aug 6, 2026
Merged

jaraco merged 145 commits into
mainfrom
distutils-e8eb87855

Conversation

@jaraco

@jaraco jaraco commented Aug 6, 2026

Copy link
Copy Markdown
Member

Sync with distutils @ e8eb878, completing Phase 1 of the compilers decoupling (#5264): the C compilers no longer reach into distutils for build/filesystem helpers, exceptions, version/platform utilities, or sysconfig, and the compiler registry and tests are decoupled.

Summary of changes

e8eb87855 Compiler registry via get_compilers; decouple compiler tests from distutils (pypa/distutils#423)
1395af8b2 Decouple the compilers from distutils.sysconfig (configure_system) (pypa/distutils#422)
b5c8f4d73 Decouple the compilers from distutils version/platform helpers (pypa/distutils#421)
ff535b2be Give the compilers a native exception hierarchy (compilers.errors) (pypa/distutils#420)
7f67e1c8e Decouple the C compilers from distutils build/filesystem helpers (pypa/distutils#417)
1c66bcd33 👹 Feed the hobgoblins (delint).
ecda79a88 Use module-level named loggers in the C compiler modules (pypa/distutils#416)
88cb39abd Emit -O1 instead of bare -O in the Cygwin and MinGW compilers (pypa/distutils#415)
d636f5f8a Use float value of ST_MTIME in copy_file function (pypa/distutils#379)
96dfa1323 Fix documentation grammar, typos, and formatting (pypa/distutils#397)
ebfb2eb6e Use dataclass to add type annotations to `Extension` so that it can easily be extended in `setuptools`. (pypa/distutils#373)
b8672ad6f Enforce ruff/refurb rules (FURB) (pypa/distutils#403)
01934af95 Remove Python 3.9 compatibility (pypa/distutils#402)
02430c54b Address ruff issues (pypa/distutils#371)
4b078db13 Handle free-threaded builds in build_ext.get_libraries (pypa/distutils#389)
2310e5d31 mypy: fix all assignment (pypa/distutils#368)
5d4dcbd8d mypy: fix all misc (pypa/distutils#364)
b0805982b mypy: Fix all union-attr (pypa/distutils#366)
06c886383 type `Command.force` (pypa/distutils#382)
4a80c90e9 mypy: fix all call-overload (pypa/distutils#367)
60ff33205 👹 Feed the hobgoblins (delint).
62171912a mypy: Fix all index (pypa/distutils#363)
56f0a5b4e mypy: Fix all str-bytes-safe (pypa/distutils#365)
12a5390a0 mypy: Remove all-fixed `func-returns-value` and `has-type` from exclusions (pypa/distutils#362)
76337674d Improve distutils.dist.Distribution from typeshed's setuptools.setup (pypa/distutils#372)
0e2786bc8 fix msvc linker bug for huge command line arguments (pypa/distutils#406)
8f341ca80 Fixed build_ext assumption that cross-compilation on Windows always uses MSVCCompiler (pypa/distutils#399)
56d23a351 Honor but deprecate newlines when processing keywords/platforms (pypa/distutils#347)
578b9fc48 Merge pull request pypa/distutils#409 from pypa/feature/util-anystr-typevar
94b8e14c2 Merge pull request pypa/distutils#392 from Avasam/ci_setuptools--Skip-tests-that-distutils-has-no-effect-on
12dc775f7 Remove news fragments; merged downstream.

frenzymadness and others added 30 commits March 19, 2025 11:05
Newlines in `keywords` or `platforms` can break
the produced metadata in PKG-INFO or METADATA files.

Fixes: #4887
This updates build_ext.get_libraries to handle free-threaded builds
of Python when using non-MSVC compilers.
Co-authored-by: Isuru Fernando <isuruf@gmail.com>
…hange_root

typing.AnyStr is deprecated (PEP 695 era) and slated for removal. Use an
explicit TypeVar constrained to (str, bytes) to preserve the same typing
behavior for change_root.

Originally proposed by @sobolevn in #5232, where distutils is
vendored; making the change here so it persists through the next sync.

Co-Authored-By: sobolevn <46602750+sobolevn@users.noreply.github.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…-tests-that-distutils-has-no-effect-on

ci_setuptools: Skip tests that distutils has no effect on
Rather than silently joining newline-separated items with spaces, treat
each line as a separate item (replacing newlines with commas) and warn
that the behavior is deprecated, since newlines were never a valid
separator for these fields. Extract the repair into a documented helper.

Ref #4887

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Emit a plain (UserWarning) rather than DeprecationWarning, which is
  suppressed by default and wouldn't reach setuptools users (per the
  distutils.log.Log precedent).
- Drop empty items so consecutive/leading/trailing newlines don't create
  blank keywords; cover this in the tests.
- Reframe the news and docstring as forgiving invalid input rather than
  implying newlines followed any specification.
Replace the hand-rolled comma split/strip/drop-empties with the reusable
jaraco.text.SeparatedValues, promoting jaraco.text to a runtime dependency.
_repair_newlines still handles the invalid-newline repair and deprecation.
jaraco and others added 24 commits July 26, 2026 10:10
Drop the macOS deployment-target injection (_inject_macos_ver) and the
shutil.which PATH resolution. subprocess.check_call searches PATH itself,
and the macOS injection only ever mattered for compiler/linker
invocations, which now carry it in the compilers package. spawn now just
logs the command, forwards cmd and **kwargs to check_call, and maps
OSError/CalledProcessError to DistutilsExecError. Command.spawn no longer
forwards the now-defunct search_path.

Ref #5267.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Introduce Compiler.call as the modern subprocess entry point: it logs,
injects the macOS deployment target (now compilers.platform.macos._inject_ver,
moved out of distutils.spawn), and calls check_call, letting native
subprocess exceptions propagate. Compiler.spawn becomes a deprecation
shim that translates them to DistutilsExecError (imported late).

Internal call sites across unix/cygwin/zos/msvc move to call() and catch
subprocess.CalledProcessError/OSError; MSVC's PATH injection moves from a
spawn override to a call override. Eliminates the vendored compilers.spawn
module.

Ref #5267.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Import the macos module (from ..platform import macos) so the injection
reads as macos._inject_ver at the call site. Extract the subprocess ->
DistutilsExecError translation into a _translate_errors context manager
in distutils.spawn, shared by distutils.spawn.spawn and the deprecated
Compiler.spawn (imported late).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Updates style to match Ruff 0.16 rules.
# Conflicts:
#	distutils/compilers/C/msvc.py
#	distutils/compilers/C/zos.py
- Drop the unittest.mock import left orphaned in test_msvc by the merge
  with the delint commit (F401).
- Collapse a now-short self.call(...) line in unix.py (ruff format).
- Make compilers._modified._Path a proper TypeAlias so mypy accepts it
  as a type.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Split the compiler exceptions so the language-agnostic ones (Error,
UnknownFileType, and a new PlatformError) live at compilers.errors while
the C/C++-specific CompileError/LinkError/LibError/PreprocessError stay
in compilers.C.errors (subclassing compilers.errors.Error).

The compilers stop borrowing distutils' framework errors: they raise
compilers.errors.PlatformError in place of DistutilsPlatformError/
DistutilsModuleError, and compilers._modified raises the stdlib
FileNotFoundError. distutils.errors keeps its own framework exceptions
unchanged and re-exports the compiler ones for backward compatibility;
since CCompilerError is compilers.errors.Error, distutils' top-level
except (DistutilsError, CCompilerError) handlers still catch the new
PlatformError.

Ref #5270.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The compilers now raise compilers.errors.PlatformError instead of
distutils.errors.DistutilsPlatformError, an incompatible interface
change, so the changelog entry belongs under removals.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Replace the deprecated distutils.version.LooseVersion (and its
suppress_known_deprecation guard) with packaging.version.Version in the
deprecated gcc_version property, dropping the distutils.version coupling.

Ref #5268.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Move the -Wl linker-arg consolidation helper out of distutils.compat and
into the compilers package, dropping unix.py's distutils.compat coupling.

Ref #5268.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Bring the macOS deployment-target logic (get_macosx_target_ver and
friends) and compiler_fixup into compilers.platform.macos, so the
compilers no longer reach into distutils.util / distutils._macos_compat
for them. compiler_fixup wraps stdlib _osx_support; the deployment-target
mismatch now raises compilers.errors.PlatformError (was
DistutilsPlatformError). sysconfig lookups still go through distutils
pending the sysconfig decoupling (#5269).

Ref #5268.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Move get_platform/get_host_platform/is_mingw out of distutils.util into
compilers.platform.detect (get_host_platform delegating to stdlib
sysconfig), and point base.py/msvc.py at them. Completes removing the
compilers' distutils.util/version/compat/_macos_compat couplings.

Ref #5268.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…lers

Rather than keeping duplicate copies, make the compilers package the
single home for the platform, macOS, and linker-arg helpers and have
distutils re-export them:

- distutils.util re-exports get_platform/get_host_platform/is_mingw from
  compilers.platform.detect and the macOS deployment-target helpers from
  compilers.platform.macos (removing the duplicated definitions).
- consolidate_linker_args is renamed _consolidate_linker_args (private,
  with an intent docstring replacing the stale "retain until" note) and
  removed from distutils.compat, which no longer uses it.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
consolidate_linker_args existed only to temporarily restore the old
single-string rpath return after pypa/distutils#214 split it into a list,
giving downstream projects (mpi4py et al.) a window to adopt list
support. The "retain until 2025-04-31" window has long passed, so drop
the shim entirely and have UnixCCompiler.runtime_library_dir_option
return the ["-Wl,--enable-new-dtags", "-Wl,-rpath,<dir>"] list directly
for GNU ld.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Read the build configuration from the standard library's sysconfig
instead of distutils.sysconfig (whose get_config_vars/get_config_h_filename
already just forward to stdlib), and move compiler customization into an
OO Compiler.configure_system() method:

- base.Compiler.configure_system() is a no-op.
- unix.Compiler.configure_system() carries the former customize_compiler
  body (reading CC/CXX/CFLAGS/LDSHARED/... with env overrides + macOS
  fixups via compilers.platform.macos, then set_executables).
- MinGW32Compiler overrides it to apply only when running under mingw.
- zos calls self.configure_system() instead of sysconfig.customize_compiler.

distutils.sysconfig.customize_compiler becomes a thin wrapper that calls
compiler.configure_system() (retained for build_ext/build_clib/config and
setuptools), and its now-unused _customize_macos/_get_str_config_vars/
_add_flags helpers are removed. zos's python_build check uses stdlib
sysconfig.is_python_build(). Tests updated to patch stdlib sysconfig.

Ref #5269.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Replace the static compiler_class registry with get_compilers(), which
imports the concrete compiler modules and collects the non-abstract
Compiler subclasses into a {compiler_type: class} map. Each compiler now
carries its own short name (compiler_type) and description as class
attributes (zos gains its own compiler_type='zos' instead of inheriting
unix's). new_compiler resolves classes through get_compilers, dropping
the dynamic distutils.<module> import.

show_compilers is a CLI concern, not a compilers-package one, so it moves
to distutils.ccompiler (built from get_compilers + FancyGetopt).
compiler_class is retained in distutils.compat.numpy for older numpy.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Remove all distutils-package imports from the C compiler tests so they
can run standalone (in preparation for extracting compilers.C):

- Drop distutils.tests.support.TempdirManager in favor of pytest's
  tmp_path (test_cygwin, test_unix).
- Import errors/platform/util helpers from the compilers package
  (compilers.errors.PlatformError, compilers.platform.detect.get_platform/
  is_mingw, compilers._util.split_quoted) instead of distutils.
- Use the compilers' own cygwin.Compiler instead of the
  distutils.cygwinccompiler shim; read config vars from stdlib sysconfig.
- Call compiler.configure_system() instead of sysconfig.customize_compiler.
- Add a local conftest providing disable_macos_customization.

The only remaining non-relative dependency is CPython's test.support
(os_helper) in test_unix -- stdlib test infrastructure, not distutils.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
# Conflicts:
#	.github/workflows/main.yml
#	.pre-commit-config.yaml
#	conftest.py
#	docs/conf.py
#	docs/deprecated/distutils/examples.rst
#	mypy.ini
#	pyproject.toml
#	pytest.ini
#	ruff.toml
#	setuptools/_distutils/_dataclass.py
#	setuptools/_distutils/compat/py310.py
#	setuptools/_distutils/compat/py39.py
#	setuptools/_distutils/compilers/C/tests/conftest.py
#	setuptools/_distutils/compilers/_modified.py
#	setuptools/_distutils/compilers/_util.py
#	setuptools/_distutils/compilers/errors.py
#	setuptools/_distutils/compilers/logging.py
#	setuptools/_distutils/compilers/platform/detect.py
#	setuptools/_distutils/compilers/platform/macos.py
#	setuptools/_distutils/tests/compat/py39.py
@mergify

mergify Bot commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Tick the box to add this pull request to the merge queue (same as @mergifyio queue).

  • Queue this pull request

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

10 participants