Skip to content

test_peg_generator.test_c_parser emits DeprecationWarning due to distutils #87591

Closed as not planned
Closed as not planned
test_peg_generator.test_c_parser emits DeprecationWarning due to distutils#87591
Labels
3.10 (EOL)end of lifependingThe issue will be closed if no feedback is providedtestsTests in the Lib/test dirtype-bugAn unexpected behavior, bug, or error

Description

@tirkarthi
tirkarthi
opened on Mar 7, 2021
BPO 43425
Nosy @terryjreedy, @vstinner, @ambv, @zooba, @corona10, @pablogsal, @miss-islington, @tirkarthi
PRs
  • bpo-43425: Update test_c_parser not to use TempdirManager #26693
  • [3.10] bpo-43425: Update test_c_parser not to use TempdirManager (GH-26693) #26713
  • bpo-43425: Update _osx_support not to use distutils.log #26968
  • bpo-43425: Update setup.py not to use distutils.log #26969
  • [3.10] bpo-43425: Update _osx_support not to use distutils.log (GH-26968) #26978
  • bpo-43425: Remove test2to3 from Tools #26981
  • [3.10] bpo-43425: Update setup.py not to use distutils.log (GH-26969) #26986
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = None
created_at = <Date 2021-03-07.05:07:09.546>
labels = ['type-bug', 'tests', '3.10']
title = 'test_peg_generator.test_c_parser emits DeprecationWarning due to distutils'
updated_at = <Date 2021-09-15.13:15:47.651>
user = 'https://github.com/tirkarthi'

    bugs.python.org fields:

    activity = <Date 2021-09-15.13:15:47.651>
actor = 'vstinner'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Tests']
creation = <Date 2021-03-07.05:07:09.546>
creator = 'xtreak'
dependencies = []
files = []
hgrepos = []
issue_num = 43425
keywords = ['patch']
message_count = 13.0
messages = ['388222', '388242', '388243', '388572', '388780', '395716', '395768', '396805', '396808', '396837', '396838', '397716', '401831']
nosy_count = 8.0
nosy_names = ['terry.reedy', 'vstinner', 'lukasz.langa', 'steve.dower', 'corona10', 'pablogsal', 'miss-islington', 'xtreak']
pr_nums = ['26693', '26713', '26968', '26969', '26978', '26981', '26986']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue43425'
versions = ['Python 3.10']

    Activity

    1. tirkarthi commented on Mar 7, 2021

      @tirkarthi
      tirkarthi
      on Mar 7, 2021
      MemberAuthor

      distutils was deprecated for removal in Python 3.10. It is used in test_peg_generator.test_c_parser which emits a deprecation warning. It also seems to be used in test_support for missing_compiler_executable that will emit a deprecation warning.

      ./python -Wall -m test test_peg_generator test_c_parser
      0:00:00 load avg: 0.02 Run tests sequentially
      0:00:00 load avg: 0.02 [1/2] test_peg_generator
      /root/cpython/Lib/test/test_peg_generator/test_c_parser.py:4: DeprecationWarning: The distutils package is deprecated and slated for removal in Python 3.12. Use setuptools or check PEP-632 for potential alternatives
      from distutils.tests.support import TempdirManager

      Other places on grep :

      rg 'from distutils' | rg -v 'Lib/distutils|rst'
      Modules/_decimal/tests/formathelper.py:from distutils.spawn import find_executable
      Doc/includes/setup.py:from distutils.core import setup, Extension
      Doc/includes/test.py:from distutils.util import get_platform
      setup.py:from distutils import log
      setup.py:from distutils.command.build_ext import build_ext
      setup.py:from distutils.command.build_scripts import build_scripts
      setup.py:from distutils.command.install import install
      setup.py:from distutils.command.install_lib import install_lib
      setup.py:from distutils.core import Extension, setup
      setup.py:from distutils.errors import CCompilerError, DistutilsError
      setup.py:from distutils.spawn import find_executable
      Lib/_osx_support.py: from distutils import log
      Lib/_osx_support.py: Currently called from distutils.sysconfig
      Lib/test/support/init.py: from distutils import ccompiler, sysconfig, spawn, errors
      Lib/test/test_importlib/test_windows.py:from distutils.util import get_platform
      Lib/test/test_peg_generator/test_c_parser.py:from distutils.tests.support import TempdirManager
      Tools/peg_generator/pegen/build.py: from distutils.core import Distribution, Extension
      Tools/peg_generator/pegen/build.py: from distutils.command.clean import clean # type: ignore
      Tools/peg_generator/pegen/build.py: from distutils.command.build_ext import build_ext # type: ignore
      Tools/peg_generator/pegen/build.py: from distutils.tests.support import fixup_build_ext # type: ignore
      Tools/test2to3/setup.py:from distutils.core import setup
      Tools/test2to3/setup.py: from distutils.command.build_py import build_py_2to3 as build_py
      Tools/test2to3/setup.py: from distutils.command.build_py import build_py
      Tools/test2to3/setup.py: from distutils.command.build_scripts import build_scripts_2to3 as build_scripts
      Tools/test2to3/setup.py: from distutils.command.build_scripts import build_scripts
      Tools/test2to3/test/runtests.py: from distutils.util import copydir_run_2to3
      Misc/HISTORY:- Issue bpo-5394: removed > 2.3 syntax from distutils.msvc9compiler.

    2. tirkarthi
      added
      3.10 (EOL)end of life
      testsTests in the Lib/test dir
      type-bugAn unexpected behavior, bug, or error
      on Mar 7, 2021

    3. pablogsal commented on Mar 7, 2021

      @pablogsal
      pablogsal
      on Mar 7, 2021
      Member

      We just need to suppress the warning unless I am missing somey. Distutils will still be used internally by CPython.

    4. tirkarthi commented on Mar 7, 2021

      @tirkarthi
      tirkarthi
      on Mar 7, 2021
      MemberAuthor

      Thanks for the clarification. As per bpo-41282 I thought disutils will be removed. The accepted PEP-632 states this scenario so I guess silencing the warning is good enough.

      The final dependency on distutils is CPython itself, which uses it to build native extension modules in the standard library (except on Windows). Because this is a CPython build-time dependency, it is possible to continue to use distutils for this specific case without it being part of the standard library.

    5. terryjreedy commented on Mar 13, 2021

      @terryjreedy
      terryjreedy
      on Mar 13, 2021
      Member

      Steve, PEP-632 lacks a reference to the SC acceptance thereof. Could you add one?
      https://mail.python.org/archives/list/python-dev@python.org/thread/TXU6TVOMBLQU3SV57DMMOA5Y2E67AW7P/
      And can you verify my interpretation below?

      The warning is correct as to removal. PEP-632 says, perhaps a bit tersely, that when the 3.11 branch is established for 3.11.0b1 and master/main becomes the 3.12.0a development branch, disutils/* be no longer be installed as part of the stdlib (ie, in /Lib). At that point, 'import disutils' will cease working in installed CPython, outside of a cpython repository. Although this could be worked around by catching the import error and skipping disutils-dependent tests, the dependency should better be removed, if possible, before 3.12.0a.

      The PEP gives no timetable for doing so, and anticipates separate PRs like this.

      It further says that when "the CPython build process no longer depends on distutils being in the standard library, the entire Lib/distutils directory and Lib/test/test_distutils.py file will be removed from the repository." Leaving it deprecated but present was rejected.

      Part of the extensive discussion at https://discuss.python.org/t/pep-632-deprecate-distutils-module/5134/128 was about possibly putting parts of disutiles under /Tools and replacing 'import disutils' with a reference to <python dir>/Tools/distutiles/whatever. If this is done for tests, the access and use has to be conditional. Our Windows installer only installs 4 or 21 Tools subdirectories thought appropriate for uses and others may include less, and maybe omit Tools altogether.

      Karthikeyan posted the opening of this and another issue 5 days ago on bpo-41281, with no comment so far.

    6. zooba commented on Mar 15, 2021

      @zooba
      zooba
      on Mar 15, 2021
      Member

      The link appears to be in the Resolution header of PEP-632?

      But yes, your analysis is correct. If/when we move distutils elsewhere, we'll probably have to update all of these sections.

      Ideally we'd have a helper function to do the initial "import distutils", handle failure (because it's not in Tools), and suppress the warning. But I don't know that there's anywhere we could put it reliably anyway, so probably just need to suppress the warnings directly and we'll handle failures if/when we move it into Tools.

    7. corona10 commented on Jun 12, 2021

      @corona10
      corona10
      on Jun 12, 2021
      Member

      New changeset 736ed6f by Dong-hee Na in branch 'main':
      bpo-43425: Update test_c_parser not to use TempdirManager (GH-26693)
      736ed6f

    8. miss-islington commented on Jun 14, 2021

      @miss-islington
      miss-islington
      on Jun 14, 2021
      Contributor

      New changeset 2b57ad3 by Miss Islington (bot) in branch '3.10':
      bpo-43425: Update test_c_parser not to use TempdirManager (GH-26693)
      2b57ad3

    9. corona10 commented on Jul 1, 2021

      @corona10
      corona10
      on Jul 1, 2021
      Member

      New changeset c8979f7 by Dong-hee Na in branch 'main':
      bpo-43425: Update _osx_support not to use distutils.log (GH-26968)
      c8979f7

    10. miss-islington commented on Jul 1, 2021

      @miss-islington
      miss-islington
      on Jul 1, 2021
      Contributor

      New changeset 94a4136 by Miss Islington (bot) in branch '3.10':
      bpo-43425: Update _osx_support not to use distutils.log (GH-26968)
      94a4136

    11. corona10 commented on Jul 2, 2021

      @corona10
      corona10
      on Jul 2, 2021
      Member

      New changeset a7e251b by Dong-hee Na in branch 'main':
      bpo-43425: Update setup.py not to use distutils.log (GH-26969)
      a7e251b

    12. miss-islington commented on Jul 2, 2021

      @miss-islington
      miss-islington
      on Jul 2, 2021
      Contributor

      New changeset 956f1fc by Miss Islington (bot) in branch '3.10':
      bpo-43425: Update setup.py not to use distutils.log (GH-26969)
      956f1fc

    13. ambv commented on Jul 17, 2021

      @ambv
      ambv
      on Jul 17, 2021
      Contributor

      New changeset d0b2b00 by Dong-hee Na in branch 'main':
      bpo-43425: Remove test2to3 from Tools (bpo-26981)
      d0b2b00

    14. vstinner commented on Sep 15, 2021

      @vstinner
      vstinner
      on Sep 15, 2021
      Member

      test_peg_generator still emits DeprecationWarning warnings. Example:

      vstinner@apu$ ./python -m test test_peg_generator -v
      == CPython 3.11.0a0 (heads/main:b49263b698, Sep 15 2021, 14:44:10) [GCC 11.2.1 20210728 (Red Hat 11.2.1-1)]
      == Linux-5.13.14-200.fc34.x86_64-x86_64-with-glibc2.33 little-endian
      == cwd: /home/vstinner/python/main/build/test_python_193641æ
      == CPU count: 8
      == encodings: locale=UTF-8, FS=utf-8
      0:00:00 load avg: 15.94 Run tests sequentially
      0:00:00 load avg: 15.94 [1/1] test_peg_generator
      test_advanced_left_recursive (test.test_peg_generator.test_c_parser.TestCParser) ...

      /home/vstinner/python/main/Lib/test/support/init.py:1713: DeprecationWarning: The distutils package is deprecated and slated for removal in Python 3.12. Use setuptools or check PEP-632 for potential alternatives
      from distutils import ccompiler, sysconfig, spawn, errors

      /home/vstinner/python/main/Lib/test/support/init.py:1713: DeprecationWarning: The distutils.sysconfig module is deprecated, use sysconfig instead
      from distutils import ccompiler, sysconfig, spawn, errors
      (...)

    15. ezio-melotti
      transferred this issue fromon Apr 10, 2022

    16. iritkatriel commented on Apr 6, 2023

      @iritkatriel
      iritkatriel
      on Apr 6, 2023
      Member

      test_peg_generator still emits DeprecationWarning warnings.

      Not anymore - it's skipped since 0faa0ba.

    17. iritkatriel
      added
      pendingThe issue will be closed if no feedback is provided
      on Apr 6, 2023
    18. iritkatriel
      closed this as not plannedon Apr 25, 2023
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

    Metadata

    Metadata

    Assignees

    No one assigned

      Labels

      3.10 (EOL)end of lifependingThe issue will be closed if no feedback is providedtestsTests in the Lib/test dirtype-bugAn unexpected behavior, bug, or error

      Projects

      No projects

        Milestone

        No milestone

        Relationships

        None yet

        Development

        No branches or pull requests

        Issue actions