Skip to content

test.test_regrtest.ArgsTestCase.test_unload_tests is failing in Python 3.11.7 #113166

Description

@agustinhenze

Bug report

Bug description:

Hi there, I have downloaded https://www.python.org/ftp/python/3.11.7/Python-3.11.7.tar.xz uncompressed, then ./configure --enable-optimizations && make install and then executed python3 -m test -v -j $(nproc) "test_regrtest" I get the following output

root@71f3105e8fc0:/Python-3.11.7# python3 -m test -v -j $(nproc) "test_regrtest"
== CPython 3.11.7 (main, Dec 15 2023, 13:32:22) [GCC 12.2.0]
== Linux-6.5.0-4-amd64-x86_64-with-glibc2.36 little-endian
== Python build: release PGO
== cwd: /tmp/test_python_worker_35953æ
== CPU count: 8
== encodings: locale=UTF-8 FS=utf-8
== resources: all test resources are disabled, use -u option to unskip tests

Using random seed: 1265715576
0:00:00 load avg: 1.10 Run 1 test in parallel using 1 worker process
0:00:09 load avg: 1.25 [1/1/1] test_regrtest failed (1 failure)
test_add_python_opts (test.test_regrtest.ArgsTestCase.test_add_python_opts) ... ok
test_cleanup (test.test_regrtest.ArgsTestCase.test_cleanup) ... ok
test_coverage (test.test_regrtest.ArgsTestCase.test_coverage) ... ok
test_crashed (test.test_regrtest.ArgsTestCase.test_crashed) ... ok
test_doctest (test.test_regrtest.ArgsTestCase.test_doctest) ... ok
test_env_changed (test.test_regrtest.ArgsTestCase.test_env_changed) ... ok
test_failing_test (test.test_regrtest.ArgsTestCase.test_failing_test) ... ok
test_forever (test.test_regrtest.ArgsTestCase.test_forever) ... ok
test_fromfile (test.test_regrtest.ArgsTestCase.test_fromfile) ... ok
test_huntrleaks (test.test_regrtest.ArgsTestCase.test_huntrleaks) ... skipped 'need a debug build'
test_huntrleaks_fd_leak (test.test_regrtest.ArgsTestCase.test_huntrleaks_fd_leak) ... skipped 'need a debug build'
test_huntrleaks_mp (test.test_regrtest.ArgsTestCase.test_huntrleaks_mp) ... skipped 'need a debug build'
test_ignorefile (test.test_regrtest.ArgsTestCase.test_ignorefile) ... ok
test_interrupted (test.test_regrtest.ArgsTestCase.test_interrupted) ... ok
test_leak_tmp_file (test.test_regrtest.ArgsTestCase.test_leak_tmp_file) ... ok
test_list_cases (test.test_regrtest.ArgsTestCase.test_list_cases) ... ok
test_list_tests (test.test_regrtest.ArgsTestCase.test_list_tests) ... ok
test_matchfile (test.test_regrtest.ArgsTestCase.test_matchfile) ... ok
test_multiprocessing_timeout (test.test_regrtest.ArgsTestCase.test_multiprocessing_timeout) ... ok
test_no_test_ran_some_test_exist_some_not (test.test_regrtest.ArgsTestCase.test_no_test_ran_some_test_exist_some_not) ... ok
test_no_tests_ran (test.test_regrtest.ArgsTestCase.test_no_tests_ran) ... ok
test_no_tests_ran_multiple_tests_nonexistent (test.test_regrtest.ArgsTestCase.test_no_tests_ran_multiple_tests_nonexistent) ... ok
test_no_tests_ran_skip (test.test_regrtest.ArgsTestCase.test_no_tests_ran_skip) ... ok
test_print_warning (test.test_regrtest.ArgsTestCase.test_print_warning) ... ok
test_python_command (test.test_regrtest.ArgsTestCase.test_python_command) ... ok
test_random (test.test_regrtest.ArgsTestCase.test_random) ... ok
test_random_seed (test.test_regrtest.ArgsTestCase.test_random_seed) ... ok
test_random_seed_workers (test.test_regrtest.ArgsTestCase.test_random_seed_workers) ... ok
test_rerun_async_setup_hook_failure (test.test_regrtest.ArgsTestCase.test_rerun_async_setup_hook_failure) ... ok
test_rerun_async_teardown_hook_failure (test.test_regrtest.ArgsTestCase.test_rerun_async_teardown_hook_failure) ... ok
test_rerun_fail (test.test_regrtest.ArgsTestCase.test_rerun_fail) ... ok
test_rerun_setup_class_hook_failure (test.test_regrtest.ArgsTestCase.test_rerun_setup_class_hook_failure) ... ok
test_rerun_setup_hook_failure (test.test_regrtest.ArgsTestCase.test_rerun_setup_hook_failure) ... ok
test_rerun_setup_module_hook_failure (test.test_regrtest.ArgsTestCase.test_rerun_setup_module_hook_failure) ... ok
test_rerun_success (test.test_regrtest.ArgsTestCase.test_rerun_success) ... ok
test_rerun_teardown_class_hook_failure (test.test_regrtest.ArgsTestCase.test_rerun_teardown_class_hook_failure) ... ok
test_rerun_teardown_hook_failure (test.test_regrtest.ArgsTestCase.test_rerun_teardown_hook_failure) ... ok
test_rerun_teardown_module_hook_failure (test.test_regrtest.ArgsTestCase.test_rerun_teardown_module_hook_failure) ... ok
test_resources (test.test_regrtest.ArgsTestCase.test_resources) ... ok
test_skip (test.test_regrtest.ArgsTestCase.test_skip) ... ok
test_slowest (test.test_regrtest.ArgsTestCase.test_slowest) ... ok
test_slowest_interrupted (test.test_regrtest.ArgsTestCase.test_slowest_interrupted) ... ok
test_success (test.test_regrtest.ArgsTestCase.test_success) ... ok
test_threading_excepthook (test.test_regrtest.ArgsTestCase.test_threading_excepthook) ... ok
test_uncollectable (test.test_regrtest.ArgsTestCase.test_uncollectable) ... ok
test_unicode_guard_env (test.test_regrtest.ArgsTestCase.test_unicode_guard_env) ... ok
test_unload_tests (test.test_regrtest.ArgsTestCase.test_unload_tests) ... FAIL
test_unraisable_exc (test.test_regrtest.ArgsTestCase.test_unraisable_exc) ... ok
test_verbose3 (test.test_regrtest.ArgsTestCase.test_verbose3) ... ok
test_wait (test.test_regrtest.ArgsTestCase.test_wait) ... ok
test_worker_decode_error (test.test_regrtest.ArgsTestCase.test_worker_decode_error) ... ok
test_worker_output_on_failure (test.test_regrtest.ArgsTestCase.test_worker_output_on_failure) ... ok
test_finds_expected_number_of_tests (test.test_regrtest.CheckActualTests.test_finds_expected_number_of_tests)
Check that regrtest appears to find the expected set of tests. ... ok
test_arg (test.test_regrtest.ParseArgsTestCase.test_arg) ... ok
test_arg_option_arg (test.test_regrtest.ParseArgsTestCase.test_arg_option_arg) ... ok
test_coverage (test.test_regrtest.ParseArgsTestCase.test_coverage) ... ok
test_coverdir (test.test_regrtest.ParseArgsTestCase.test_coverdir) ... ok
test_dont_add_python_opts (test.test_regrtest.ParseArgsTestCase.test_dont_add_python_opts) ... ok
test_exclude (test.test_regrtest.ParseArgsTestCase.test_exclude) ... ok
test_failfast (test.test_regrtest.ParseArgsTestCase.test_failfast) ... ok
test_fast_ci (test.test_regrtest.ParseArgsTestCase.test_fast_ci) ... ok
test_fast_ci_python_cmd (test.test_regrtest.ParseArgsTestCase.test_fast_ci_python_cmd) ... ok
test_fast_ci_resource (test.test_regrtest.ParseArgsTestCase.test_fast_ci_resource) ... ok
test_forever (test.test_regrtest.ParseArgsTestCase.test_forever) ... ok
test_fromfile (test.test_regrtest.ParseArgsTestCase.test_fromfile) ... ok
test_header (test.test_regrtest.ParseArgsTestCase.test_header) ... ok
test_help (test.test_regrtest.ParseArgsTestCase.test_help) ... ok
test_huntrleaks (test.test_regrtest.ParseArgsTestCase.test_huntrleaks) ... ok
test_long_option__partial (test.test_regrtest.ParseArgsTestCase.test_long_option__partial) ... ok
test_match (test.test_regrtest.ParseArgsTestCase.test_match) ... ok
test_memlimit (test.test_regrtest.ParseArgsTestCase.test_memlimit) ... ok
test_multiprocess (test.test_regrtest.ParseArgsTestCase.test_multiprocess) ... ok
test_nocoverdir (test.test_regrtest.ParseArgsTestCase.test_nocoverdir) ... ok
test_nowindows (test.test_regrtest.ParseArgsTestCase.test_nowindows) ... ok
test_option_and_arg (test.test_regrtest.ParseArgsTestCase.test_option_and_arg) ... ok
test_option_with_empty_string_value (test.test_regrtest.ParseArgsTestCase.test_option_with_empty_string_value) ... ok
test_quiet (test.test_regrtest.ParseArgsTestCase.test_quiet) ... ok
test_randomize (test.test_regrtest.ParseArgsTestCase.test_randomize) ... ok
test_randseed (test.test_regrtest.ParseArgsTestCase.test_randseed) ... ok
test_rerun (test.test_regrtest.ParseArgsTestCase.test_rerun) ... ok
test_runleaks (test.test_regrtest.ParseArgsTestCase.test_runleaks) ... ok
test_single (test.test_regrtest.ParseArgsTestCase.test_single) ... ok
test_slow_ci (test.test_regrtest.ParseArgsTestCase.test_slow_ci) ... ok
test_slowest (test.test_regrtest.ParseArgsTestCase.test_slowest) ... ok
test_start (test.test_regrtest.ParseArgsTestCase.test_start) ... ok
test_testdir (test.test_regrtest.ParseArgsTestCase.test_testdir) ... ok
test_threshold (test.test_regrtest.ParseArgsTestCase.test_threshold) ... ok
test_timeout (test.test_regrtest.ParseArgsTestCase.test_timeout) ... ok
test_two_options (test.test_regrtest.ParseArgsTestCase.test_two_options) ... ok
test_unknown_option (test.test_regrtest.ParseArgsTestCase.test_unknown_option) ... ok
test_unrecognized_argument (test.test_regrtest.ParseArgsTestCase.test_unrecognized_argument) ... ok
test_use (test.test_regrtest.ParseArgsTestCase.test_use) ... ok
test_verbose (test.test_regrtest.ParseArgsTestCase.test_verbose) ... ok
test_verbose3 (test.test_regrtest.ParseArgsTestCase.test_verbose3) ... ok
test_wait (test.test_regrtest.ParseArgsTestCase.test_wait) ... ok
test_module_autotest (test.test_regrtest.ProgramsTestCase.test_module_autotest) ... ok
test_module_from_test_autotest (test.test_regrtest.ProgramsTestCase.test_module_from_test_autotest) ... ok
test_module_regrtest (test.test_regrtest.ProgramsTestCase.test_module_regrtest) ... ok
test_module_test (test.test_regrtest.ProgramsTestCase.test_module_test) ... ok
test_pcbuild_rt (test.test_regrtest.ProgramsTestCase.test_pcbuild_rt) ... skipped 'Windows only'
test_script_autotest (test.test_regrtest.ProgramsTestCase.test_script_autotest) ... ok
test_script_regrtest (test.test_regrtest.ProgramsTestCase.test_script_regrtest) ... ok
test_tools_buildbot_test (test.test_regrtest.ProgramsTestCase.test_tools_buildbot_test) ... skipped 'test.bat script is not installed'
test_format_duration (test.test_regrtest.TestUtils.test_format_duration) ... ok
test_format_resources (test.test_regrtest.TestUtils.test_format_resources) ... ok
test_get_signal_name (test.test_regrtest.TestUtils.test_get_signal_name) ... ok
test_match_test (test.test_regrtest.TestUtils.test_match_test) ... ok
test_normalize_test_name (test.test_regrtest.TestUtils.test_normalize_test_name) ... ok

======================================================================
FAIL: test_unload_tests (test.test_regrtest.ArgsTestCase.test_unload_tests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/test/test_regrtest.py", line 2040, in test_unload_tests
    output = self.run_python(args)
             ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/test/test_regrtest.py", line 722, in run_python
    proc = self.run_command(args, **kw)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/test/test_regrtest.py", line 713, in run_command
    self.fail(msg)
AssertionError: Command ['/usr/local/bin/python3', '-X', 'faulthandler', '-I', '-Wd', '-E', '-bb', '-m', 'test', '--testdir=/usr/local/lib/python3.11/test/regrtestdata/import_from_tests', 'test_regrtest_a', 'test_regrtest_b', 'test_regrtest_c'] failed with exit code 1, but exit code 0 expected!

stdout:
---
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/usr/local/lib/python3.11/test/__main__.py", line 2, in <module>
    main(_add_python_opts=True)
  File "/usr/local/lib/python3.11/test/libregrtest/main.py", line 672, in main
    Regrtest(ns, _add_python_opts=_add_python_opts).main(tests=tests)
  File "/usr/local/lib/python3.11/test/libregrtest/main.py", line 654, in main
    selected, tests = self.find_tests(tests)
                      ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/test/libregrtest/main.py", line 191, in find_tests
    alltests = findtests(testdir=self.test_dir,
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/test/libregrtest/findtests.py", line 41, in findtests
    for name in os.listdir(testdir):
                ^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: '/usr/local/lib/python3.11/test/regrtestdata/import_from_tests'

---


----------------------------------------------------------------------
Ran 108 tests in 9.797s

FAILED (failures=1, skipped=5)
test test_regrtest failed

== Tests result: FAILURE ==

1 test failed:
    test_regrtest

Total duration: 9.9 sec
Total tests: run=108 failures=1 skipped=5
Total test files: run=1/1 failed=1
Result: FAILURE

CPython versions tested on:

3.11

Operating systems tested on:

Linux

Activity

  1. serhiy-storchaka commented on Dec 15, 2023

    @serhiy-storchaka
    Member

    It was fixed in #112765. Unfortunately 3.11.7 was released with this bug. It only affects tests.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    type-bugAn unexpected behavior, bug, or error

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions