Skip to content

test_asyncio and test_compileall leak temporary files/directories #109295

Closed
Closed
test_asyncio and test_compileall leak temporary files/directories#109295
Labels
testsTests in the Lib/test dir

Description

@vstinner
vstinner
opened on Sep 12, 2023

Using PR #109290 fix, I found that the two following tests leak temporary files/directories.

test_compileall:

$ ./python -m test -j1 test_compileall --fail-env-changed -m test.test_compileall.CompileallTestsWithSourceEpoch.test_ddir_empty_multiple_workers
0:00:00 load avg: 0.81 Run tests in parallel using 1 child processes
0:00:01 load avg: 0.81 [1/1/1] test_compileall failed (env changed)
Warning -- files was modified by test_compileall
Warning --   Before: []
Warning --   After:  ['pymp-kmfq1ivj/'] 
Warning -- files was modified by test_compileall
Warning --   Before: []
Warning --   After:  ['pymp-kmfq1ivj/']

== Tests result: ENV CHANGED ==

1 test altered the execution environment:
    test_compileall

Total duration: 1.2 sec
Total tests: run=1 (filtered)
Total test files: run=1/1 (filtered) env_changed=1
Result: ENV CHANGED

test.test_asyncio.test_events:

$ ./python -m test -j1 test.test_asyncio.test_events --fail-env-changed -m test.test_asyncio.test_events.TestPyGetEventLoop.test_get_event_loop_new_process 
0:00:00 load avg: 0.65 Run tests in parallel using 1 child processes
0:00:01 load avg: 0.65 [1/1/1] test.test_asyncio.test_events failed (env changed)
Warning -- files was modified by test.test_asyncio.test_events
Warning --   Before: []
Warning --   After:  ['pymp-tvmenicy/'] 
Warning -- files was modified by test.test_asyncio.test_events
Warning --   Before: []
Warning --   After:  ['pymp-tvmenicy/']

== Tests result: ENV CHANGED ==

1 test altered the execution environment:
    test.test_asyncio.test_events

Total duration: 1.4 sec
Total tests: run=1 (filtered)
Total test files: run=1/1 (filtered) env_changed=1
Result: ENV CHANGED

Linked PRs

Activity

  1. vstinner
    added
    testsTests in the Lib/test dir
    on Sep 12, 2023
  2. vstinner
    added 4 commits that reference this issue on Sep 12, 2023

    pythongh-109295: test_asyncio and test_compileall clean multiprocessing

    39f44c4

    pythongh-109295: test_asyncio and test_compileall clean multiprocessing

    3830ca6

    pythongh-109295: Fix test_os.test_access_denied() for TEMP=cwd

    3bda312

    pythongh-109295: Fix test_os.test_access_denied() for TEMP=cwd

    c0d5185

  3. vstinner commented on Sep 12, 2023

    @vstinner
    vstinner
    on Sep 12, 2023
    MemberAuthor

    Another problem: test_os fails if the TEMP environment variable is equal to the current working directory.

    ERROR: test_access_denied (test.test_os.StatAttributeTests.test_access_denied)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "D:\a\cpython\cpython\Lib\test\test_os.py", line 742, in test_access_denied
    create_file(fname, b'ABC')
  File "D:\a\cpython\cpython\Lib\test\test_os.py", line 94, in create_file
    with open(filename, "xb", 0) as fp:
         ^^^^^^^^^^^^^^^^^^^^^^^
FileExistsError: [Errno 17] File exists: 'C:\\Users\\runneradmin\\AppData\\Local\\Temp\\test_python_worker_a5djsgg5\\@test_5340_tmp�'
  4. vstinner
    added a commit that references this issue on Sep 12, 2023

    gh-109295: Clean up multiprocessing in test_asyncio and test_compilea…

    09ea4b8
  5. miss-islington
    added 2 commits that reference this issue on Sep 12, 2023

    pythongh-109295: Clean up multiprocessing in test_asyncio and test_co…

    b95540d

    pythongh-109295: Clean up multiprocessing in test_asyncio and test_co…

    ce50dad
  6. vstinner
    added a commit that references this issue on Sep 12, 2023

    gh-109295: Fix test_os.test_access_denied() for TEMP=cwd (#109299)

    7dedfd3
  7. miss-islington
    added 2 commits that reference this issue on Sep 12, 2023

    pythongh-109295: Fix test_os.test_access_denied() for TEMP=cwd (pytho…

    8c5f404

    pythongh-109295: Fix test_os.test_access_denied() for TEMP=cwd (pytho…

    76ae143
  8. vstinner
    added a commit that references this issue on Sep 12, 2023

    [3.11] gh-109295: Clean up multiprocessing in test_asyncio and test_c…

    abf3a68

  9. vstinner commented on Sep 12, 2023

    @vstinner
    vstinner
    on Sep 12, 2023
    MemberAuthor

    Last issue with test_generated_cases:

    0:19:14 load avg: 6.45 [2/3/2] test_generated_cases failed (uncaught exception)
Re-running test_generated_cases in verbose mode
test test_generated_cases crashed -- Traceback (most recent call last):
  File "D:\a\cpython\cpython\Lib\test\libregrtest\single.py", line 139, in _runtest_env_changed_exc
    _load_run_test(result, runtests)
  File "D:\a\cpython\cpython\Lib\test\libregrtest\single.py", line 87, in _load_run_test
    test_mod = importlib.import_module(module_name)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\a\cpython\cpython\Lib\importlib\__init__.py", line 88, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1381, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1354, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1325, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 929, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 1006, in exec_module
  File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
  File "D:\a\cpython\cpython\Lib\test\test_generated_cases.py", line 11, in <module>
    import generate_cases
  File "D:\a\cpython\cpython\Tools\cases_generator\generate_cases.py", line 41, in <module>
    DEFAULT_INPUT = os.path.relpath(os.path.join(ROOT, "Python/bytecodes.c"))
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\a\cpython\cpython\Lib\ntpath.py", line 791, in relpath
    raise ValueError("path is on mount %r, start on mount %r" % (
ValueError: path is on mount 'D:', start on mount 'C:'

    See PR #109290 for the background.

  10. vstinner
    added 4 commits that reference this issue on Sep 12, 2023

    pythongh-109295: Skip test_generated_cases if different mount drives

    7f10fe4

    pythongh-109295: Skip test_generated_cases if different mount drives

    4ad15ef

    [3.11] gh-109295: Fix test_os.test_access_denied() for TEMP=cwd (GH-1…

    5d47fb1

    gh-109295: Skip test_generated_cases if different mount drives (#109308)

    9363769
  11. vstinner
    closed this as completedon Sep 12, 2023
  12. Yhg1s
    added 2 commits that reference this issue on Sep 12, 2023

    [3.12] gh-109295: Clean up multiprocessing in test_asyncio and test_c…

    8acfb42

    [3.12] gh-109295: Fix test_os.test_access_denied() for TEMP=cwd (GH-1…

    7479a7a
  13. vstinner
    added 3 commits that reference this issue on Sep 13, 2023

    pythongh-109295: Clean up multiprocessing in test_asyncio and test_co…

    2a94b5f

    pythongh-109295: Fix test_os.test_access_denied() for TEMP=cwd (pytho…

    f2e598b

    pythongh-109295: Skip test_generated_cases if different mount drives (p…

    aea41fd
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    testsTests in the Lib/test dir

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions