Skip to content

test_external_inspection.test_only_active_thread is flaky #136186

Closed
Closed
test_external_inspection.test_only_active_thread is flaky#136186
Labels
testsTests in the Lib/test dirtype-bugAn unexpected behavior, bug, or error

Description

@colesbury
colesbury
opened on Jul 1, 2025

Bug report

Bug description:

test test_external_inspection failed -- Traceback (most recent call last):
  File "/home/runner/work/cpython/cpython/Lib/test/test_external_inspection.py", line 991, in test_only_active_thread
    self.assertEqual(len(gil_traces), 1, "Should have exactly one GIL holder")
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: 0 != 1 : Should have exactly one GIL holder

See the failures in https://github.com/python/cpython/actions/runs/16006058028/job/45152923015 on main

cc @pablogsal

See also:

CPython versions tested on:

CPython main branch

Operating systems tested on:

No response

Linked PRs

Activity

  1. colesbury
    added
    type-bugAn unexpected behavior, bug, or error
    testsTests in the Lib/test dir
    on Jul 1, 2025

  2. pablogsal commented on Jul 1, 2025

    @pablogsal
    pablogsal
    on Jul 1, 2025
    Member

    Will take a look 👀

  3. StanFromIreland commented on Jul 2, 2025

    @StanFromIreland
    StanFromIreland
    on Jul 2, 2025
    Member

    Also fails on buildbots, e.g. this run

  4. vstinner commented on Jul 2, 2025

    @vstinner
    vstinner
    on Jul 2, 2025
    Member

    I can reproduce the issue on Linux with the command:

    $ ./python -m test test_external_inspection -v -m test_only_active_thread -F -j40
(...)
AssertionError: 0 != 1 : Should have exactly one GIL holder

  5. sergey-miryanov commented on Jul 2, 2025

    @sergey-miryanov
    sergey-miryanov
    on Jul 2, 2025
    Contributor

    I can reproduce the issue on Linux with the command:

    $ ./python -m test test_external_inspection -v -m test_only_active_thread -F -j40
(...)
AssertionError: 0 != 1 : Should have exactly one GIL holder

    With this command I got following error on Windows (JFYI):

    PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'D:\\Sources\\_pythonish\\cpython\\main\\build\\test_python_50684\xe6' 
    0:00:11 load avg: 59.23 [ 11/1] test_external_inspection worker non-zero exit code (Exit code 1)
test_only_active_thread (test.test_external_inspection.TestGetStackTrace.test_only_active_thread) ... ok

----------------------------------------------------------------------
Ran 1 test in 3.596s

OK
Traceback (most recent call last):
  File "D:\Sources\_pythonish\cpython\main\Lib\test\support\os_helper.py", line 540, in temp_dir
    yield path
  File "D:\Sources\_pythonish\cpython\main\Lib\test\support\os_helper.py", line 599, in temp_cwd
    yield cwd_dir
  File "D:\Sources\_pythonish\cpython\main\Lib\test\libregrtest\worker.py", line 127, in main
    worker_process(worker_json)
    ~~~~~~~~~~~~~~^^^^^^^^^^^^^
  File "D:\Sources\_pythonish\cpython\main\Lib\test\libregrtest\worker.py", line 113, in worker_process
    sys.exit(0)
    ~~~~~~~~^^^
SystemExit: 0

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "D:\Sources\_pythonish\cpython\main\Lib\test\support\__init__.py", line 208, in _force_run
    return func(*args)
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'D:\\Sources\\_pythonish\\cpython\\main\\build\\test_python_50684\xe6'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "D:\Sources\_pythonish\cpython\main\Lib\runpy.py", line 198, in _run_module_as_main
    return _run_code(code, main_globals, None,
                     "__main__", mod_spec)
  File "D:\Sources\_pythonish\cpython\main\Lib\runpy.py", line 88, in _run_code
    exec(code, run_globals)
    ~~~~^^^^^^^^^^^^^^^^^^^
  File "D:\Sources\_pythonish\cpython\main\Lib\test\libregrtest\worker.py", line 131, in <module>
    main()
    ~~~~^^
  File "D:\Sources\_pythonish\cpython\main\Lib\test\libregrtest\worker.py", line 126, in main
    with os_helper.temp_cwd(work_dir, quiet=True):
         ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^
  File "D:\Sources\_pythonish\cpython\main\Lib\contextlib.py", line 162, in __exit__
    self.gen.throw(value)
    ~~~~~~~~~~~~~~^^^^^^^
  File "D:\Sources\_pythonish\cpython\main\Lib\test\support\os_helper.py", line 597, in temp_cwd
    with temp_dir(path=name, quiet=quiet) as temp_path:
         ~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\Sources\_pythonish\cpython\main\Lib\contextlib.py", line 162, in __exit__
    self.gen.throw(value)
    ~~~~~~~~~~~~~~^^^^^^^
  File "D:\Sources\_pythonish\cpython\main\Lib\test\support\os_helper.py", line 545, in temp_dir
    rmtree(path)
    ~~~~~~^^^^^^
  File "D:\Sources\_pythonish\cpython\main\Lib\test\support\os_helper.py", line 497, in rmtree
    _rmtree(path)
    ~~~~~~~^^^^^^
  File "D:\Sources\_pythonish\cpython\main\Lib\test\support\os_helper.py", line 441, in _rmtree
    _waitfor(lambda p: _force_run(p, os.rmdir, p), path)
    ~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\Sources\_pythonish\cpython\main\Lib\test\support\os_helper.py", line 381, in _waitfor
    func(pathname)
    ~~~~^^^^^^^^^^
  File "D:\Sources\_pythonish\cpython\main\Lib\test\support\os_helper.py", line 441, in <lambda>
    _waitfor(lambda p: _force_run(p, os.rmdir, p), path)
                       ~~~~~~~~~~^^^^^^^^^^^^^^^^
  File "D:\Sources\_pythonish\cpython\main\Lib\test\support\__init__.py", line 219, in _force_run
    return func(*args)
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'D:\\Sources\\_pythonish\\cpython\\main\\build\\test_python_50684\xe6'
Kill <WorkerThread #1 running test=test_external_inspection pid=67748 time=11.2 sec> process
Kill <WorkerThread #2 running test=test_external_inspection pid=75760 time=11.2 sec> process
Kill <WorkerThread #3 running test=test_external_inspection pid=37312 time=11.2 sec> process
Kill <WorkerThread #4 running test=test_external_inspection pid=54008 time=136 ms> process
Kill <WorkerThread #5 running test=test_external_inspection pid=71532 time=11.2 sec> process
Kill <WorkerThread #6 running test=test_external_inspection pid=75024 time=908 ms> process
Kill <WorkerThread #7 running test=test_external_inspection pid=76908 time=11.2 sec> process
Kill <WorkerThread #8 running test=test_external_inspection pid=67792 time=11.2 sec> process
Kill <WorkerThread #9 running test=test_external_inspection pid=77664 time=111 ms> process
Kill <WorkerThread #11 running test=test_external_inspection pid=77316 time=11.2 sec> process
Kill <WorkerThread #13 running test=test_external_inspection pid=77100 time=11.2 sec> process
Kill <WorkerThread #14 running test=test_external_inspection pid=67208 time=11.1 sec> process
Kill <WorkerThread #15 running test=test_external_inspection pid=70904 time=11.1 sec> process
Kill <WorkerThread #16 running test=test_external_inspection pid=76680 time=2.4 sec> process
Kill <WorkerThread #17 running test=test_external_inspection pid=69020 time=11.1 sec> process
Kill <WorkerThread #18 running test=test_external_inspection pid=68684 time=1.8 sec> process
Kill <WorkerThread #20 running test=test_external_inspection pid=71156 time=973 ms> process
Kill <WorkerThread #21 running test=test_external_inspection pid=77156 time=11.1 sec> process
Kill <WorkerThread #22 running test=test_external_inspection pid=62752 time=11.1 sec> process
Kill <WorkerThread #23 running test=test_external_inspection pid=74376 time=11.1 sec> process
Kill <WorkerThread #24 running test=test_external_inspection pid=59932 time=1.8 sec> process
Kill <WorkerThread #25 running test=test_external_inspection pid=71704 time=11.1 sec> process
Kill <WorkerThread #26 running test=test_external_inspection pid=76612 time=11.1 sec> process
Kill <WorkerThread #27 running test=test_external_inspection pid=76964 time=2.3 sec> process
Kill <WorkerThread #28 running test=test_external_inspection pid=76940 time=11.1 sec> process
Kill <WorkerThread #29 running test=test_external_inspection pid=77228 time=1.9 sec> process
Kill <WorkerThread #30 running test=test_external_inspection pid=56868 time=11.1 sec> process
Kill <WorkerThread #31 running test=test_external_inspection pid=76344 time=11.1 sec> process
Kill <WorkerThread #32 running test=test_external_inspection pid=77052 time=11.1 sec> process
Kill <WorkerThread #33 running test=test_external_inspection pid=61188 time=11.1 sec> process
Kill <WorkerThread #34 running test=test_external_inspection pid=60384 time=11.1 sec> process
Kill <WorkerThread #35 running test=test_external_inspection pid=74884 time=107 ms> process
Kill <WorkerThread #36 running test=test_external_inspection pid=36240 time=11.0 sec> process
Kill <WorkerThread #37 running test=test_external_inspection pid=77264 time=11.0 sec> process
Kill <WorkerThread #38 running test=test_external_inspection pid=77176 time=11.0 sec> process
Kill <WorkerThread #39 running test=test_external_inspection pid=68476 time=11.1 sec> process
Kill <WorkerThread #40 running test=test_external_inspection pid=70452 time=11.0 sec> process

== Tests result: FAILURE ==

1 test failed:
    test_external_inspection

  6. pablogsal commented on Jul 6, 2025

    @pablogsal
    pablogsal
    on Jul 6, 2025
    Member

    I can reproduce the issue on Linux with the command:

    $ ./python -m test test_external_inspection -v -m test_only_active_thread -F -j40
(...)
AssertionError: 0 != 1 : Should have exactly one GIL holder

    With this command I got following error on Windows (JFYI):

    PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'D:\\Sources\\_pythonish\\cpython\\main\\build\\test_python_50684\xe6' 
    0:00:11 load avg: 59.23 [ 11/1] test_external_inspection worker non-zero exit code (Exit code 1)
test_only_active_thread (test.test_external_inspection.TestGetStackTrace.test_only_active_thread) ... ok

----------------------------------------------------------------------
Ran 1 test in 3.596s

OK
Traceback (most recent call last):
  File "D:\Sources\_pythonish\cpython\main\Lib\test\support\os_helper.py", line 540, in temp_dir
    yield path
  File "D:\Sources\_pythonish\cpython\main\Lib\test\support\os_helper.py", line 599, in temp_cwd
    yield cwd_dir
  File "D:\Sources\_pythonish\cpython\main\Lib\test\libregrtest\worker.py", line 127, in main
    worker_process(worker_json)
    ~~~~~~~~~~~~~~^^^^^^^^^^^^^
  File "D:\Sources\_pythonish\cpython\main\Lib\test\libregrtest\worker.py", line 113, in worker_process
    sys.exit(0)
    ~~~~~~~~^^^
SystemExit: 0

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "D:\Sources\_pythonish\cpython\main\Lib\test\support\__init__.py", line 208, in _force_run
    return func(*args)
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'D:\\Sources\\_pythonish\\cpython\\main\\build\\test_python_50684\xe6'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "D:\Sources\_pythonish\cpython\main\Lib\runpy.py", line 198, in _run_module_as_main
    return _run_code(code, main_globals, None,
                     "__main__", mod_spec)
  File "D:\Sources\_pythonish\cpython\main\Lib\runpy.py", line 88, in _run_code
    exec(code, run_globals)
    ~~~~^^^^^^^^^^^^^^^^^^^
  File "D:\Sources\_pythonish\cpython\main\Lib\test\libregrtest\worker.py", line 131, in <module>
    main()
    ~~~~^^
  File "D:\Sources\_pythonish\cpython\main\Lib\test\libregrtest\worker.py", line 126, in main
    with os_helper.temp_cwd(work_dir, quiet=True):
         ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^
  File "D:\Sources\_pythonish\cpython\main\Lib\contextlib.py", line 162, in __exit__
    self.gen.throw(value)
    ~~~~~~~~~~~~~~^^^^^^^
  File "D:\Sources\_pythonish\cpython\main\Lib\test\support\os_helper.py", line 597, in temp_cwd
    with temp_dir(path=name, quiet=quiet) as temp_path:
         ~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\Sources\_pythonish\cpython\main\Lib\contextlib.py", line 162, in __exit__
    self.gen.throw(value)
    ~~~~~~~~~~~~~~^^^^^^^
  File "D:\Sources\_pythonish\cpython\main\Lib\test\support\os_helper.py", line 545, in temp_dir
    rmtree(path)
    ~~~~~~^^^^^^
  File "D:\Sources\_pythonish\cpython\main\Lib\test\support\os_helper.py", line 497, in rmtree
    _rmtree(path)
    ~~~~~~~^^^^^^
  File "D:\Sources\_pythonish\cpython\main\Lib\test\support\os_helper.py", line 441, in _rmtree
    _waitfor(lambda p: _force_run(p, os.rmdir, p), path)
    ~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\Sources\_pythonish\cpython\main\Lib\test\support\os_helper.py", line 381, in _waitfor
    func(pathname)
    ~~~~^^^^^^^^^^
  File "D:\Sources\_pythonish\cpython\main\Lib\test\support\os_helper.py", line 441, in <lambda>
    _waitfor(lambda p: _force_run(p, os.rmdir, p), path)
                       ~~~~~~~~~~^^^^^^^^^^^^^^^^
  File "D:\Sources\_pythonish\cpython\main\Lib\test\support\__init__.py", line 219, in _force_run
    return func(*args)
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'D:\\Sources\\_pythonish\\cpython\\main\\build\\test_python_50684\xe6'
Kill <WorkerThread #1 running test=test_external_inspection pid=67748 time=11.2 sec> process
Kill <WorkerThread #2 running test=test_external_inspection pid=75760 time=11.2 sec> process
Kill <WorkerThread #3 running test=test_external_inspection pid=37312 time=11.2 sec> process
Kill <WorkerThread #4 running test=test_external_inspection pid=54008 time=136 ms> process
Kill <WorkerThread #5 running test=test_external_inspection pid=71532 time=11.2 sec> process
Kill <WorkerThread #6 running test=test_external_inspection pid=75024 time=908 ms> process
Kill <WorkerThread #7 running test=test_external_inspection pid=76908 time=11.2 sec> process
Kill <WorkerThread #8 running test=test_external_inspection pid=67792 time=11.2 sec> process
Kill <WorkerThread #9 running test=test_external_inspection pid=77664 time=111 ms> process
Kill <WorkerThread #11 running test=test_external_inspection pid=77316 time=11.2 sec> process
Kill <WorkerThread #13 running test=test_external_inspection pid=77100 time=11.2 sec> process
Kill <WorkerThread #14 running test=test_external_inspection pid=67208 time=11.1 sec> process
Kill <WorkerThread #15 running test=test_external_inspection pid=70904 time=11.1 sec> process
Kill <WorkerThread #16 running test=test_external_inspection pid=76680 time=2.4 sec> process
Kill <WorkerThread #17 running test=test_external_inspection pid=69020 time=11.1 sec> process
Kill <WorkerThread #18 running test=test_external_inspection pid=68684 time=1.8 sec> process
Kill <WorkerThread #20 running test=test_external_inspection pid=71156 time=973 ms> process
Kill <WorkerThread #21 running test=test_external_inspection pid=77156 time=11.1 sec> process
Kill <WorkerThread #22 running test=test_external_inspection pid=62752 time=11.1 sec> process
Kill <WorkerThread #23 running test=test_external_inspection pid=74376 time=11.1 sec> process
Kill <WorkerThread #24 running test=test_external_inspection pid=59932 time=1.8 sec> process
Kill <WorkerThread #25 running test=test_external_inspection pid=71704 time=11.1 sec> process
Kill <WorkerThread #26 running test=test_external_inspection pid=76612 time=11.1 sec> process
Kill <WorkerThread #27 running test=test_external_inspection pid=76964 time=2.3 sec> process
Kill <WorkerThread #28 running test=test_external_inspection pid=76940 time=11.1 sec> process
Kill <WorkerThread #29 running test=test_external_inspection pid=77228 time=1.9 sec> process
Kill <WorkerThread #30 running test=test_external_inspection pid=56868 time=11.1 sec> process
Kill <WorkerThread #31 running test=test_external_inspection pid=76344 time=11.1 sec> process
Kill <WorkerThread #32 running test=test_external_inspection pid=77052 time=11.1 sec> process
Kill <WorkerThread #33 running test=test_external_inspection pid=61188 time=11.1 sec> process
Kill <WorkerThread #34 running test=test_external_inspection pid=60384 time=11.1 sec> process
Kill <WorkerThread #35 running test=test_external_inspection pid=74884 time=107 ms> process
Kill <WorkerThread #36 running test=test_external_inspection pid=36240 time=11.0 sec> process
Kill <WorkerThread #37 running test=test_external_inspection pid=77264 time=11.0 sec> process
Kill <WorkerThread #38 running test=test_external_inspection pid=77176 time=11.0 sec> process
Kill <WorkerThread #39 running test=test_external_inspection pid=68476 time=11.1 sec> process
Kill <WorkerThread #40 running test=test_external_inspection pid=70452 time=11.0 sec> process

== Tests result: FAILURE ==

1 test failed:
    test_external_inspection

    That's unfortunately another problem that's unique for windows. If you check the bug tracker you will see is a race between processes and the temporary directory

  7. pablogsal
    added 2 commits that reference this issue on Jul 7, 2025

    pythongh-136186: Fix race condition in test_external_inspection.test_…

    c61e93c

    gh-136186: Fix race condition in test_external_inspection.test_only_a…

    ba9c198
  8. miss-islington
    added a commit that references this issue on Jul 8, 2025

    pythongh-136186: Fix race condition in test_external_inspection.test_…

    b3f3c34
  9. pablogsal
    added a commit that references this issue on Jul 8, 2025

    [3.14] gh-136186: Fix race condition in test_external_inspection.test…

    78c1227
  10. AndPuQing
    added a commit that references this issue on Jul 11, 2025

    pythongh-136186: Fix race condition in test_external_inspection.test_…

    c2a485f
  11. Pranjal095
    added a commit that references this issue on Jul 12, 2025

    pythongh-136186: Fix race condition in test_external_inspection.test_…

    cb92cb6
  12. picnixz
    added a commit that references this issue on Jul 13, 2025

    pythongh-136186: Fix race condition in test_external_inspection.test_…

    6042500
  13. kumaraditya303
    closed this as completedon Jul 16, 2025

  14. 1 remaining item

  15. Agent-Hellboy
    added a commit that references this issue on Aug 19, 2025

    pythongh-136186: Fix race condition in test_external_inspection.test_…

    1fb4e05
  16. kumaraditya303
    added a commit that references this issue on Sep 9, 2025

    [3.14] pythongh-136186: Fix race condition in test_external_inspectio…

    da1a1c0

  17. colesbury commented on Dec 23, 2025

    @colesbury
    colesbury
    on Dec 23, 2025
    ContributorAuthor

    I'm still seeing flakiness with this test on CI. For example, on a PR (Hypothesis tests on Ubuntu):

    https://github.com/python/cpython/actions/runs/20466572446/job/58811763808?pr=143109

    ======================================================================
ERROR: test_only_active_thread (test.test_external_inspection.TestGetStackTrace.test_only_active_thread)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/runner/work/cpython/cpython-ro-srcdir/Lib/test/test_external_inspection.py", line 1712, in test_only_active_thread
    all_traces = unwinder_all.get_stack_trace()
RuntimeError: Failed to parse initial frame in chain

----------------------------------------------------------------------
  18. colesbury
    reopened this on Dec 23, 2025

  19. pablogsal commented on Dec 23, 2025

    @pablogsal
    pablogsal
    on Dec 23, 2025
    Member

    Let me look into it more

  20. pablogsal commented on Dec 23, 2025

    @pablogsal
    pablogsal
    on Dec 23, 2025
    Member

    Hopefully #143110 does the trick

  21. pablogsal
    added 4 commits that reference this issue on Dec 23, 2025

    Merge remote-tracking branch 'upstream/main' into pythongh-136186

    9309fbc

    Merge remote-tracking branch 'upstream/main' into pythongh-136186

    0d68f47

    Merge remote-tracking branch 'upstream/main' into pythongh-136186

    48cba1f

    gh-136186: Fix flaky tests in test_external_inspection (#143110)

    9af7a20

  22. pablogsal commented on Dec 24, 2025

    @pablogsal
    pablogsal
    on Dec 24, 2025
    Member

    I ran buildbots in #143110 3 times and they were happy with it. Also I stress tested it locally and didn't see any failure so hopefully this is it

  23. pablogsal
    closed this as completedon Dec 24, 2025
  24. pablogsal
    added 2 commits that reference this issue on Dec 28, 2025

    pythongh-136186: Fix more flaky tests in test_external_inspection

    323ecfd

    gh-136186: Fix more flaky tests in test_external_inspection (#143235)

    836b281
  25. thunder-coding
    added a commit that references this issue on Feb 15, 2026

    pythongh-136186: Fix more flaky tests in test_external_inspection (py…

    e7bbfbe
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    testsTests in the Lib/test dirtype-bugAn unexpected behavior, bug, or error

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions