Skip to content

test_pyrepl: test_cursor_back_write() blocks on input_hook() when tests are run sequentially #121008

Description

@vstinner

Command:

$ ./python -m test -u all,-gui 
(...)
0:24:10 load avg: 0.92 [326/478] test_pyclbr
0:24:13 load avg: 0.92 [327/478] test_pyexpat
0:24:13 load avg: 0.92 [328/478] test_pyrepl

gdb:

(gdb) where
#0  0x00007f3b0325ec37 in select () from /lib64/libc.so.6
#1  0x00007f3af40f4497 in Sleep (milli=<optimized out>) at ./Modules/_tkinter.c:371
#2  0x00007f3af40f4faa in EventHook () at ./Modules/_tkinter.c:3350
#3  0x0000000000666d86 in os__inputhook_impl (module=<optimized out>) at ./Modules/posixmodule.c:16795
#4  0x0000000000666dad in os__inputhook (module=<optimized out>, _unused_ignored=_unused_ignored@entry=0x0)
    at ./Modules/clinic/posixmodule.c.h:12134
(...)

(gdb) py-bt
Traceback (most recent call first):
  <built-in method _inputhook of module object at remote 0x7f3af53d9850>
  File "/home/vstinner/python/main/Lib/_pyrepl/reader.py", line 719, in handle1
    input_hook()
  File "/home/vstinner/python/main/Lib/test/test_pyrepl/support.py", line 75, in handle_all_events
    reader.handle1()
  File "/home/vstinner/python/main/Lib/test/test_pyrepl/test_unix_console.py", line 197, in test_cursor_back_write
    _, con = handle_events_unix_console(events)
  File "/home/vstinner/python/main/Lib/unittest/mock.py", line 1423, in patched
    return func(*newargs, **newkeywargs)
  File "/home/vstinner/python/main/Lib/unittest/case.py", line 606, in _callTestMethod
    result = method()
  File "/home/vstinner/python/main/Lib/unittest/case.py", line 660, in run
    self._callTestMethod(testMethod)
(...)

Linked PRs

Activity

  1. vstinner commented on Jun 25, 2024

    @vstinner
    MemberAuthor

    Reproducer with less tests:

    ./python -m test -u all,-gui test_idle test_pyrepl
    

    test_idle sets PyOS_InputHook in:

    (gdb) where
    #0  0x00007fffe96b927a in EnableEventHook () at ./Modules/_tkinter.c:3380
    #1  0x00007fffe96be707 in Tkapp_New (screenName=screenName@entry=0x0, className=className@entry=0x7fffe9cbf588 "Tk", 
        interactive=interactive@entry=0, wantobjects=wantobjects@entry=1, wantTk=wantTk@entry=0, sync=sync@entry=0, use=0x0)
        at ./Modules/_tkinter.c:736
    #2  0x00007fffe96be8f5 in _tkinter_create_impl (module=module@entry=<module at remote 0x7fffe96f5a90>, screenName=screenName@entry=0x0, 
        baseName=baseName@entry=0x7fffe8389328 "__main__", className=className@entry=0x7fffe9cbf588 "Tk", interactive=interactive@entry=0, 
        wantobjects=wantobjects@entry=1, wantTk=0, sync=0, use=0x0) at ./Modules/_tkinter.c:3176
    #3  0x00007fffe96bed69 in _tkinter_create (module=<module at remote 0x7fffe96f5a90>, args=args@entry=0x7ffff7fba430, nargs=nargs@entry=8)
        at ./Modules/clinic/_tkinter.c.h:820
    
    (gdb) py-bt
    Traceback (most recent call first):
      <built-in method create of module object at remote 0x7fffe96f5a90>
      File "/home/vstinner/python/main/Lib/tkinter/__init__.py", line 2459, in __init__
        self.tk = _tkinter.create(screenName, baseName, className, interactive, wantobjects, useTk, sync, use)
      File "/home/vstinner/python/main/Lib/tkinter/__init__.py", line 2572, in Tcl
        return Tk(screenName, baseName, className, useTk)
      File "/home/vstinner/python/main/Lib/idlelib/run.py", line 94, in <module>
        tcl = tkinter.Tcl()
      <built-in method exec of module object at remote 0x7ffff7c94770>
    (...)
    
  2. vstinner commented on Jun 25, 2024

    @vstinner
    MemberAuthor
  3. terryjreedy commented on Jun 26, 2024

    @terryjreedy
    Member

    I cannot reproduce on Windows. After some time, the test call above hung until I hit return, then it was a success.

    The apparently offending chain of events is that test_idle runs (idlelib.idle_test.)test_run which imports idlelib.run which calls tkinter.Tcl which at top level calls into _tkinter which sets tcl to get tk events. Idlelib.run is the only place IDLE calls tkinter.Tcl, but I suspect that calls to _tkinter from tkinter.tk do the same. However, IDLE test that call tk() do so explicitly and later call root.destroy(). Since Tcl.destroy exists, a fix might be to add the following top level cleanup function near the top of test_run.

    def tearDownModule():
        run.tcl.destroy()
    

    I'm off to bed.

  4. vstinner commented on Jun 26, 2024

    @vstinner
    MemberAuthor

    If I just add tearDownModule() to Lib/idlelib/idle_test/test_run.py, I get:

    ERROR: tearDownModule (idlelib.idle_test.test_run)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "/home/vstinner/python/main/Lib/idlelib/idle_test/test_run.py", line 433, in tearDownModule
        run.tcl.destroy()
        ~~~~~~~~~~~~~~~^^
      File "/home/vstinner/python/main/Lib/tkinter/__init__.py", line 2503, in destroy
        self.tk.call('destroy', self._w)
        ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
    _tkinter.TclError: invalid command name "destroy"
    
  5. terryjreedy commented on Jun 26, 2024

    @terryjreedy
    Member

    Tcl() returns a Tk object created with useTk False instead of True. So the tkinter destroy method is the one called normally.

    def Tcl(screenName=None, baseName=None, className='Tk', useTk=False):
        return Tk(screenName, baseName, className, useTk)
    

    But this and perhaps other methods are not valid for such object. This is a tkinter bug.

    Alternate try: in idlelib.run, replace lines 94-100

    tcl = tkinter.Tcl()
    
    def handle_tk_events(tcl=tcl):
        """Process any tk events that are ready to be dispatched if tkinter
        has been imported, a tcl interpreter has been created and tk has been
        loaded."""
        tcl.eval("update")
    

    with

    if idlelib.testing:
        def handle_tk_events(): pass
    else:
        tcl = tkinter.Tcl()
    
        def handle_tk_events(tcl=tcl):
            """Process any tk events that are ready to be dispatched if tkinter
            has been imported, a tcl interpreter has been created and tk has been
            loaded."""
            tcl.eval("update")
    

    This might not be a permanent solution but should work. I cannot test for several hours.

  6. added a commit that references this issue on Jun 26, 2024
  7. vstinner commented on Jun 26, 2024

    @vstinner
    MemberAuthor

    I wrote a fix based on @terryjreedy's recipe and it works for me!

  8. added a commit that references this issue on Jun 26, 2024
  9. added a commit that references this issue on Jun 26, 2024
  10. added a commit that references this issue on Jun 26, 2024
  11. vstinner commented on Jun 26, 2024

    @vstinner
    MemberAuthor

    Fixed by 44eafd6.

  12. vstinner commented on Jun 26, 2024

    @vstinner
    MemberAuthor

    Thank you @terryjreedy for the fix!

  13. terryjreedy commented on Jun 27, 2024

    @terryjreedy
    Member

    If I ever want to unittest run's handling of interactive tkinter calls, I will ask Serhiy to add a DisableEventHook to _tkinter, if not there already, and make it accessible from Python code, perhaps by making Tcl a subclass that defines a destroy method that works.

  14. added a commit that references this issue on Jun 30, 2024
  15. added a commit that references this issue on Jul 11, 2024
  16. added a commit that references this issue on Jul 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions