Tkinter is not thread safe

@loewis

BPO	11077
Nosy	@loewis, @terryjreedy, @amauryfa, @abalkin, @pitrou, @asvetlov, @serhiy-storchaka
Files	TkinterCrash2.py TkinterCrash3.py TkinterCrash2.py tk_v_raw.py: Short python script attempting raw input and tkInter mainloop TkinterCrash2-2.py

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2014-07-30.07:45:13.416>
created_at = <Date 2011-01-31.15:40:32.008>
labels = ['expert-tkinter', 'type-crash']
title = 'Tkinter is not thread safe'
updated_at = <Date 2018-05-13.04:22:27.761>
user = 'https://bugs.python.org/PythonInTheGrass'

bugs.python.org fields:

activity = <Date 2018-05-13.04:22:27.761>
actor = 'terry.reedy'
assignee = 'none'
closed = True
closed_date = <Date 2014-07-30.07:45:13.416>
closer = 'serhiy.storchaka'
components = ['Tkinter']
creation = <Date 2011-01-31.15:40:32.008>
creator = 'PythonInTheGrass'
dependencies = []
files = ['20630', '20632', '20684', '33637', '35927']
hgrepos = []
issue_num = 11077
keywords = []
message_count = 26.0
messages = ['127604', '127611', '127638', '127645', '127648', '127962', '127979', '127981', '127982', '127985', '127988', '127991', '127993', '127995', '127996', '183774', '184521', '208862', '208870', '208871', '222644', '222647', '222648', '222825', '316240', '316449']
nosy_count = 11.0
nosy_names = ['loewis', 'terry.reedy', 'amaury.forgeotdarc', 'belopolsky', 'pitrou', 'asvetlov', 'cgohlke', 'PythonInTheGrass', 'serhiy.storchaka', 'Honest Abe', 'Keith.Bannister']
pr_nums = []
priority = 'normal'
resolution = 'works for me'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'crash'
url = 'https://bugs.python.org/issue11077'
versions = ['Python 2.7']

The more I look at GUI support in Python, the more I realize that the lack of basic thread safety in GUI support is simply a bug. I know Java's Swing has the same thread limitation, but that doesn't make it right. Xlib is thread safe. The Windows SDK is thread safe. Python is supposed to be the language that's easy to use, and there is nothing easy about teaching new programmers that they have to mess with queues and timers just to get a basic set of displays running, just because when threads are in use.

I'm in the position of teaching folk with little-to-no programming experience, how to script simple applications in Python. The modules they have to use are inherently threaded, and delivery hunks of data from multiple sources to them. The most natural instinct is to put up some graphs and other widgits to display the data, and all of it is completely trivial right up until I have to explain that drawing a line isn't canvas.line(from, to), but becomes an exercise in Queue.Queue and theRoot.after(n, myself), before you even get to learn about widgits. Threading is supposed to simplify problems, not add to them. Having to hack around with special timers and polling, just to get some simple graphs up, is plain unpythonic.

Please consider this a bug, a glaring misfeature, in a language that is otherwise a very reasonable choice to get technical but non-programmerish people into toolmaking self-sufficiency.

One possible solution was presented in bpo-1252236: move tkinter event loop into Python main loop. However, to consider this report a bug, we need an example code that shows the behavior that you consider incorrect. Even then, it is likely that fixing the behavior won't be possible without introducing new features. I am reclassifying this as a feature request.

I don't have an opinion on 1252236. I'm not certain it would help.

I have an extension that runs a bunch of (alien) threads into Python code. The threads deliver information for all sorts of real world events, asynchronously. Multiple threads are used, because people occasionally want to do blocking operations in these alien threads, but still want to be able to handle other incoming information at full speed, with the other threads.

If any of these threads attempts to update a Tkinter widgit - and this is the first thing I tried to do - then tkinter will except or crash in some horrible way, sooner or later.

The attached .py (in Python 2.7.1) does it without any extensions. Click on Launch quickly 10-15 times; if it doesn't crash, kill and restart. It rarely takes more than 4-5 runs to get a traceback, or occasionally python itself just crashes. Bottom line, one thread in .mainloop() and another thread calling virtually any Tkinter function, even something as simple as .after(), is an invitation a crash.

Hmm, either my hand is too slow or my laptop is too fast, but I cannot reproduce the crash. Can you create a non-interactive script? Maybe start a separate thread generating "launch" events?

What do you mean by "crash"? Do you get a python backtrace or OS diagnostic saying that the process was killed by a signal of some sort? If it is the former, please post the backtrace. If the latter, can you produce C backtrace?

I converted TkinterCrash2.py to 3.x using 2to3 (result attached as TkinterCrash3.py) and it works with 3.2rc2 just fine.

'Thread' is not found in the somewhat skimpy tkinter doc. Given the general state of Python with threads, I think it reasonable to take 'not thread-safe' as the default, making this a feature request. In any case, the point is moot until there is a tested fix. The use case presented, though, does make this look like a desireable new feature.

My claim is that Tkinter is thread-safe as it stands. A lot of thought has been put into making Tkinter thread-safe, so if there is any claim to the contrary, we would need more details: what exact Python version is being used, what exact operating system is being used, what exact code is run, and what exact output is produced.

I'll look into making the crash easier to reproduce this coming week.

Is Tkinter's thread safety new? Because after I started getting crashes, I did my due diligence in Google and found a number of people writing about how it was necessary to use a Queue and Tkinter's after() timer, to draw from multiple threads. And in experiments in 2.7.1 on Windows, it turned out that it wasn't just the drawing calls from a thread other than the mainloop() thread, that invited a crash; even just trying to set an .after() timer from a different thread caused tracebacks, randomly.

At any rate this is less of an issue now, for me at least. I'm putting together a small graphic library for my coworkers. It will have a tiny fraction of Tkinter's capabilities, it will be Windows-only and won't even have native Windows look and feel; but it will be completely, rock-solid, idiot-proof thread safe - any operation (including widget deletes), any thread, any time, with no polling or timers needed. My coworkers are not technical and will get instantly lost if I have to describe queues and timers just to draw a line on a screen.

The "UpdateStringProc should not be invoked for type cmdName" message (as quoted above in the traceback) apparently can mean that there's a mismanagement of Tcl reference counts.

From http://sourceforge.net/tracker/?func=detail&atid=110894&aid=1326087&group_id=10894:

“This crash (actually, a panic) hints at defective Tcl_Obj
handling - possibly in the core, more likely in an extension
if you're using one. It indicates that Tcl_GetString() or
Tcl_GetStringFromObj() has been called on a cmdNameType
Tcl_Obj that has no string representation, a state that
should never occur.”

Intuitively, cmdNameType seems to refer to createcommand() / deletecommand(). Also, the following code in Tkinter.py looks a bit suspicious:

    def after(self, ms, func=None, *args):
        """Call function once after given time.

    MS specifies the time in milliseconds. FUNC gives the
    function which shall be called. Additional parameters
    are given as parameters to the function call.  Return
    identifier to cancel scheduling with after_cancel."""
    if not func:
        # I'd rather use time.sleep(ms*0.001)
        self.tk.call('after', ms)
    else:
        def callit():
            try:
                func(*args)
            finally:
                try:
                    self.deletecommand(name)
                except TclError:
                    pass
        name = self._register(callit)
        return self.tk.call('after', ms, name)

That is, we call deletecommand() while the command is still being executed.
Perhaps that, together with concurrent memory allocation from another thread (Tcl_Alloc() doesn't use Python's "Tcl lock"), might explain why the cmdName or other things sometimes become corrupted ("Tcl_DeleteCommand deletes a command from a command interpreter. Once the call completes, attempts to invoke cmdName in interp will result in errors").

As a side note, if Tkinter is intended to be thread safe, the
documentation should say so. Clearly, and in the first paragraph.

I'm no Tkinter specialist but, judging by its source code, Tkinter (the Python module) *is* intended to be thread-safe using locking and/or marshalling.

As for wxPython, you are right that it is not thread safe.

I am somewhat puzzled by the claim that tkinter *is* thread safe (as opposed to *intended to be*), in the sense of accessing the gui from multiple threads.

First, http://www.astro.washington.edu/users/rowen/TkinterSummary.html
"all Tkinter access must be from the main thread (or more precisely,
from the thread that calls the mainloop). Violating this is likely to
cause nasty and mysterious symptoms such as freezes and core dumps."

This statement must be based on some sort of experience.

Second, there have been many reports of problems with tkinter and threads. From currently open tracker issues:
bpo-11029 Same Scott M as here fixed problem by feeding data to single gui thread with queue.
bpo-16823 I turned this into a doc issue to say (perhaps erroneously) that tkinter is not thread safe, quoting the above.
bpo-6933 I closed this as invalid because of using multiple threads.
bpo-10909 This is ambiguous because problem continued after stopping 2nd thread access.
I believe these have most of the details requested.

There have been posts on python-list where people were advised to and solved problems by only accessing tkinter from one thread, and the main thread at that. Ditto for Stackoverflow: searching [python tkinter threads] gets 368 hits. Just one example:
http://stackoverflow.com/questions/13753545/tkinter-freeze-on-grid-remove/13753865#13753865 from Adrian Larsen got this response:

"Tkinter is not thread safe. If you do anything in a thread other than the main thread that touches a GUI object then you will get unpredictable results. ..." (Brian Oakley, experienced responder)

Adrian responded "... The real problem is that it is not in the correct thread (i.e. the mainloop). I solved the problem by sending the ungrid order to a queue in the GUI, which then issues the command from inside the mainloop."

I have been researching this due to the recurring recommendations, on stackoverflow.com, to use the queue.Queue + after() technique when using multiple threads.

———

From comments in _tkinter.c:

The Tcl interpreter is only valid in the thread that created it,
and all Tk activity must happen in this thread, also.
That means that the mainloop must be invoked in the thread that
created the interpreter. Invoking commands from other threads is possible; _tkinter will queue an event for the interpreter thread, which will then execute the command and pass back the result. If the main thread is not in the mainloop, and invoking commands causes an exception; if the main loop is running but not processing events, the command invocation will block.

http://hg.python.org/cpython/file/ef8ea052bcc4/Modules/_tkinter.c#l174

/* This is the main entry point for calling a Tcl command.
It supports three cases, with regard to threading:

Tcl is not threaded: Must have the Tcl lock, then can invoke command in
the context of the calling thread.
Tcl is threaded, caller of the command is in the interpreter thread:
Execute the command in the calling thread. Since the Tcl lock will
not be used, we can merge that with case 1.
Tcl is threaded, caller is in a different thread: Must queue an event to
the interpreter thread. Allocation of Tcl objects needs to occur in the
interpreter thread, so we ship the PyObject* args to the target thread,
and perform processing there. */

http://hg.python.org/cpython/file/ef8ea052bcc4/Modules/_tkinter.c#l1314

———

The following quote seems to imply that it is thread safe:

"Invoking commands from other threads is possible;
_tkinter will queue an event for the interpreter thread,
which will then execute the command and pass back the result."

But... Allen B. Taylor, author of mtTkinter, states:

The problems stem from the fact that the _tkinter module attempts to gain control of the main thread via a polling technique when processing calls from other threads. If it succeeds, all is well. If it fails (i.e., after a timeout), the application receives an exception with the message: "RuntimeError: main thread is not in main loop".

http://tkinter.unpythonic.net/wiki/mtTkinter

This seems to be somewhat validated by the first quote from the source code.
However, the statement "main thread is not in main loop"
seems backwards to me. Wouldn't a loop be in a thread?
How can the thread that spawned the mainloop ever be inside of the mainloop? Note that all of my experience is with Python, not C.

Hi,

I'm trying to make an application with GNU readline support, and some plots. But it seems that tkInter and raw_input don't play nicely.

The attached script (18 lines) crashes immediately on my Mac with SIGABRT and outputs the following:

$ python tk_v_raw.py
Yes? > Tcl_WaitForEvent: Notifier not initialized
Abort trap: 6

Is this related to this issue?

tk_v_raw.py is not obviously related to this issue. First, you only use tkinter in the main thread. Second, it works on Windows 7-64, 3.4 Idle.
>>> 
Yes? > abc
Got cmd abc
Yes? > def
Got cmd def
Yes? > 
It also runs in the console on 2.7, 3.3, 3.4. The main problem is that closing the Label window does not stop while True: in the thread, making a graceful stop difficult. The loop needs
        if cmd == 'q': break

Keith, did you install the appropriate ActiveState tcl/tk, as described here: http://www.python.org/download/mac/tcltk/

Is this still a problem with 2.7.6, which has a few hundred bug fixes since 2.7.1.

Is this still an issue with 2.7.8 which has presumably had more fixes since 2.7.6?

Summarizing other messages, it seems that tkinter is intended to be thread-safe (as long tk.mainloop is started in the tk thread) and mostly is until it is not (probably from timing issue). With fresh 2.7.8 install, I ran TkinterCrash2.py. I got the following twice:

Exception in thread Thread-12:
Traceback (most recent call last):
  File "C:\Programs\Python27\lib\threading.py", line 810, in __bootstrap_inner
    self.run()
  File "tkcrash2.py", line 49, in run
    self.deliverToQueue((self.target, z, y))
  File "tkcrash2.py", line 131, in arrival_122
    new_yz[1])
  File "C:\Programs\Python27\lib\lib-tk\Tkinter.py", line 2282, in create_line
    return self._create('line', args, kw)
  File "C:\Programs\Python27\lib\lib-tk\Tkinter.py", line 2270, in _create
    *(args + self._options(cnf, kw))))
ValueError: invalid literal for int() with base 10: 'None'

and the progrm continued. After another 100-200 launches, I tried to close the tk window and got the "python has stopped working" message from Windows. After this, every time I restart, I soon got the message again. Something seems to have been altered, though I do not see a relevant process in task manager.

I closed and reopened the command prompt process and restarted with pythonw instead of python and the program ran until too sluggish to continue. I then closed and reran and got 'pythonw has stopped working' at about launch 5. Third time ran until I closed at 50 launches. Fourth time stopped after 10 launches.

Serhiy, have any fresh insight?

With 3.4.1, (64 bit), I ran, saturated (200 launches?) and closed 4 windows without incident. This extends Alexander's claim, 'not 3.x problem', in msg127648, from 3.2 to 3.4.

I can't reproduce the bug on 2.7.8+, but on 2.7.3 I sometimes got messages like "Exception in thread Thread-26 (most likely raised during interpreter shutdown):" or:

File "/usr/lib/python2.7/lib-tk/Tkinter.py", line 2204, in create_line
File "/usr/lib/python2.7/lib-tk/Tkinter.py", line 2184, in _create
<type 'exceptions.TypeError'>: 'NoneType' object is not callable

This points to shutdown issue. When TkinterCrash2.py call quit(), other threads are still work. And when modules (including Tkinter and __main__) are cleaned, they try to call global functions which are set to None. After fixing TkinterCrash2.py to not invoke any code during shutdown, error messages on 2.7.3 gone.

I don't see any bugs in Tkinter exposed by this script, only a bug in the script itself.

Here is a script with quick and dirty fix. This solution is not perfect, there is small chance of race conditions (happened once in about 500 runs). Correct solution should be more complicated. If there no other errors in TkinterCrash2-2.py, this issue can be closed.

This issue was effectively reopened by bpo-33257, with altered versions of TkinterCrash2-2. Serhiy's original thereof fails for me on 2.7.15. I will put details on the new issue.

The information missing from this issue is that tcl and tk can be compiled without and with thread support, and that the latter became default in 8.6, which is included with recent 3.x versions. Hence the launch example, as posted on bpo-33257, works for me in 3.7 but not 2.7.

I opened tkinter doc bpo-33479, with this additional info, as a followup to msg183774 above.

Navigation Menu

Repository navigation

Tkinter is not thread safe #55286

7 remaining items

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Tkinter is not thread safe #55286

Description

Activity

PythonInTheGrass commented on Jan 31, 2011

abalkin commented on Jan 31, 2011

PythonInTheGrass commented on Jan 31, 2011

abalkin commented on Jan 31, 2011

abalkin commented on Jan 31, 2011

terryjreedy commented on Feb 5, 2011

loewis commented on Feb 5, 2011

PythonInTheGrass commented on Feb 5, 2011

7 remaining items

pitrou commented on Feb 5, 2011

pitrou commented on Feb 5, 2011

terryjreedy commented on Mar 8, 2013

HonestAbe commented on Mar 18, 2013

KeithBannister commented on Jan 22, 2014

terryjreedy commented on Jan 23, 2014

terryjreedy commented on Jan 23, 2014

BreamoreBoy commented on Jul 9, 2014

terryjreedy commented on Jul 10, 2014

terryjreedy commented on Jul 10, 2014

serhiy-storchaka commented on Jul 12, 2014

terryjreedy commented on May 6, 2018

terryjreedy commented on May 13, 2018

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions