Exceptions when a thread exits

@tim-one

BPO	754449
Nosy	@tim-one, @brettcannon, @doko42
Files	tracer.py: Contains trace_currentThread which logs to stderr when (threading.)?currentThread becomes None

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/brettcannon'
closed_at = <Date 2005-11-04.13:33:25.000>
created_at = <Date 2003-06-14.10:32:05.000>
labels = ['interpreter-core']
title = 'Exceptions when a thread exits'
updated_at = <Date 2005-11-04.13:33:25.000>
user = 'https://github.com/doko42'

bugs.python.org fields:

activity = <Date 2005-11-04.13:33:25.000>
actor = 'justincjohnson'
assignee = 'brett.cannon'
closed = True
closed_date = None
closer = None
components = ['Interpreter Core']
creation = <Date 2003-06-14.10:32:05.000>
creator = 'doko'
dependencies = []
files = ['919']
hgrepos = []
issue_num = 754449
keywords = []
message_count = 26.0
messages = ['16385', '16386', '16387', '16388', '16389', '16390', '16391', '16392', '16393', '16394', '16395', '16396', '16397', '16398', '16399', '16400', '16401', '16402', '16403', '16404', '16405', '16406', '16407', '16408', '16409', '16410']
nosy_count = 7.0
nosy_names = ['tim.peters', 'brett.cannon', 'doko', 'jgoerzen', 'carlosayam', 'gjb1002', 'justincjohnson']
pr_nums = []
priority = 'low'
resolution = 'fixed'
stage = None
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue754449'
versions = ['Python 2.3']

Logged In: YES
user_id=357491

Does anyone know how to cause this error in isolation? I have tried a
bunch of different ways but cannot cause an exception to be raised at the
correct point in Threading.__bootstrap() to lead to self.__stop() to be
called while the interpreter is tearing itself down.

Logged In: YES
user_id=357491

To force this error I inserted code to set all attributes of the threading
module to None and then throw and exception. After commenting out
the two calls to currentThread in _Condition the thrown exception to
trigger the problem propogated to the top correctly.

I did have the clean-up function give out since I set *everything* to
None, but it doesn't look like a normal issue as shown in the exception
traceback in the OP. I don't think it is an issue.

So it seems commenting the two calls to currentThread in _Condition
solves the problem. But since this is threading code and the comment
explicitly states it is for the side-effect I am going to try to get a second
opinion.

Logged In: YES
user_id=357491

OK, this has been fixed in Python 2.4 (rev. 1.41 of Lib/threading.py; just
removed the two calls to currentThread in the _Condition class that were
commented to be for their side-effect).

Just because I don't feel rock-solid on my fix to the point of it not
changing some obscure semantics that I would be willing to put a gun to
my head over it I am not going to backport this until I hear from
someone who had this bug and reports that they have not had any issues
for a long time. Otherwise I will just leave the fix in 2.4 only.

Logged In: YES
user_id=357491

OK, this has been fixed in Python 2.4 (rev. 1.41 of Lib/threading.py; just
removed the two calls to currentThread in the _Condition class that were
commented to be for their side-effect).

Just because I don't feel rock-solid on my fix to the point of it not
changing some obscure semantics that I would be willing to put a gun to
my head over it I am not going to backport this until I hear from
someone who had this bug and reports that they have not had any issues
for a long time. Otherwise I will just leave the fix in 2.4 only.

Logged In: YES
user_id=357491

Discovered this is not fixed after all (previous fix didn't go far enough;
still needed, though). Patch 954922 is the second attempt to fix this.
This time, though, I had code that could trigger the problem reliably and
thus this should be the proper, and final, fix.

Logged In: YES
user_id=357491

How to reproduce the bug:

Follow the instructions at http://mnetproject.org/repos/ on how to get a
copy of the in-dev version of mnet (I used the instructions for using wget
and it worked fine).
Run python setup.py. Do realize that this app will download other
Python code (crypto stuff mostly) to finish its install. Takes up about 54
MB on my machine after it is compiled.
Run python setup.py test -a. This executes the testing suite. The
bug manifests itself after the testing suite finishes execution.

This will consistently cause the bug every time.

Logged In: YES
user_id=990060

bcannon said on 2004-02-17

> Does anyone know how to cause this error in isolation?

I'm getting the same error when I start a new thread,
daemonise it and the thread goes into a very slow database
operation (loading a large object); meanwhile the main
thread starts a GUI; then I close the window, exiting the
main loop and the python interpreter ends (or tries to end.)

In relation to bcannon comment on how to reproduce the error
(setting all variables to None in the module), my guess is
that while exiting, the python interpreter is somehow
freeing all variables in the module (cleaning the module or
something), but the module is still running and that raises
the error... is this possible?

Note: if the thread is not daemonised, the problem desapears
but the script (the python interpreter) takes a while to finish.

Logged In: YES
user_id=357491

Yep, that is how Python cleans up a module; sets everything in the
module to None and then removes it from sys.modules .

As for your case, I don't know enough about daemonized threads. My
patch for this only tries to let the exception make it to the top without
'threading' hiding the exception by causing its own exception from
interpreter shutdown. In other words I don't know if this is a related
issue or not.

Logged In: YES
user_id=31435

Brett, FYI, a daemon thread differs from a non-daemon
thread in only one respect: Python shuts down when only
daemon threads remain. It waits for non-daemon threads to
finish. So a daemon thread can keep running after the
interpreter has torn itself down completely. For that reason,
problems in daemon threads doing non-trivial things are
almost guaranteed.

Logged In: YES
user_id=990060

I don't understand one thing: how can python shut down
itself and, at the same time, keep a python code running in
a daemonised thread? or is that thread running in a
different "threaded" python byte-code interpreter?

Logged In: YES
user_id=357491

Fixed in 2.4 thanks to rev. 1.42 to Lib/threading. Don't know if this will
be backported to 2.3 . Possible, but need someone else to agree first.

Logged In: YES
user_id=357491

For 2.3, rev.1.38.6.1 has the required fixes (required also backporting
rev. 1.41 from HEAD).

Thanks to everyone who tried to help with this nasty thing. And a special
thanks to the Mnet guys for giving me a way to reproduce this bug
consistently.

Logged In: YES
user_id=769182

I have this bug on Python 2.2.3 : occasionally getting a
very similar stack when my daemon thread exits. I don't get
the error for non-daemon threads. My problem seems pretty
similar to carlosayam's problem, described in June.

Logged In: YES
user_id=357491

Python 2.2 is no longer actively maintained, so I wouldn't count on the fix
being backported.

Logged In: YES
user_id=1049561

I'm seeing this problem still in Python 2.4.1 on Windows.

Python 2.4.1 (#65, Mar 30 2005, 09:13:57) [MSC v.1310 32
bit (Intel)] on win32

C:\src\Twisted>trial -r iocp twisted.test.test_iutils
Using twisted.internet.iocpreactor reactor
Running 6 tests.
twisted.test.test_iutils
UtilsTestCase
testOutput ... ?
[32;1m[OK]?[0m
testOutputAndValue ... ?
[32;1m[OK]?[0m
testOutputSignal ... ?
[32;1m[OK]?[0m

testOutputWithErrorCollected ... ?
[32;1m[OK]?[0m

testOutputWithErrorIgnored ... ?
[32;1m[OK]?[0m
testValue ... ?
[32;1m[OK]?[0m

-------------------------------------------------------------------------------
Ran 6 tests in 1.513s

PASSED (successes=6)
Exception in thread 
iocpreactor.process_waiter.ProcessWaiter.waitForProcessExi
t
pid=2156 (most likely raised during interpreter shutdown):
Traceback (most recent call last):
  File "C:\Python24\lib\threading.py", line 442, in __bootstrap
  File "C:\Python24\lib\threading.py", line 422, in run
  
File "C:\src\Twisted\twisted\internet\iocpreactor\process_wait
er.py", line 165, in doWaitForProcessExit
exceptions.TypeError: 'NoneType' object is not callable

Logged In: YES
user_id=357491

This is not Python's fault. If you look at what triggers
the exception it is code from Twisted. All of this is
happening because the threads being run by the test are not
being properly torn down before the interpreter begins to
tear itself down.

If you make sure all the threads have terminated before
ending the test I am sure the error will go away.

Logged In: YES
user_id=1049561

Oh, I see. I realized the interpreter was shutting down, but I
thought it was shutting down because of an exception in one
of the threads, and that the bug in Python was preventing me
from seeing the original exception raised in the thread. Now I
see the light.

Thanks for the response.

Navigation Menu

Repository navigation

Exceptions when a thread exits #38648

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Exceptions when a thread exits #38648

Description

Activity

brettcannon commented on Feb 17, 2004

brettcannon commented on Mar 3, 2004

brettcannon commented on Mar 8, 2004

brettcannon commented on Mar 8, 2004

brettcannon commented on May 16, 2004

brettcannon commented on May 17, 2004

carlosayam commented on Jun 3, 2004

brettcannon commented on Jun 3, 2004

tim-one commented on Jun 3, 2004

carlosayam commented on Jun 3, 2004

brettcannon commented on Jul 3, 2004

brettcannon commented on Jul 3, 2004

gjb1002 commented on Sep 17, 2004

brettcannon commented on Sep 17, 2004

justincjohnson commented on Nov 3, 2005

brettcannon commented on Nov 4, 2005

justincjohnson commented on Nov 4, 2005

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions