Repository navigation
curses crash on FreeBSD #51633
Description
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = None
closed_at = <Date 2010-07-17.12:41:55.048>
created_at = <Date 2009-11-23.20:03:46.996>
labels = ['extension-modules', 'type-bug']
title = 'curses crash on FreeBSD'
updated_at = <Date 2010-07-17.12:41:55.047>
user = 'https://github.com/mdickinson'bugs.python.org fields:
activity = <Date 2010-07-17.12:41:55.047>
actor = 'skrah'
assignee = 'none'
closed = True
closed_date = <Date 2010-07-17.12:41:55.048>
closer = 'skrah'
components = ['Extension Modules']
creation = <Date 2009-11-23.20:03:46.996>
creator = 'mark.dickinson'
dependencies = []
files = ['16935', '16963', '16973', '17023', '17050', '17064', '17528', '17997']
hgrepos = []
issue_num = 7384
keywords = ['patch', 'buildbot']
message_count = 52.0
messages = ['95652', '97709', '97722', '99657', '99658', '99659', '103231', '103256', '103261', '103263', '103264', '103265', '103267', '103295', '103307', '103308', '103393', '103394', '103395', '103429', '103432', '103497', '103503', '103828', '103838', '103980', '103996', '103997', '104000', '104002', '104054', '104057', '104070', '104071', '104074', '104283', '104302', '104311', '104315', '106199', '106939', '106940', '106948', '107323', '107999', '110222', '110224', '110225', '110238', '110271', '110378', '110550']
nosy_count = 8.0
nosy_names = ['akuchling', 'mark.dickinson', 'vstinner', 'asmodai', 'rpetrov', 'Arfrever', 'r.david.murray', 'skrah']
pr_nums = []
priority = 'normal'
resolution = 'accepted'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue7384'
versions = ['Python 2.6', 'Python 3.1', 'Python 2.7', 'Python 3.2']
Activity
mdickinson commentedon Nov 23, 2009
test_curses is currently causing the test runs to abort on the FreeBSD 6.4
and 7.2 buildbots.
I can reproduce this on a FreeBSD 7.2 /amd64 machine by doing
./python Lib/test/regrtest.py -uall test___all__ test_curses
This dumps core, and the traceback points at the call to delwin() in
PyCursesWindow_Dealloc, but it's far from obvious (to me) what's going
wrong. wo->win is not NULL here, and appears to point to a valid WINDOW.
However, stdscr is NULL! As far as I can tell, this shouldn't happen.
test_curses by itself doesn't crash, unless I add an 'import readline' or
'import rlcompleter' to the top of test_curses.py.
I expect to have access to the FreeBSD machine for a few more days. Any
hints about what to try next would be appreciated.
mdickinson commentedon Jan 13, 2010
I've not had any success tracking the cause of this failure down, and no longer have the resources to do so. It does appear that curses itself is broken on FreeBSD: it's not just a problem with the tests.
Adding Andrew Kuchling to the nosy in case he has any ideas what's wrong here.
Since the test_curses crash is currently aborting the test run, and so preventing us from getting feedback from the other tests on the FreeBSD buildbots, I propose that test_curses be skipped with a "the curses module is broken on FreeBSD" message.
[-]test_curses crash on FreeBSD buildbots[/-][+]curses crash on FreeBSD[/+]bitdancer commentedon Jan 13, 2010
Given your diagnosis so far, +1 on the skip.
mdickinson commentedon Feb 21, 2010
Rereading this, it doesn't say what I meant it to say: I meant that the Python curses module seems to be broken, not that the system-level curses library is broken (though that seems possible too).
mdickinson commentedon Feb 21, 2010
Applied the test_curses skip in r78281 (trunk); will merge to the other branches.
Leaving this issue open, since the root cause isn't fixed.
mdickinson commentedon Feb 21, 2010
Merged to the other 3 branches in revisions r78282 (release26-maint), r78283 (py3k), r78284 (release31-maint).
mdickinson commentedon Apr 15, 2010
I'm looking at this again, after installing FreeBSD 8.0/amd64 in a VM.
I've reduced Lib/test/test_curses.py to the following 9 lines:
I then get:
From looking at the core dump, and tracing through with gdb, the core dump occurs when delwin is called (from PyCursesWindow_Dealloc) on the result of curses.getwin(f), as a result of garbage collection.
The 'import rlcompleter' line appears to be necessary to cause this; I've no idea why.
mdickinson commentedon Apr 15, 2010
Here's the top of the backtrace. (Thanks asmodai for helping me out with working out how to build a FreeBSD system ncurses with debugging information.)
#0 0x0000000801460714 in cannot_delete (win=0x80116b1d0)
at /usr/src/lib/ncurses/ncursesw/../../../contrib/ncurses/ncurses/base/lib_delwin.c:54
p = (struct _win_list *) 0xdbdbdbdbdbdbdbdb
result = false
#1 0x0000000801460773 in delwin (win=0x80116b1d0)
at /usr/src/lib/ncurses/ncursesw/../../../contrib/ncurses/ncurses/base/lib_delwin.c:71
result = -1
#2 0x000000080170d140 in PyCursesWindow_Dealloc (wo=0x800eb74c0)
at /usr/home/dickinsm/python/svn/trunk/Modules/cursesmodule.c:357
No locals.
#3 0x000000000046325f in _Py_Dealloc (op=0x800eb74c0) at Objects/object.c:2211
dealloc = 0x80170d110 <PyCursesWindow_Dealloc>
#4 0x00000000004578d8 in PyDict_DelItem (op=0x800f121b0, key=0x8011062e0)
at Objects/dictobject.c:829
mp = (PyDictObject *) 0x800f121b0
hash = -3668919459648339544
ep = (PyDictEntry *) 0x8010cb5a8
old_value = (PyObject *) 0x800eb74c0
old_key = (PyObject *) 0x8011062e0
__func_ = "PyDict_DelItem"
#5 0x0000000000458a48 in dict_ass_sub (mp=0x800f121b0, v=0x8011062e0, w=0x0)
---Type <return> to continue, or q <return> to quit---
at Objects/dictobject.c:1184
No locals.
#6 0x000000000041aadd in PyObject_DelItem (o=0x800f121b0, key=0x8011062e0)
at Objects/abstract.c:205
m = (PyMappingMethods *) 0x6c2960
akuchling commentedon Apr 15, 2010
Could I get a login on the buildbot to make a fix?
I bet the problem is with the stdscr object. PyCurses_InitScr()
does 'return (PyObject *)PyCursesWindow_New(stdscr);'.
PyCursesWindow_Dealloc() does:
if (wo->win != stdscr) delwin(wo->win);
I bet FreeBSD is clearing contents of the stdscr global variable. The condition in PyCursesWindow_Dealloc() is then true, and it tries to delwin() the old value, which is in wo->win.
One fix might be to keep a reference to that PyCursesWindow object holding stdscr, and change dealloc to 'if (wo != saved_stdscr_object)'. Or maybe, since multiple calls to initscr() will create multiple window objects holding the value of stdscr, window objects should have a 'do_not_delwin' flag.
akuchling commentedon Apr 15, 2010
Here's a possible patch; it at least doesn't seem to break the module on MacOS, though MacOS doesn't crash with the current code either.
29 remaining items
vstinner commentedon May 20, 2010
I tested bpo-7384-5-py3k.patch on FreeBSD 8.0: it fixes the crash.
skrah commentedon Jun 3, 2010
I think it would be nice to get this into 2.7. I don't expect buildbot
failures, since the 2.7 patch is essentially the same as the py3k version,
which has been tested extensively.
mdickinson commentedon Jun 3, 2010
Agreed. I think you should go ahead and commit it.
skrah commentedon Jun 3, 2010
Mark, thanks. Committed in r81669; I'll keep an eye on the buildbots.
skrah commentedon Jun 8, 2010
Committed in r81669,r81672,r81683 (trunk) and r81830,81831 (py3k).
What to do with the releases? To recap, the fix is:
Detect if readline is already linked against ncurses and
if so, skip any further selection. This must be done.
Use the same version of ncurses for readline.so and _curses.so.
should be done in any case. 2) could change the behavior for
users who previously had readline/ncurses, cursesmodule/ncursesw,
but only use the cursesmodule in an application.
skrah commentedon Jun 17, 2010
Committed a conservative version implementing part 1) in r82017 (2.6) and
r82019 (3.1). Part 2) can be enabled by uncommenting a couple of lines in
setup.py.
The buildbots look good, but I'm setting this to 'pending' in case
someone would like part 2) of the fix in the releases.
Arfrever commentedon Jul 13, 2010
These changes break building of Python 3.* in some locales in Gentoo.
In lt_LT.UTF-8 locale, readline_termcap_lib file contains:
ne dinaminis paleidžiamasis failas
In en_US.UTF-8 locale, this file would contain:
not a dynamic executable
do_readline is "/usr/lib64/libreadline.so".
/usr/lib64/libreadline.so is a linker script with the following content:
/* GNU ld script
Since Gentoo has critical dynamic libraries in /lib, and the static versions
in /usr/lib, we need to have a "fake" dynamic lib in /usr/lib, otherwise we
run into linking problems. This "fake" dynamic lib is a linker script that
redirects the linker to the real lib. And yes, this works in the cross-
compiling scenario as the sysroot-ed linker will prepend the real path.
See bug http://bugs.gentoo.org/4411 for more info.
*/
OUTPUT_FORMAT ( elf64-x86-64 )
GROUP ( /lib64/libreadline.so.6 )
I think that using ldd is a wrong idea.
skrah commentedon Jul 13, 2010
In Ubuntu I can build just fine with lt_LT.UTF-8. So perhaps this problem
should be addressed in Gentoo.
Arfrever commentedon Jul 13, 2010
You shouldn't use ldd. I suggest that setup.py try to link a small executable, which would use a function from libcurses and would be linked against libreadline, but not libcurses. If linking succeeds, then you libreadline is linked against libcurses. If linking fails, then repeat this procedure with libcursesw, libncurses, libncursesw, libtinfo.
vstinner commentedon Jul 13, 2010
You can run ldd without LANG variable to get the original (english, ascii only) message.
skrah commentedon Jul 14, 2010
So you have garbage from stderr in readline_termcap_lib. Since that's
useless anyway (no matter what locale is set), let's check the return
value of os.system().
The attached patch skips readline linkage detection if ldd fails. In
that case, linking will be done in the same manner as before r81830.
Please report if the patch allows you to build py3k in the problematic
locale.
Your method of detecting readline linkage looks interesting, but I
doubt that I'm going to implement it: These cross platform issues
take an *immense* amount of time, since you have to test on all
buildbot platforms (+ OpenBSD and OpenSolaris), with different
compilers (icc, suncc).
If you want that done, the best way is to open another issue, submit a
patch (probably for configure.in) _and_ do all the testing.
Arfrever commentedon Jul 15, 2010
This patch allows to build Python 3.* in this locale.
It might be safer to open tmpfile in binary mode to avoid potential problems with non-ASCII characters in paths to libraries.
skrah commentedon Jul 17, 2010
ldd return value check committed in r82927, r82928, r82929 and r82930.
Thanks for reporting this!