Repository navigation
Fatal Python error "XXX block stack overflow" when exception stacks >10 #84115
Description
| BPO | 39934 |
|---|---|
| Nosy | @ambv, @markshannon, @serhiy-storchaka, @pablogsal, @myzhang1029, @iritkatriel |
| PRs | |
| Files |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = None
closed_at = <Date 2021-05-05.10:53:27.428>
created_at = <Date 2020-03-11.13:24:52.447>
labels = ['interpreter-core', '3.10', '3.8', '3.9', 'type-crash']
title = 'Fatal Python error "XXX block stack overflow" when exception stacks >10'
updated_at = <Date 2021-05-05.10:53:27.427>
user = 'https://github.com/myzhang1029'bugs.python.org fields:
activity = <Date 2021-05-05.10:53:27.427>
actor = 'iritkatriel'
assignee = 'none'
closed = True
closed_date = <Date 2021-05-05.10:53:27.428>
closer = 'iritkatriel'
components = ['Interpreter Core']
creation = <Date 2020-03-11.13:24:52.447>
creator = 'myzhang1029'
dependencies = []
files = ['48969']
hgrepos = []
issue_num = 39934
keywords = ['patch']
message_count = 18.0
messages = ['363914', '377433', '377434', '377437', '377439', '377440', '377441', '377444', '377445', '377490', '378178', '378192', '381280', '385035', '385037', '385039', '385042', '393002']
nosy_count = 6.0
nosy_names = ['lukasz.langa', 'Mark.Shannon', 'serhiy.storchaka', 'pablogsal', 'myzhang1029', 'iritkatriel']
pr_nums = ['22395', '23303']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'crash'
url = 'https://bugs.python.org/issue39934'
versions = ['Python 3.8', 'Python 3.9', 'Python 3.10']
Activity
myzhang1029 commentedon Mar 11, 2020
I apologize for describing this issue badly, but I'll try anyway.
The code to demonstrate the issue is attached, so it might be better to read that instead.
I noticed that when more than 10 exceptions are raised sequentially (i.e. one from another or one during the handling of another), the interpreter crashes saying "Fatal Python error: XXX block stack overflow".
This happens in python 3.7, 3.8 and development(git 39c3493) versions, but not in python2.7. Using ipython also fixes this issue.
I know this case is rare, but the maximum number of recursions is more than 2000, and the maximum number of statically nested blocks sepcified in frameobject.c is 20, so I'm pretty sure this isn't intended behavior.
iritkatriel commentedon Sep 23, 2020
The error is coming from here:
cpython/Objects/frameobject.c
Line 958 in cb9879b
This is not about recursion or about exception, it's about static nesting level.
There is an example here showing an input that gives the same error with >20 nested while blocks:
cpython/Lib/test/test_syntax.py
Line 545 in c8f29ad
iritkatriel commentedon Sep 23, 2020
In summary, I think this is not-a-bug.
serhiy-storchaka commentedon Sep 24, 2020
It is a bug. Compiler explicitly checks if the number of nested "try" blocks does not exceed the limit of CO_MAXBLOCKS, but it does not count implicit "try" blocks inserted when your assign an exception in the "except" clause.
is actually translated to
So we have double number of nested "try" blocks.
9 remaining items
iritkatriel commentedon Sep 24, 2020
Another oddity:
This gives the error:
for x in '1':
for x in '2':
for x in '3':
for x in '4':
for x in '5':
for x in '6':
for x in '8':
for x in '9':
for x in '10':
for x in '11':
for x in '12':
for x in '13':
for x in '14':
for x in '15':
for x in '16':
for x in '17':
for x in '18':
for x in '19':
for x in '20':
for x in '21':
for x in '22':
pass
but this doesn't:
markshannon commentedon Sep 24, 2020
iritkatriel
What error do you see and on what version?
iritkatriel commentedon Sep 24, 2020
On windows 10, master:
That's when x.py has the nested for loops without else. The error goes away if I add the else.
markshannon commentedon Sep 25, 2020
New changeset 02d126a by Mark Shannon in branch 'master':
bpo-39934: Account for control blocks in 'except' in compiler. (GH-22395)
02d126a
iritkatriel commentedon Oct 7, 2020
After studying Mark's patch and the rest of the code, I understand the for loop oddity.
The else block of the for loop comes after the
compiler_pop_fblock(c, FOR_LOOP, start);
So you can do this all day and it won't complain:
I guess the same goes for while loops, and the else of a try-except.
Since If blocks were deliberately left out of this game, I'm assuming that this "static nesting" is actually number of frames, rather than true static nesting. If that is the case then there is no issue here and we can close this ticket.
But if this is something to be fixed, then I am happy to make a patch along the lines of Mark's (which I partially already have).
pablogsal commentedon Oct 7, 2020
PR 22395 should be backported to 3.8 and 3.9
markshannon commentedon Nov 17, 2020
New changeset 48a9c0e by Irit Katriel in branch '3.9':
[3.9] bpo-39934: Account for control blocks in 'except' in compiler. (GH-22395) (GH-23303)
48a9c0e
markshannon commentedon Jan 13, 2021
Does this need backporting to 3.8, or is 3.9 sufficient?
iritkatriel commentedon Jan 13, 2021
There were additional merge conflicts when I tried to create a manual 3.8 backport, more significant than the 3.9 ones IIRC.
markshannon commentedon Jan 13, 2021
Pablo, are you OK closing this without a 3.8 backport?
pablogsal commentedon Jan 13, 2021
I would be supportive, but we should check with Łukasz as he is the release manager of 3.8
iritkatriel commentedon May 5, 2021
This was fixed in 3.9+ and 3.8 is in security fix now, so closing.