Repository navigation
ast.literal_eval Segmentation Fault in Python 3.9/3.10 #126711
Description
Activity
@EgodPrime, please avoid creating duplicate issues.
FWIW, the 3.9 docs says: "It is possible to crash the Python interpreter with a sufficiently large/complex string due to stack depth limitations in Python’s AST compiler."
3.10+ docs are more vague: "It is possible to crash the Python interpreter due to stack depth limitations in Python’s AST compiler."
So, I'm not sure if we should count this as a bug.
Reacted by Gregory P. Smith@skirpichev, thanks for your reply. But I am wondering whether it means that Python 3.9/3.10, which is still in use in production, is not safe for such inputs.
Documentation (since 3.10) explicitly says, that ast.litaral_eval() is not safe for arbitrary input: "This function had been documented as “safe” in the past without defining what that meant. That was misleading. This is specifically designed not to execute Python code, unlike the more general eval(). There is no namespace, no name lookups, or ability to call out. But it is not free from attack: A relatively small input can lead to memory exhaustion or to C stack exhaustion, crashing the process. There is also the possibility for excessive CPU consumption denial of service on some inputs. Calling it on untrusted data is thus not recommended."
Reacted by Zachary Ware and Thiruvalluvan KamarajI see it. And I now think this is not a bug but just a poc to trigger the crash (C stack exhaustion) of the officially declared unsafe
ast.literal_eval.Hello! Thank you for the report. Since
ast.literal_evalhas been documented as a function that can potentially lead to interpreter crashes, I don't think it's worthwhile to fix it in 3.9. However, it seems reasonable to me to backport #95919 to 3.9 version. cc @gpsheadEDIT: I saw a Christian comment: #95588 (comment). Though, I can't say that I agree with it. This function was probably never safe and the documentation was just wrong.
Reacted by Sergey B KirpichevI'll prepare a patch.
Well, 3.9 is for security-fixes only. But here is a docs backport: #126729
Reacted by Kirill PodoprigoraI would like to consider this issue as a security problem because crash of interpreter can lead to DOS.
Reacted by Peter BiermaMy main concern here is that we can't undo documenting
ast.literal_evalas safe. Plenty of people are using it in seemingly unsafe places as a result, and I'm not sure it's a good idea to just stick a big bandaid on it for 3.9. If there's absolutely nothing we can do to fix it, then perhaps we could emit a warning when using it on 3.9?#126711 already backports the doc change to 3.9. So yes we can. We can never prevent people from writing code that doesn't behave as they desire, all we can do is make expectations more clear.
We cannot meaningfully add a warning no matter what branch because there are plenty of valid uses of
ast.literal_evaland it is impossible to tell which ones are and are not.Reacted by Kirill Podoprigora and Peter Bierma
Crash report
What happened?
The above code makes Python 3.9/3.10 crash with Segmentation Fault.
This error input can be correctly detected in Python 3.11+ with
RecursionError: maximum recursion depth exceeded during ast construction, but it still results into crash in 3.9.20 and 3.10.15.A smaller
lsuch as200will get the correct errorValue Error: malformed node or stringfor all Python 3.9+CPython versions tested on:
3.9, 3.10
Operating systems tested on:
Linux
Output from running 'python -VV' on the command line:
Python 3.9.20 (main, Oct 3 2024, 07:27:41) [GCC 11.2.0], Python 3.10.15 (main, Oct 3 2024, 07:27:34) [GCC 11.2.0]