Python "zipfile" can't detect "quoted-overlap" zipbomb that can be used as a DoS attack #109858
Description
Bug report
Bug description:
Just found this vulnerability in the latest Python 3.11.5 (and previous 3.10.10).
If we craft a zipbomb using the "quoted-overlap" way (as mentioned https://www.bamsoftware.com/hacks/zipbomb/), this can't be detected by Python's zip file and the zip will be extracted and thus potentially cause a DoS attack by consuming all the storage.
This issue is related to CVE-2019-9674 but not the same. CVE-2019-9674 is talking about the "normal" overlap-zipbomb which is a "full" overlap. This can already be detected by Python's new version of zipfile. However, when we craft a "quoted-overlap" zip, as indicated by https://www.bamsoftware.com/hacks/zipbomb/, python can't detect and happily starts to extract.
For example, the following is the python to extract a zip file, 116 KB before extraction, goes to as large as 17GB after extraction. The size after extraction can be easily increased to multi TBs or even PBs by adjusting the zip-creation.
import zipfile
import sys
import os
def extract_zip(zip_path):
"""
Extracts the contents of a ZIP file to the current directory.
:param zip_path: Path to the ZIP file
"""
if not os.path.exists(zip_path):
print(f"Error: {zip_path} does not exist.")
return
with zipfile.ZipFile(zip_path, 'r') as zip_ref:
zip_ref.extractall()
print(f"Extracted contents of {zip_path} to the current directory.")
if __name__ == "__main__":
if len(sys.argv) != 2:
print("Usage: python extract_zip.py <path_to_zip_file>")
sys.exit(1)
zip_file_path = sys.argv[1]
extract_zip(zip_file_path)CPython versions tested on:
3.11
Operating systems tested on:
Linux
Linked PRs
- gh-109858: Protect zipfile from "quoted-overlap" zipbomb #110016
- [3.12] gh-109858: Protect zipfile from "quoted-overlap" zipbomb (GH-110016) #113912
- [3.11] gh-109858: Protect zipfile from "quoted-overlap" zipbomb (GH-110016) #113913
- [3.10] gh-109858: Protect zipfile from "quoted-overlap" zipbomb (GH-110016) #113914
- [3.9] gh-109858: Protect zipfile from "quoted-overlap" zipbomb (GH-110016) #113915
- [3.8] gh-109858: Protect zipfile from "quoted-overlap" zipbomb (GH-110016) #113916
- Add @requires_zlib() decorator for gh-109858 tests #113918
Metadata
Metadata
Assignees
Labels
Projects
- Status
Activity
gpshead commentedon Sep 25, 2023
Additional context: This was reported to us on the Python Security Response Team on 2023-09-06. We ultimately concluded it should be handled in public. The form of zip file attack is not "new" (as evidence by that zipbomb link) and several other zip implementations have had to deal with this over the years. Ours still needs to.
I'll summarize more of our discussions there in a followup comment.
gpshead commentedon Sep 25, 2023
While we do at least have an existing warning on ZipFile.extractall() in our docs "Warning: Never extract archives from untrusted sources without prior inspection." - that text goes on to focus on other reasons. We expect that there are people who don't actually understand the full implications of handling untrusted compressed complex-referential data like zips and just call an API like
.extractall()without sufficient checks regardless.@SethMichaelLarson & @di looked into PyPI's warehouse to check how PyPI handles this situation with regards to uploaded packages. It already has a decent seeming zip file checking loop in place that rejects zip files who's extracted size would be huge compared to the input as seen in pypi/warehouse#13877 - that seems to reject the example zipbombs already. @pradyunsg later confirmed that while
pipitself doesn't do such checks (it probably should) "we do benefit from PyPI's checks in the default configuration and those benefits transfer to mirrored PyPI indexes as well." - so in effect we don't currently believe PyPI can host zipbomb wheels - ameliorating this for the community'spip installinvocations by default.Workarounds & Recommendations
Document the workaround
We should centralize our guidance for Python users on how to avoid the problem when processing untrusted zip files, perhaps with a code snippet such as the overall extraction size checking loop from Warehouse as an example of the kinds of protection users of
zipfiletoday should employ. Having applications and libraries do that is good defense in depth and doesn't require a patched Python runtime.We also recommend that code handling untrusted zip files always run in a resource constrained container regardless.
Fixing
Improving our
zipfileinternals to do consistency checks to raise an exception when processing zip files with overlapping sections or inconsistent metadata or whatnot (all of the kinds of things that enable this form of malicious structured file).How feasible it'll be to backport that is unknown. But as this is only a DoS, for something that most code isn't doing (processing untrusted zips), and it can be worked around, I don't think focusing on designing this backporting this all the way back to 3.8 is worth spending extra time on if it requires any.
Reaching out to specific applications that process arbitrary zip files to make sure they're aware of this issue is a good idea.
pythongh-109858: Protect zipfile from "quoted-overlap" zipbomb
32 remaining items
serhiy-storchaka commentedon Feb 5, 2024
No. All necessary backports were merged. Thank you for the reminder.
pythongh-109858: Protect zipfile from "quoted-overlap" zipbomb (pytho…
Add @requires_zlib() decorator for pythongh-109858 tests (pythonGH-11…
sparrowt commentedon Apr 3, 2024
Does anyone here have any contacts in GitHub who know about GHSA?
Their entry for this (CVE-2024-0450) GHSA-jm46-725r-hh9v says that 3.12.2 and 3.11.8 are vulnerable, but AFAICS those 2 versions contain the fix (see
gh-109858mentioned in 3.12.2 release notes and 3.11.8 release notes).I've attempted to open an improvement PR here but I'm not sure if it will be accepted seeing as it says 'Unreviewed' and it seems python is not on the list of things they care about; because the "suggest improvements" button (via which I did it) forced me to select "Affected products" where [c]python was not an option 😢 the nearest thing was 'pip'.
AFAICS this GHSA data feeds through into tools like Grype which currently say "Fixed?"=unknown and have no "Fixed in" so it would be good to know how to correct the record on which python versions are vulnerable/fixed.
sethmlarson commentedon Apr 3, 2024
Thanks for reporting this @sparrowt, I'll take a look and update the CVE records accordingly. In theory this will trickle down into GitHub Security Advisories as well since they source a lot of their information from that database.
sparrowt commentedon Apr 3, 2024
Amazing thank you! I'm not sure how the affected versions / patched versions thing is going to work given GHSA seems to only support packages within certain 'ecosystems' but seemingly not the ecosystem runtime (e.g. python) itself:
sethmlarson commentedon Apr 3, 2024
I've fixed the metadata for the CVE. I believe the "syncing" process will happen outside of the manual update process but could be wrong? The GHSA team is aware of the whole ecosystem issue for updating GHSA records.
[3.8] pythongh-109858: Protect zipfile from "quoted-overlap" zipbomb (p…
sparrowt commentedon May 15, 2024
Thanks! Sadly it's still wrong on GHSA-jm46-725r-hh9v
[CVE-2024-0450] Protect zipfile from "quoted-overlap" zipbomb
pythongh-109858: Protect zipfile from "quoted-overlap" zipbomb (pytho…
Add @requires_zlib() decorator for pythongh-109858 tests (pythonGH-11…