Repository navigation
buffer overflow in Zipfile when wrinting more than 2gig file #50683
Description
| BPO | 6434 |
|---|---|
| Nosy | @amauryfa |
| Superseder | |
| Files |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = None
closed_at = <Date 2011-11-03.12:17:17.767>
created_at = <Date 2009-07-07.17:10:57.536>
labels = ['type-bug', 'library']
title = 'buffer overflow in Zipfile when wrinting more than 2gig file'
updated_at = <Date 2011-11-03.12:17:17.766>
user = 'https://bugs.python.org/segfault42'bugs.python.org fields:
activity = <Date 2011-11-03.12:17:17.766>
actor = 'nadeem.vawda'
assignee = 'none'
closed = True
closed_date = <Date 2011-11-03.12:17:17.767>
closer = 'nadeem.vawda'
components = ['Library (Lib)']
creation = <Date 2009-07-07.17:10:57.536>
creator = 'segfault42'
dependencies = []
files = ['14469']
hgrepos = []
issue_num = 6434
keywords = []
message_count = 11.0
messages = ['90242', '90243', '90261', '90282', '92142', '92147', '121826', '146505', '146527', '146911', '146922']
nosy_count = 6.0
nosy_names = ['amaury.forgeotdarc', 'nadeem.vawda', 'lambacck', 'segfault42', 'enlavin', 'Paul']
pr_nums = []
priority = 'normal'
resolution = 'duplicate'
stage = 'resolved'
status = 'closed'
superseder = '9720'
type = 'behavior'
url = 'https://bugs.python.org/issue6434'
versions = ['Python 2.7', 'Python 3.2', 'Python 3.3']
Activity
segfault42 commentedon Jul 7, 2009
Hello,
I have a problem with the librairy zipfile.py
http://svn.python.org/view/python/trunk/Lib/zipfile.py?revision=73565&view=markup
Zinfo structure limit the size of a file to an int max value with the
ZIP64_LIMIT value ( equal to "(1 << 31) - 1 " so to 2147483647 .
The problem is happening when you write a big file in the line 1095 :
zinfo.file_size is limited to a int size and if you have a file bigger
than ZIP64_LIMIT you make a buffer overflow even if you set the flag
allowZip64 to true.
segfault42 commentedon Jul 7, 2009
look like bpo-1182788
amauryfa commentedon Jul 8, 2009
I don't see how it can be a buffer overflow. Or is it an exception
raised by the struct.pack function?
segfault42 commentedon Jul 8, 2009
yes it's zinfo.file_size which is bigger than the long specify in the
struct.pack
There's must have a solution with the extra header because a lot of tools
can zip big file and these zip file can be open by zipfile.py
it's easy to reproduice with a big file of 3 gig.
i think that the problem come from that the write methode do not take
care of the flag allowZip64
segfault42 commentedon Sep 1, 2009
still no one to help on this problem ? is someone has some idea ?
amauryfa commentedon Sep 1, 2009
I did reproduce the problem, but I'm sorry I don't have the time to fix
it. However, I will review any proposed patch.
lambacck commentedon Nov 21, 2010
This should be closed as a dup of bpo-1182788 which the OP identified as being the same bug and which is now fixed due to the implementation. of ZIP64.
Paul commentedon Oct 27, 2011
This is a problem with python2.7 as well. A change in struct between python2.6 and 2.7 raises an exception on overflow instead of silently allowing it. This prevents zipping any file larger than 4.5G. This exception concurs when writing the 32-bit headers (which are not used on large files anyway)
The patch should be simple. Just wrap line 1100:
...struct.pack("<LLL",...
with a try: except: to revert to the old behavior. Alternatively, check if size is bigger than ZIP64_LIMIT and set to anything less than ZIP64_LIMIT.
Paul commentedon Oct 27, 2011
I attempted to "re-allow overflow" in the struct(...) call by replacing
zinfo.file_sizewithZIP64_LIMIT % zinfo.file_sizein zipfile.py, and successfully produced a compressed file from a 10G file, but the resulting compressed file could not be uncompressed and was deemed "invalid" by any unzip util I tried.enlavin commentedon Nov 3, 2011
I think it's a dup of http://bugs.python.org/issue9720
That issue has a patch that allows the generation of zip files with >2GB files.
nadeemvawda commentedon Nov 3, 2011
Marking as duplicate.