Repository navigation
Permit to easily use distutils "--formats=tar,gztar,bztar" on all systems #46194
Description
| BPO | 1886 |
|---|---|
| Nosy | @akuchling, @pjeby, @gustaebel, @giampaolo, @tarekziade, @merwok |
| Superseder | |
| Files |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = 'https://github.com/tarekziade'
closed_at = <Date 2010-08-09.04:21:07.567>
created_at = <Date 2008-01-21.21:28:39.040>
labels = ['type-feature', 'library']
title = 'Permit to easily use distutils "--formats=tar,gztar,bztar" on all systems'
updated_at = <Date 2010-08-09.04:21:07.524>
user = 'https://github.com/giampaolo'bugs.python.org fields:
activity = <Date 2010-08-09.04:21:07.524>
actor = 'eric.araujo'
assignee = 'tarek'
closed = True
closed_date = <Date 2010-08-09.04:21:07.567>
closer = 'eric.araujo'
components = ['Distutils']
creation = <Date 2008-01-21.21:28:39.040>
creator = 'giampaolo.rodola'
dependencies = []
files = ['9253', '9262', '9264', '12456']
hgrepos = []
issue_num = 1886
keywords = ['patch']
message_count = 13.0
messages = ['61451', '61453', '61508', '61514', '61515', '61518', '78300', '78301', '78302', '78303', '78355', '78435', '113376']
nosy_count = 10.0
nosy_names = ['akuchling', 'nnorwitz', 'pje', 'lars.gustaebel', 'techtonik', 'giampaolo.rodola', 'tarek', 'eric.araujo', 'rpetrov', 'till']
pr_nums = []
priority = 'high'
resolution = 'duplicate'
stage = 'resolved'
status = 'closed'
superseder = '6048'
type = 'enhancement'
url = 'https://bugs.python.org/issue1886'
versions = ['Python 3.2']
Activity
giampaolo commentedon Jan 21, 2008
Currently creating tar, gztar, bztar source distributions using the
"--formats=tar,gztar,bztar" distutils option requires external utilities
(tar and possibly one of gzip, bzip2, or compress) to be installed on
the system since distutils uses them.
This is not a problem on most UNIX systems which provide these utilities
by default. On all other systems (e.g. Windows) it is required to
install such utilities separately and have the executables available in
a directory on your PATH in order to have them work.
I discussed this on distutils mailing list [1] and tried to fix this
hassle.
The patch in attachment permit to use the distutils
"--formats=tar,gztar,bztar" option without need of having tar/gzip/bzip2
utilities installed by using the tarfile module instead of os.spawn().
It also fixes bug bpo-1885 [2].
[1] http://mail.python.org/pipermail/distutils-sig/2008-January/008654.html
[2] http://bugs.python.org/issue1885
pjeby commentedon Jan 21, 2008
On systems where the gzip or bz2 modules aren't installed, this patch
can raise a tarfile.CompressionError, which should be caught and handled
correctly. The import of tarfile should also be inside the relevant
function (as is done for make_zipfile) instead of at the top level.
Other than those two points, the patch looks great.
giampaolo commentedon Jan 22, 2008
Both things fixed. The new patch is in attachment.
Just one thing: I tried a diff between a tar.gz file generated by
tarfile module and another one generated by the UNIX tar utility and it
seems there are some differences.
I don't know if this could represent a problem for distributions
generated by distutils and eventually easy_install.py integration.
Maybe someone who knows the tarfile implementation could clarify this point.
gustaebel commentedon Jan 22, 2008
I just did some tests and could not find any major difference. Which are
the differences you found?
BTW, in your patch the "dist" directory is not automatically created.
giampaolo commentedon Jan 22, 2008
diff utility doesn't provide any output since the compared files are
binary. As far as I can tell the two files have a different size
(tarfile generates a little bigger file).
Moreover, by using 7zip on Windows I notice that the archive generated
by tarfile module looks different. Once I open it I see the following
structure:
dist (directory) -> archive_name.tar -> archive_name (directory) ->
other subfiles
...while the archive generated with the UNIX tar utility looks like this:
archive_name.tar -> archive_name (directory) -> other subfiles
You're right. patch2.zip file in attachment fixes that.
gustaebel commentedon Jan 22, 2008
Hm, on my Linux box both files are more or less identical. Sorry, I
cannot reproduce your problem.
techtonik commentedon Dec 26, 2008
I attach an improved patch1886.use.tarfile.module.diff that fixes all
bugs addressed in this issue.
It also removes second call to compression program leaving that
privilege to tar.
This bug is illustrated and patched (not released yet as somebody needs
to reopen it) in bpo-1885. The extra path component is taken from the
name passed to GzipFile.
techtonik commentedon Dec 26, 2008
Oh, my mistake - the bug with extra dir component in archive is in bpo-4750
Second shell call to compression program is removed only for the case
when tarfile or any of required compression modules are not available.
rpetrov commentedon Dec 26, 2008
about patch1886.use.tarfile.module.diff - the original tar don't support
-j flag.
techtonik commentedon Dec 26, 2008
What are the systems where does this original tar still exist as a
default utility?
If there is no tarfile module on this systems and "tar" version is so
old then you need a more modern system to wrap your releases or more
modern "tar". The ratio of such systems to windows systems without tar
at all is less than 1%, so it doesn't worth the hassle.
But the true reason why I won't revert the part where command line "tar"
is used, to a double call scheme, is that when you do not pipe "tar"
with compression program and use them separately with relative path -
you will get the same weird header in your tar.gz That's why it is not
uncommon on windows to meet tar.gz python packages with weird paths
inside. Luckily, windows users are infrequently install something from
sources and if they do - the .zip option is often preferred.
rpetrov commentedon Dec 27, 2008
Did I misunderstood something about -j -z -Z flags ?
http://src.opensolaris.org /source/xref/onnv/onnv-gate/usr/src/cmd/tar/tar.c
techtonik commentedon Dec 28, 2008
Roumen, could you be more specific about what are you trying to say with
this 200kB piece of code? Was it the intention to post a link to man or
another piece of spec?
merwok commentedon Aug 9, 2010
distutils in 2.7 and 3.1 use tarfile.