Skip to content

Permit to easily use distutils "--formats=tar,gztar,bztar" on all systems #46194

Closed
Closed
Permit to easily use distutils "--formats=tar,gztar,bztar" on all systems#46194
Assignees
tarekziade
Labels
stdlibStandard Library Python modules in the Lib/ directorytype-featureA feature request or enhancement

Description

@giampaolo
giampaolo
opened on Jan 21, 2008
BPO 1886
Nosy @akuchling, @pjeby, @gustaebel, @giampaolo, @tarekziade, @merwok
Superseder
  • bpo-6048: make distutils use the tarfile module
    		•
    Files
  • file.diff
  • patch.zip
  • patch2.zip
  • patch1886.use.tarfile.module.diff: patch
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/tarekziade'
closed_at = <Date 2010-08-09.04:21:07.567>
created_at = <Date 2008-01-21.21:28:39.040>
labels = ['type-feature', 'library']
title = 'Permit to easily use distutils "--formats=tar,gztar,bztar" on all systems'
updated_at = <Date 2010-08-09.04:21:07.524>
user = 'https://github.com/giampaolo'

    bugs.python.org fields:

    activity = <Date 2010-08-09.04:21:07.524>
actor = 'eric.araujo'
assignee = 'tarek'
closed = True
closed_date = <Date 2010-08-09.04:21:07.567>
closer = 'eric.araujo'
components = ['Distutils']
creation = <Date 2008-01-21.21:28:39.040>
creator = 'giampaolo.rodola'
dependencies = []
files = ['9253', '9262', '9264', '12456']
hgrepos = []
issue_num = 1886
keywords = ['patch']
message_count = 13.0
messages = ['61451', '61453', '61508', '61514', '61515', '61518', '78300', '78301', '78302', '78303', '78355', '78435', '113376']
nosy_count = 10.0
nosy_names = ['akuchling', 'nnorwitz', 'pje', 'lars.gustaebel', 'techtonik', 'giampaolo.rodola', 'tarek', 'eric.araujo', 'rpetrov', 'till']
pr_nums = []
priority = 'high'
resolution = 'duplicate'
stage = 'resolved'
status = 'closed'
superseder = '6048'
type = 'enhancement'
url = 'https://bugs.python.org/issue1886'
versions = ['Python 3.2']

    Activity

    1. giampaolo commented on Jan 21, 2008

      @giampaolo
      giampaolo
      on Jan 21, 2008
      ContributorAuthor

      Currently creating tar, gztar, bztar source distributions using the
      "--formats=tar,gztar,bztar" distutils option requires external utilities
      (tar and possibly one of gzip, bzip2, or compress) to be installed on
      the system since distutils uses them.
      This is not a problem on most UNIX systems which provide these utilities
      by default. On all other systems (e.g. Windows) it is required to
      install such utilities separately and have the executables available in
      a directory on your PATH in order to have them work.

      I discussed this on distutils mailing list [1] and tried to fix this
      hassle.
      The patch in attachment permit to use the distutils
      "--formats=tar,gztar,bztar" option without need of having tar/gzip/bzip2
      utilities installed by using the tarfile module instead of os.spawn().
      It also fixes bug bpo-1885 [2].

      [1] http://mail.python.org/pipermail/distutils-sig/2008-January/008654.html
      [2] http://bugs.python.org/issue1885

    2. giampaolo
      added
      stdlibStandard Library Python modules in the Lib/ directory
      on Jan 21, 2008

    3. pjeby commented on Jan 21, 2008

      @pjeby
      pjebymannequin
      on Jan 21, 2008
      Mannequin

      On systems where the gzip or bz2 modules aren't installed, this patch
      can raise a tarfile.CompressionError, which should be caught and handled
      correctly. The import of tarfile should also be inside the relevant
      function (as is done for make_zipfile) instead of at the top level.

      Other than those two points, the patch looks great.

    4. tiran
      added
      type-featureA feature request or enhancement
      on Jan 22, 2008

    5. giampaolo commented on Jan 22, 2008

      @giampaolo
      giampaolo
      on Jan 22, 2008
      ContributorAuthor

      Both things fixed. The new patch is in attachment.

      Just one thing: I tried a diff between a tar.gz file generated by
      tarfile module and another one generated by the UNIX tar utility and it
      seems there are some differences.
      I don't know if this could represent a problem for distributions
      generated by distutils and eventually easy_install.py integration.
      Maybe someone who knows the tarfile implementation could clarify this point.

    6. gustaebel commented on Jan 22, 2008

      @gustaebel
      gustaebelmannequin
      on Jan 22, 2008
      Mannequin

      I just did some tests and could not find any major difference. Which are
      the differences you found?

      BTW, in your patch the "dist" directory is not automatically created.

    7. giampaolo commented on Jan 22, 2008

      @giampaolo
      giampaolo
      on Jan 22, 2008
      ContributorAuthor

      I just did some tests and could not find any major difference.
      Which are the differences you found?

      diff utility doesn't provide any output since the compared files are
      binary. As far as I can tell the two files have a different size
      (tarfile generates a little bigger file).
      Moreover, by using 7zip on Windows I notice that the archive generated
      by tarfile module looks different. Once I open it I see the following
      structure:

      dist (directory) -> archive_name.tar -> archive_name (directory) ->
      other subfiles

      ...while the archive generated with the UNIX tar utility looks like this:

      archive_name.tar -> archive_name (directory) -> other subfiles

      BTW, in your patch the "dist" directory is not automatically created.

      You're right. patch2.zip file in attachment fixes that.

    8. gustaebel commented on Jan 22, 2008

      @gustaebel
      gustaebelmannequin
      on Jan 22, 2008
      Mannequin

      Hm, on my Linux box both files are more or less identical. Sorry, I
      cannot reproduce your problem.

    9. akuchling
      self-assigned this
      on Oct 11, 2008

    10. techtonik commented on Dec 26, 2008

      @techtonik
      techtonikmannequin
      on Dec 26, 2008
      Mannequin

      I attach an improved patch1886.use.tarfile.module.diff that fixes all
      bugs addressed in this issue.

      It also removes second call to compression program leaving that
      privilege to tar.

      I notice that the archive generated by tarfile module looks
      different. Once I open it I see the following structure:

      dist (directory) -> archive_name.tar -> archive_name (directory)
      -> other subfiles

      This bug is illustrated and patched (not released yet as somebody needs
      to reopen it) in bpo-1885. The extra path component is taken from the
      name passed to GzipFile.

    11. techtonik commented on Dec 26, 2008

      @techtonik
      techtonikmannequin
      on Dec 26, 2008
      Mannequin

      Oh, my mistake - the bug with extra dir component in archive is in bpo-4750

      Second shell call to compression program is removed only for the case
      when tarfile or any of required compression modules are not available.

    12. rpetrov commented on Dec 26, 2008

      @rpetrov
      rpetrovmannequin
      on Dec 26, 2008
      Mannequin

      about patch1886.use.tarfile.module.diff - the original tar don't support
      -j flag.

    13. techtonik commented on Dec 26, 2008

      @techtonik
      techtonikmannequin
      on Dec 26, 2008
      Mannequin

      What are the systems where does this original tar still exist as a
      default utility?

      If there is no tarfile module on this systems and "tar" version is so
      old then you need a more modern system to wrap your releases or more
      modern "tar". The ratio of such systems to windows systems without tar
      at all is less than 1%, so it doesn't worth the hassle.

      But the true reason why I won't revert the part where command line "tar"
      is used, to a double call scheme, is that when you do not pipe "tar"
      with compression program and use them separately with relative path -
      you will get the same weird header in your tar.gz That's why it is not
      uncommon on windows to meet tar.gz python packages with weird paths
      inside. Luckily, windows users are infrequently install something from
      sources and if they do - the .zip option is often preferred.

    14. rpetrov commented on Dec 27, 2008

      @rpetrov
      rpetrovmannequin
      on Dec 27, 2008
      Mannequin

      Did I misunderstood something about -j -z -Z flags ?
      http://src.opensolaris.org /source/xref/onnv/onnv-gate/usr/src/cmd/tar/tar.c

    15. techtonik commented on Dec 28, 2008

      @techtonik
      techtonikmannequin
      on Dec 28, 2008
      Mannequin

      Roumen, could you be more specific about what are you trying to say with
      this 200kB piece of code? Was it the intention to post a link to man or
      another piece of spec?

    16. akuchling
      removed their assignment
      on Jul 26, 2010
    17. merwok
      assigned
      tarekziade
      on Jul 27, 2010

    18. merwok commented on Aug 9, 2010

      @merwok
      merwok
      on Aug 9, 2010
      Member

      distutils in 2.7 and 3.1 use tarfile.

    19. merwok
      closed this as completedon Aug 9, 2010
    20. ezio-melotti
      transferred this issue fromon Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

    Metadata

    Metadata

    Assignees

    Labels

    stdlibStandard Library Python modules in the Lib/ directorytype-featureA feature request or enhancement

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions