Skip to content

Use The CygwinCCompiler Under Cygwin #46697

Closed

Description

@dstanek
mannequin
BPO 2445
Nosy @tarekziade, @merwok, @embray, @zooba, @ma8ma
PRs
  • bpo-2445: Allow the UnixCCompiler.find_library_file to correctly find DLL import libs (.dll.a) on Cygwin #4136
  • Files
  • cygwin-smaller.diff: Slimmed down Cygwin patch.
  • 3.4-distutils-shlibext.patch: using UnixCCompiler on Cygwin
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/tarekziade'
    closed_at = <Date 2021-02-03.18:19:07.597>
    created_at = <Date 2008-03-21.16:41:36.688>
    labels = ['build', 'library']
    title = 'Use The CygwinCCompiler Under Cygwin'
    updated_at = <Date 2021-02-03.18:19:07.597>
    user = 'https://bugs.python.org/dstanek'

    bugs.python.org fields:

    activity = <Date 2021-02-03.18:19:07.597>
    actor = 'steve.dower'
    assignee = 'tarek'
    closed = True
    closed_date = <Date 2021-02-03.18:19:07.597>
    closer = 'steve.dower'
    components = ['Distutils']
    creation = <Date 2008-03-21.16:41:36.688>
    creator = 'dstanek'
    dependencies = []
    files = ['9804', '39141']
    hgrepos = []
    issue_num = 2445
    keywords = ['patch']
    message_count = 15.0
    messages = ['64245', '64246', '64248', '67058', '73674', '73696', '73702', '73703', '73706', '101295', '194383', '222631', '241618', '273918', '386325']
    nosy_count = 10.0
    nosy_names = ['jlt63', 'janssen', 'ocean-city', 'tarek', 'jayvee', 'eric.araujo', 'rpetrov', 'erik.bray', 'steve.dower', 'masamoto']
    pr_nums = ['4136']
    priority = 'high'
    resolution = 'out of date'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'compile error'
    url = 'https://bugs.python.org/issue2445'
    versions = ['Python 3.5', 'Python 3.6']

    Activity

    1. dstanek commented on Mar 21, 2008

      dstanekmannequin
      MannequinAuthor

      I was having an issue building extension with a fresh checkout of
      revision 61699. distutils was using the UnixCCompiler. This is not able
      to find thec correct libraries to link against. I made a few changes in
      this patch:

      • distutils now uses the CygwinCCompiler when building extensions on the
        Cygwin platform
      • CygwinCCompiler.static_lib_extension needed to be .lib.a instead of
        just .a
      • Added some files to the svn:ignore property on a handful of directories.
    2. added
      stdlibStandard Library Python modules in the Lib/ directory
      buildThe build process and cross-build
      on Mar 21, 2008
    3. tiran commented on Mar 21, 2008

      @tiran
      Member

      The patch contains lots of unrelated changes. Can you please provide a
      clean patch and some doc updates? The rest looks fine to me.

    4. dstanek commented on Mar 21, 2008

      dstanekmannequin
      MannequinAuthor

      As Christian suggested I removed the unrelated svn:ignore changes.

    5. jayvee commented on May 19, 2008

      jayveemannequin
      Mannequin

      The following will also let environment variables be passed to the
      CygwinCCompiler.

      --- orig/sysconfig.py	2008-05-19 00:26:03.953125000 -0700
      +++ copy/sysconfig.py	2008-05-19 00:14:51.609375000 -0700
      @@ -145,7 +145,7 @@
           Mainly needed on Unix, so we can plug in the information that
           varies across Unices and is stored in Python's Makefile.
           """
      -    if compiler.compiler_type == "unix":
      +    if compiler.compiler_type in ("unix", "cygwin"):
               (cc, cxx, opt, cflags, ccshared, ldshared, so_ext) = \
                   get_config_vars('CC', 'CXX', 'OPT', 'CFLAGS',
                                   'CCSHARED', 'LDSHARED', 'SO')
    6. ocean-city commented on Sep 23, 2008

      ocean-citymannequin
      Mannequin

      # I found this infomation via "svn blame"

      According to bpo-403947, cygwin once used CygwinCCompiler, and problem
      was there, swiched to UnixCCompier. (r19674)

      If this issue is not solved yet, maybe it's not good to go back to
      CygwinCCompiler. (maybe)

    7. rpetrov commented on Sep 24, 2008

      rpetrovmannequin
      Mannequin

      May be check for compiler.compiler_type (from sysconfig.py ) has to be
      replaced with a check for descendant classes of UnixCCompiler, i.e. to
      include mingw32 too ?

      Also CygwinCCompiler __init__ has to be reviewed too. As example :
      -------------
      # Hard-code GCC because that's what this is all about.
      # XXX optimization, warnings etc. should be customizable.
      self.set_executables(compiler='gcc -mcygwin -O -Wall',
      .....
      -------------
      May override in unexpected way settings from customize_compiler.

      I thin that proposed modification in sysconfig.py with removing(or
      replacing) of self.set_executables from CygwinCCompiler __init__ will
      help me for bpo-3871 (cross building python for mingw32 with distutils).

      As example I will remove a hack in the setup.py:
      -----------------------

      @@ -196,8 +196,26 @@
               if compiler is not None:
                   (ccshared,cflags) =
      sysconfig.get_config_vars('CCSHARED','CFLAGS')
                   args['compiler_so'] = compiler + ' ' + ccshared + ' ' + cflags
      +
      +        # FIXME: Is next correct ?
      +        # To link modules we need LDSHARED passed to setup.py otherwise
      +        # distutils will use linker from build system if cross-compiling.
      +        linker_so = os.environ.get('LDSHARED')
      +        if linker_so is not None:
      +            args['linker_so'] = linker_so
      +
               self.compiler.set_executables(**args)

      Thanks to Hirokazu who point me, in an another thread, that cygwin build
      don't use CygwinCCompiler.

    8. rpetrov commented on Sep 24, 2008

      rpetrovmannequin
      Mannequin

      P.S. : about: static_lib_extension = ".dll.a" - it is suffix for import
      library and "unixccompiler.py.diff" patch from bpo-1706863 propose
      dylib_lib_extension = ".dll.a" .

    9. rpetrov commented on Sep 24, 2008

      rpetrovmannequin
      Mannequin

      I forgot an another issue in CygwinCCompiler __init__:
      if gcc isn't version "2.91.57" then method will set dll_libraries to
      result of get_msvcr(), but the result may be is None. In this case link
      method fail on the line libraries.extend(self.dll_libraries).

    10. jlt63 commented on Sep 24, 2008

      jlt63mannequin
      Mannequin

      I don't think the patch, cygwin-smaller.diff, is correct. The value of
      static_lib_extension needs to remain ".a". Otherwise, shared
      extensions attempting to build against a library that only offers a
      static version will fail to link.

      AFAICT, Cygwin needs a concept that is not currently supported by
      distutils -- the ability to specify a shared library *link* extension
      (i.e., ".dll.a") which is different from the shared library extension
      (i.e., ".dll"). On Unix they are the same (e.g., ".so"), but on Cygwin
      they are different because of how Windows handles DLLs -- import
      library versus DLL.

    11. self-assigned this
      on Feb 6, 2009
    12. janssen commented on Mar 19, 2010

      janssenmannequin
      Mannequin

      Shouldn't the import library name end with .lib? I've run up against this building PyLucene with MinGW. There seems to be code to figure this out already in CygwinCCompiler, but it's commented out (and arguably incomplete).

    13. rpetrov commented on Aug 4, 2013

      rpetrovmannequin
      Mannequin

      Hi,
      Now bpo-18654 "modernize mingw&cygwin compiler classes" contain enhancement that could be reused by this issue: patch "0007-MINGW-compiler-cygwin-provides-its-own-C-runtime.patch
      " from archive , i.e. lets avoid change in get_msvcr() that return. Mingw is also impacted but is addresses in another patches. As separate patch is 'compiler customization' - more advanced version then proposed by Jeevan Varshney (jayvee)

      A separate bpo-18634 "find import library" address .dll.a suffix based on existing distutil functionality.

      Also bpo-18633 "Mingw32CCompiler as default compiler for mingw* build" adds same fixes from unix compiler class into Cygwin one - ref '# Chop off the drive'

      So with above I think that cygwin compiler could be switched back from unix to own.

    14. BreamoreBoy commented on Jul 9, 2014

      BreamoreBoymannequin
      Mannequin

      From msg194383 issues bpo-18633, bpo-18634 and bpo-18654 are linked to this so is anybody up for providing a patch so that we can take this forward. See also bpo-4032.

    15. ma8ma commented on Apr 20, 2015

      ma8mamannequin
      Mannequin

      CygwinCCompiler seems legacy code.
      In v3.4 latest rivision, Distutils has made to work that is a patching to UnixCCompiler.
      So, I have a think that doesn't need to use CygwinCCompiler to build on Cygwin.
      I have upload a patch for UnixCCompiler. The patch has modified from http://cygwin.org package.

      Other big issues and patches for build on Cygwin:
      bpo-21085, bpo-13756 -- compile error
      bpo-14598, bpo-21124 -- C extension module

    16. embray commented on Aug 30, 2016

      @embray
      Contributor

      I have confirmed Masayuki's patch (3.4-distutils-shlibext.patch) fixes a few build issues on Cygwin.

      At its core, what it's fixing is allowing UnixCCompiler.find_library_file to find import libraries on Cygwin.

      That in turn is necessary for --with-system-ffi to work, which is currently needed; on Cygwin64 at least the ffi compile as part of _ctypes fails (a separate issue to be addressed separately). But in the meantime making --with-system-ffi work again goes a long way.

      It should be noted there is a patch at bpo-18654 to make the CygwinCCompiler class work again, and switch to using it to handle Cygwin-specific build issues as origially intended. I intend to review that ticket and spend some time on it, but it's more complex and not necessarily the simplest way forward.

      In the meantime this one-line patch accomplishes a fair bit.

    17. zooba commented on Feb 3, 2021

      @zooba
      Member

      Distutils is now deprecated (see PEP-632) and all tagged issues are being closed. From now until removal, only release blocking issues will be considered for distutils.

      If this issue does not relate to distutils, please remove the component and reopen it. If you believe it still requires a fix, most likely the issue should be re-reported at https://github.com/pypa/setuptools

    18. transferred this issue fromon Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

    Metadata

    Metadata

    Assignees

    Labels

    buildThe build process and cross-buildstdlibStandard Library Python modules in the Lib/ directory

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions