Skip to content

Use -Wl,--dynamic-list=x.list, not -Xlinker -export-dynamic #54321

Closed as not planned
Closed as not planned
Use -Wl,--dynamic-list=x.list, not -Xlinker -export-dynamic#54321
Labels
buildThe build process and cross-buildpendingThe issue will be closed if no feedback is providedperformancePerformance or resource usage

Description

@jankratochvil
jankratochvil
mannequin
opened on Oct 15, 2010
BPO 10112
Nosy @loewis, @pitrou, @vstinner, @merwok, @davidmalcolm, @meadori
Files
  • hidden4.patch: Fixed patch
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = None
created_at = <Date 2010-10-15.10:16:24.300>
labels = ['build', 'performance']
title = 'Use -Wl,--dynamic-list=x.list, not -Xlinker -export-dynamic'
updated_at = <Date 2019-01-23.15:47:02.674>
user = 'https://bugs.python.org/jankratochvil'

    bugs.python.org fields:

    activity = <Date 2019-01-23.15:47:02.674>
actor = 'vstinner'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Build']
creation = <Date 2010-10-15.10:16:24.300>
creator = 'jankratochvil'
dependencies = []
files = ['24575']
hgrepos = []
issue_num = 10112
keywords = ['patch']
message_count = 7.0
messages = ['118756', '151146', '153394', '153459', '153764', '153810', '237136']
nosy_count = 9.0
nosy_names = ['loewis', 'pitrou', 'vstinner', 'eric.araujo', 'Arfrever', 'dmalcolm', 'meador.inge', 'jankratochvil', 'tromey']
pr_nums = []
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'resource usage'
url = 'https://bugs.python.org/issue10112'
versions = ['Python 3.3']

    Activity

    1. jankratochvil commented on Oct 15, 2010

      @jankratochvil
      jankratochvilmannequin
      on Oct 15, 2010
      MannequinAuthor

      FSF GDB (and future Fedora GDBs) became 250KB smaller than before. Python recently disabled this GDB build optimization so GDB is 250KB larger again.

      -rwxr-xr-x 1 4524488 gdb-7.1-19.fc13.x86_64/usr/bin/gdb
      -rwxr-xr-x 1 4266728 gdb-7.1-19dynamiclist.fc13.x86_64/usr/bin/gdb
      -rw-r--r-- 1 2364656 gdb-7.1-19.fc13.x86_64.rpm
      -rw-r--r-- 1 2274300 gdb-7.1-19dynamiclist.fc13.x86_64.rpm

      Some Python binaries/libraries could probably also benefit from smaller pageable code image.

      GDB regressed due to /usr/lib*/python*/config/Makefile contains:
      LINKFORSHARED=-Xlinker -export-dynamic

      and GDB thus has to use it even for its own link, effectively disabling its:
      -Wl,--dynamic-list=./proc-service.list

      AFAIK Python already contains the required exports list but it is used only on MS-Windows. It should be utilized even for GNU/Linux builds.

    2. jankratochvil
      added
      stdlibStandard Library Python modules in the Lib/ directory
      performancePerformance or resource usage
      on Oct 15, 2010

    3. jankratochvil commented on Jan 12, 2012

      @jankratochvil
      jankratochvilmannequin
      on Jan 12, 2012
      MannequinAuthor

      Here is the implementation.

      Python/getargs.c was modified for:
      ImportError: /usr/lib64/python2.7/lib-dynload/fcntlmodule.so: undefined symbol: _PyArg_ParseTuple_SizeT
      but I guess that patch part should be different. There is no need to do #ifdef HAVE_DECLSPEC_DLL as PyAPI_FUNC is properly defined even without it. So the declaration+export block should be unified there.

      The Modules/ files were modified for:
      *** WARNING: renaming "_ctypes_test" since importing it failed: dynamic module does not define init function (init_ctypes_test)
      *** WARNING: renaming "_elementtree" since importing it failed: dynamic module does not define init function (init_elementtree)
      *** WARNING: renaming "_hotshot" since importing it failed: dynamic module does not define init function (init_hotshot)
      *** WARNING: renaming "_json" since importing it failed: dynamic module does not define init function (init_json)
      *** WARNING: renaming "ossaudiodev" since importing it failed: dynamic module does not define init function (initossaudiodev)

      without this patch:
-rw-r--r-- 1 jkratoch jkratoch 5775829 Jan 12 17:42 python-libs-2.7.2-18.fc17.x86_64.rpm
-r-xr-xr-x 1 jkratoch jkratoch 1738264 Jan 12 17:44 ./clean/python-libs-2.7.2-18.fc17.x86_64/usr/lib64/libpython2.7.so.1.0*
1226 libpython2.7.so.1.0 .dynsym exports
+
-rw-r--r-- 1 jkratoch jkratoch 1986161 Jan 12 17:48 gdb-7.4.50.20120103-8.fc17.x86_64.rpm
-rwxr-xr-x 1 jkratoch jkratoch 5018800 Jan 12 19:13 clean/gdb-7.4.50.20120103-8.fc17.x86_64/usr/bin/gdb*

with this patch:
-rw-r--r-- 1 jkratoch jkratoch 5762537 Jan 12 19:04 python-libs-2.7.2-18hidden3.fc17.x86_64.rpm
-r-xr-xr-x 1 jkratoch jkratoch 1720920 Jan 12 19:06 ./hidden3/python-libs-2.7.2-18hidden3.fc17.x86_64/usr/lib64/libpython2.7.so.1.0*
1046 libpython2.7.so.1.0 .dynsym exports
+
-rw-r--r-- 1 jkratoch jkratoch 1886781 Jan 12 19:11 gdb-7.4.50.20120103-8.fc17.x86_64.rpm
-rwxr-xr-x 1 jkratoch jkratoch 4732080 Jan 12 19:13 hidden3/gdb-7.4.50.20120103-8.fc17.x86_64/usr/bin/gdb*

    4. jankratochvil commented on Feb 15, 2012

      @jankratochvil
      jankratochvilmannequin
      on Feb 15, 2012
      MannequinAuthor

      What more can be done to get it fixed? I do not mind Python but GDB linking is broken due to it. And while I can workaround it in GDB I am not used for workarounding one Free package in another Free package. Free software has the advantage problems can be fixed at the right place.

    5. meadori commented on Feb 16, 2012

      @meadori
      meadori
      on Feb 16, 2012
      Member

      When I try to build with the attached patch on the 2.7 branch many (if not all) of the modules fail to build:

      Failed to build these modules:
      _bisect _bsddb _codecs_cn
      _codecs_hk _codecs_iso2022 _codecs_jp
      _codecs_kr _codecs_tw _collections
      _csv _ctypes _ctypes_test
      _curses _curses_panel _elementtree
      _functools _hashlib _heapq
      _hotshot _io _json
      _locale _lsprof _multibytecodec
      _multiprocessing _random _socket
      _sqlite3 _ssl _struct
      _testcapi _tkinter array
      audioop binascii bz2
      cmath cPickle crypt
      cStringIO datetime dbm
      fcntl future_builtins gdbm
      grp itertools linuxaudiodev
      math mmap nis
      operator ossaudiodev parser
      pyexpat readline resource
      select spwd strop
      syslog termios time
      unicodedata zlib

      I see several warnings in the build output:

      *** WARNING: renaming "_codecs_iso2022" since importing it failed: build/lib.lin
      ux-x86_64-2.7/_codecs_iso2022.so: undefined symbol: PyExc_ValueError
      ..
      *** WARNING: renaming "linuxaudiodev" since importing it failed: build/lib.linux
      -x86_64-2.7/linuxaudiodev.so: undefined symbol: Py_Py3kWarningFlag
      ...
      *** WARNING: renaming "_tkinter" since importing it failed: build/lib.linux-x86_64-2.7/_tkinter.so: undefined symbol: _Py_ZeroStruct

    6. meadori
      added
      buildThe build process and cross-build
      and removed
      stdlibStandard Library Python modules in the Lib/ directory
      on Feb 16, 2012

    7. jankratochvil commented on Feb 20, 2012

      @jankratochvil
      jankratochvilmannequin
      on Feb 20, 2012
      MannequinAuthor

      There was a bug for non-shared build of Python, thanks.

      This patch is an improvement on its but it is not yet complete.
      One should also replace "-Xlinker -export-dynamic" by "-Wl,--dynamic-list" so that linking application with libpython is no longer intrusive.

      But I found out GDB does not need to be affected by this Bug. When I use -fvisibility=hidden for GDB .c files then "-Xlinker -export-dynamic" has no longer any negative effect (GDB was using only "-Wl,--dynamic-list" now which works without libpython but not with libpython). Therefore I no longer mind much about this Bug.

      Still at least:
      1771024 libpython2.7.so.1.0
      ->
      1757776 libpython2.7.so.1.0

      Even with clean tree the upstream build generates for me:
      Python build finished, but the necessary bits to build these modules were not found:
      _bsddb _ssl bsddb185
      dl imageop sunaudiodev

      (so sunaudiodev.c was changed without any testing of it)

    8. loewis commented on Feb 20, 2012

      @loewis
      loewismannequin
      on Feb 20, 2012
      Mannequin

      The patch is fairly incomplete: it assumes that the compiler supports __attribute__((visibility())), but it really needs to check for that.

      Also, there is no chance that this can go into Python 2.7; retargetting for 3.3.

    9. jankratochvil commented on Mar 3, 2015

      @jankratochvil
      jankratochvilmannequin
      on Mar 3, 2015
      MannequinAuthor

      It even crashes applications due to pollution of dynamic symbols namespace by application symbols as seen in:
      https://bugzilla.redhat.com/show_bug.cgi?id=1198158

    10. ezio-melotti
      transferred this issue fromon Apr 10, 2022

    11. iritkatriel commented on Aug 18, 2022

      @iritkatriel
      iritkatriel
      on Aug 18, 2022
      Member

      @jankratochvil Is this issue still relevant?

    12. iritkatriel
      added
      pendingThe issue will be closed if no feedback is provided
      on Aug 18, 2022

    13. jankratochvil commented on Aug 18, 2022

      @jankratochvil
      jankratochvil
      on Aug 18, 2022
      Author

      I am no longer involved with GDB. But I also think it has been fixed for python2->3:

      $ /usr/bin/python2-config --ldflags
-lpython2.7 -ldl -lm -Xlinker -export-dynamic
$ /usr/bin/python3-config --ldflags
 -L/usr/lib64  -lcrypt -ldl  -lm -lm

    14. iritkatriel commented on Aug 18, 2022

      @iritkatriel
      iritkatriel
      on Aug 18, 2022
      Member

      Great, thank you.

    15. iritkatriel
      closed this as not plannedon Aug 18, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

    Metadata

    Metadata

    Assignees

    No one assigned

      Labels

      buildThe build process and cross-buildpendingThe issue will be closed if no feedback is providedperformancePerformance or resource usage

      Projects

      No projects

        Milestone

        No milestone

        Relationships

        None yet

        Development

        No branches or pull requests

        Issue actions