Skip to content

Smelly exports (global symbols in python not prefixed with Py or _Py) #48805

Closed
Closed
Smelly exports (global symbols in python not prefixed with Py or _Py)#48805
Assignees
davidmalcolm
Labels
interpreter-core(Objects, Python, Grammar, and Parser dirs)performancePerformance or resource usage

Description

@tiran
tiran
opened on Dec 6, 2008
BPO 4555
Nosy @malemburg, @loewis, @doko42, @pitrou, @tiran, @davidmalcolm
Files
  • fix-make-smelly-on-trunk.patch
  • py3k-add-_Py-prefix-to-avoid-smelliness.patch
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/davidmalcolm'
closed_at = <Date 2013-10-12.21:08:22.424>
created_at = <Date 2008-12-06.01:29:10.017>
labels = ['interpreter-core', 'performance']
title = 'Smelly exports (global symbols in python not prefixed with Py or _Py)'
updated_at = <Date 2013-10-12.21:08:22.423>
user = 'https://github.com/tiran'

    bugs.python.org fields:

    activity = <Date 2013-10-12.21:08:22.423>
actor = 'pitrou'
assignee = 'dmalcolm'
closed = True
closed_date = <Date 2013-10-12.21:08:22.424>
closer = 'pitrou'
components = ['Interpreter Core']
creation = <Date 2008-12-06.01:29:10.017>
creator = 'christian.heimes'
dependencies = []
files = ['15861', '18094']
hgrepos = []
issue_num = 4555
keywords = ['patch']
message_count = 15.0
messages = ['77090', '77091', '77096', '77292', '77350', '77517', '84408', '93793', '97737', '110934', '110938', '176418', '176486', '177551', '199622']
nosy_count = 7.0
nosy_names = ['lemburg', 'loewis', 'doko', 'pitrou', 'christian.heimes', 'dmalcolm', 'python-dev']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'resource usage'
url = 'https://bugs.python.org/issue4555'
versions = ['Python 2.7', 'Python 3.3', 'Python 3.4']

    Activity

    1. tiran commented on Dec 6, 2008

      @tiran
      tiran
      on Dec 6, 2008
      MemberAuthor

      I just found about the smelly build target. It checks for smelly exports.

      $ make smelly
[...]
nm -p libpython2.6.a | \
                sed -n "/ [TDB] /s/.* //p" | grep -v "^_*Py" | sort -u; \

      asdl_int_seq_new
      asdl_seq_new
      init_ast
      init_codecs
      initerrno
      initgc
      initimp
      initposix
      initpwd
      initsignal
      init_sre
      init_symtable
      initthread
      initxxsubtype
      initzipimport

      nm -p libpython3.0.a | \
      sed -n "/ [TDB] /s/.* //p" | grep -v "^_*Py" | sort -u; \

      _add_one_to_index_C
      _add_one_to_index_F
      asdl_int_seq_new
      asdl_seq_new

      These should be checked and fixed.

    2. tiran
      added
      interpreter-core(Objects, Python, Grammar, and Parser dirs)
      performancePerformance or resource usage
      on Dec 6, 2008

    3. loewis commented on Dec 6, 2008

      @loewis
      loewismannequin
      on Dec 6, 2008
      Mannequin

      Are you suggesting that the init* functions in Python 2.6 should not
      longer be exported? Why? And how?

    4. tiran commented on Dec 6, 2008

      @tiran
      tiran
      on Dec 6, 2008
      MemberAuthor

      I just suggest that somebody should look into it. I neither know how
      critical the issue is nor how to change them. This bug report serves as
      a reminder.

    5. malemburg commented on Dec 8, 2008

      @malemburg
      malemburg
      on Dec 8, 2008
      Member

      For Python 2.x you cannot change the "main" function name of C
      extensions, since this is used by the Python import mechanism.

    6. loewis commented on Dec 8, 2008

      @loewis
      loewismannequin
      on Dec 8, 2008
      Mannequin

      I guess you *could* change the name of extensions that become builtin
      modules. However, I think it is not worth the trouble, especially since
      PEP-3121 solves the problem for good, for Python 3.

      I propose to simply filter out init[_a-z]+ from the set of "bad" symbols.

      That leaves us with asdl_*seq_new, and _add_one_to_index; the latter
      is already reported as bpo-3101.

    7. loewis commented on Dec 10, 2008

      @loewis
      loewismannequin
      on Dec 10, 2008
      Mannequin

      There is no proposed patch yet, so removing 2.5.3 from the target versions.

    8. pitrou commented on Mar 29, 2009

      @pitrou
      pitrou
      on Mar 29, 2009
      Member

      bpo-5591 is a duplicate of this.

    9. pitrou commented on Oct 9, 2009

      @pitrou
      pitrou
      on Oct 9, 2009
      Member

      In trunk:

      _add_one_to_index_C
      _add_one_to_index_F
      asdl_int_seq_new
      asdl_seq_new
      init_ast
      init_codecs
      initerrno
      initgc
      initimp
      initposix
      initpwd
      initsignal
      init_sre
      init_symtable
      initthread
      initxxsubtype
      initzipimport

      In py3k:

      _add_one_to_index_C
      _add_one_to_index_F
      asdl_int_seq_new
      asdl_seq_new

    10. davidmalcolm commented on Jan 13, 2010

      @davidmalcolm
      davidmalcolm
      on Jan 13, 2010
      Member

      Re: msg bpo-77350:

      I propose to simply filter out init[_a-z]+ from the set of "bad"
      symbols.

      I'm attaching a patch (to trunk) to Makefile.pre.in which filters out such symbols.

      Relevant part of output of "make smelly" on my svn build with this patch:
      nm -p libpython2.7.a | \
      sed -n "/ [TDB] /s/.* //p" | grep -v "^_*Py" | \
      grep -v -E "^init[_a-z]+" | sort -u; \

      _add_one_to_index_C
      _add_one_to_index_F
      asdl_int_seq_new
      asdl_seq_new

    11. davidmalcolm commented on Jul 20, 2010

      @davidmalcolm
      davidmalcolm
      on Jul 20, 2010
      Member

      py3k is much cleaner than python 2, due to the change in the module API.

      Here's "make smelly"'s output on a recent py3k checkout:
      nm -p libpython3.2.a | \
      sed -n "/ [TDB] /s/.* //p" | grep -v "^_*Py" | sort -u; \

      _add_one_to_index_C
      _add_one_to_index_F
      asdl_int_seq_new
      asdl_seq_new

    12. abalkin
      changed the title [-]Smelly exports[/-] [+]Smelly exports (global symbols in python not prefixed with Py or _Py)[/+] on Jul 20, 2010

    13. davidmalcolm commented on Jul 20, 2010

      @davidmalcolm
      davidmalcolm
      on Jul 20, 2010
      Member

      Patch to py3k which adds the "_Py" prefix to the four listed symbols.

      With this, the output from "make smelly" is clean (odorless, perhaps?).

      However, adding _Py does seem to go against this comment in Include/asdl.h:
      /* It would be nice if the code generated by asdl_c.py was completely
      independent of Python, but it is a goal the requires too much work
      at this stage. So, for example, I'll represent identifiers as
      interned Python strings.
      */

    14. tiran commented on Nov 26, 2012

      @tiran
      tiran
      on Nov 26, 2012
      MemberAuthor

      For Python 3.3 and 3.4 make smelly still lists two exported symbols.

      asdl_int_seq_new
      asdl_seq_new

      Do we have to check the modules, too? Some of them like _ctypes and _decimal export a lot of symbols. Here is a small shell snippet for Makefile.

      for MOD in find $(srcdir)/build -name '*.s[ol]' ; do
      EXPORT=nm -p $$MOD | sed -n "/ [TDB] /s/.* //p" | \ grep -E -v "^_*Py|^_init|^_fini" | sort -u;
      if [ -n "$$EXPORT" ]; then
      echo $$MOD;
      echo $$EXPORT;
      fi
      done

    15. loewis commented on Nov 27, 2012

      @loewis
      loewismannequin
      on Nov 27, 2012
      Mannequin

      I'd like to focus this issue; it has been open long enough, and deserves to get closed once the original issue is resolved - which was that "make smelly" reports symbols. I think dmalcolm's patch is quite a good start for that.

      It may well be that other modules need to be considered - but PLEASE not in this issue. For Unix with shared libraries, the extension modules certainly cause problems, in particular if Python gets embedded in a context (e.g. Apache) that also loads separate copies of the same libraries, and may cause problems if one of our functions collide with some library. However, I see *two* issues falling out of this: one to extend the "make smelly" target to include extension modules, and the other to then fix the extension modules - preferably with one issue per extension module.

    16. pitrou commented on Dec 15, 2012

      @pitrou
      pitrou
      on Dec 15, 2012
      Member

      This very simple patch should certainly be applied.

    17. pitrou
      assigned
      davidmalcolm
      on Dec 15, 2012

    18. python-dev commented on Oct 12, 2013

      @python-dev
      python-devmannequin
      on Oct 12, 2013
      Mannequin

      New changeset 142c62a490ce by Antoine Pitrou in branch 'default':
      Issue bpo-4555: All exported C symbols are now prefixed with either "Py" or "_Py".
      http://hg.python.org/cpython/rev/142c62a490ce

    19. pitrou
      closed this as completedon Oct 12, 2013
    20. ezio-melotti
      transferred this issue fromon Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

    Metadata

    Metadata

    Assignees

    Labels

    interpreter-core(Objects, Python, Grammar, and Parser dirs)performancePerformance or resource usage

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions