Skip to content

[C API] Clean-up the Python.h header file #89597

Description

@vstinner
BPO 45434
Nosy @Yhg1s, @vstinner, @corona10
PRs
  • bpo-45434: Clean-up Python.h header file #28883
  • bpo-45434: Python.h no longer includes <stdlib.h> #28888
  • bpo-45434: Convert Py_GETENV() macro to a function #28912
  • bpo-45434: bytearrayobject.h no longer includes <stdarg.h> #28913
  • bpo-45434: pyport.h no longer includes <stdlib.h> #28914
  • bpo-45434: Move _Py_BEGIN_SUPPRESS_IPH to pycore_fileutils.h #28922
  • bpo-45434: Remove pystrhex.h header file #28923
  • bpo-45434: Mark the PyTokenizer C API as private #28924
  • bpo-45434: Limited Python.h no longer includes stdio.h #28960
  • bpo-45434: Remove useless space in includes #28963
  • bpo-45434: Remove Include/eval.h header file #28973
  • bpo-45434: Include stdlib.h for specialize stat #29015
  • bpo-45434: Only exclude <stdlib.h> in Python 3.11 limited C API #29027
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2021-10-19.15:01:04.891>
    created_at = <Date 2021-10-11.20:08:05.746>
    labels = ['expert-C-API', '3.11']
    title = '[C API] Clean-up the Python.h header file'
    updated_at = <Date 2021-10-19.15:01:04.890>
    user = 'https://github.com/vstinner'

    bugs.python.org fields:

    activity = <Date 2021-10-19.15:01:04.890>
    actor = 'vstinner'
    assignee = 'none'
    closed = True
    closed_date = <Date 2021-10-19.15:01:04.891>
    closer = 'vstinner'
    components = ['C API']
    creation = <Date 2021-10-11.20:08:05.746>
    creator = 'vstinner'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 45434
    keywords = ['patch']
    message_count = 19.0
    messages = ['403675', '403684', '403689', '403775', '403785', '403829', '403831', '403841', '403842', '403843', '403859', '403940', '403959', '403963', '404002', '404162', '404170', '404172', '404283']
    nosy_count = 3.0
    nosy_names = ['twouters', 'vstinner', 'corona10']
    pr_nums = ['28883', '28888', '28912', '28913', '28914', '28922', '28923', '28924', '28960', '28963', '28973', '29015', '29027']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = None
    url = 'https://bugs.python.org/issue45434'
    versions = ['Python 3.11']

    Activity

    1. vstinner commented on Oct 11, 2021

      @vstinner
      MemberAuthor

      I would like to remove #include <stdlib.h> from Python.h, and make Python.h smaller.

    2. vstinner commented on Oct 11, 2021

      @vstinner
      MemberAuthor

      New changeset 47717d1 by Victor Stinner in branch 'main':
      bpo-45434: Cleanup Python.h header file (GH-28883)
      47717d1

    3. vstinner commented on Oct 11, 2021

      @vstinner
      MemberAuthor

      New changeset 03ea862 by Victor Stinner in branch 'main':
      bpo-45434: Python.h no longer includes <stdlib.h> (GH-28888)
      03ea862

    4. vstinner commented on Oct 13, 2021

      @vstinner
      MemberAuthor

      New changeset 489176e by Victor Stinner in branch 'main':
      bpo-45434: Convert Py_GETENV() macro to a function (GH-28912)
      489176e

    5. vstinner commented on Oct 13, 2021

      @vstinner
      MemberAuthor

      New changeset c63623a by Victor Stinner in branch 'main':
      bpo-45434: bytearrayobject.h no longer includes <stdarg.h> (GH-28913)
      c63623a

    6. vstinner commented on Oct 13, 2021

      @vstinner
      MemberAuthor

      New changeset 97308df by Victor Stinner in branch 'main':
      bpo-45434: Move _Py_BEGIN_SUPPRESS_IPH to pycore_fileutils.h (GH-28922)
      97308df

    7. vstinner commented on Oct 13, 2021

      @vstinner
      MemberAuthor

      New changeset bbe7497 by Victor Stinner in branch 'main':
      bpo-45434: Remove pystrhex.h header file (GH-28923)
      bbe7497

    8. vstinner commented on Oct 13, 2021

      @vstinner
      MemberAuthor

      Oh, Include/errcode.h *is* used by PyRun_InteractiveOneFlags() and its variants. It's mentioned in its documentation:

      Returns ``0`` when the input was
      executed successfully, ``-1`` if there was an exception, or an error code
      from the :file:`errcode.h` include file distributed as part of Python if
      there was a parse error. (Note that :file:`errcode.h` is not included by
      :file:`Python.h`, so must be included specifically if needed.)

    9. vstinner commented on Oct 13, 2021

      @vstinner
      MemberAuthor

      New changeset 713bb19 by Victor Stinner in branch 'main':
      bpo-45434: Mark the PyTokenizer C API as private (GH-28924)
      713bb19

    10. vstinner commented on Oct 13, 2021

      @vstinner
      MemberAuthor

      PyRun_InteractiveOneObjectEx() returns a single error code of errcode.h: E_EOF (11):

          if (mod == NULL) {
              _PyArena_Free(arena);
              if (errcode == E_EOF) {
                  PyErr_Clear();
                  return E_EOF;
              }
              return -1;
          }
    11. vstinner commented on Oct 13, 2021

      @vstinner
      MemberAuthor

      New changeset aac29af by Victor Stinner in branch 'main':
      bpo-45434: pyport.h no longer includes <stdlib.h> (GH-28914)
      aac29af

    12. vstinner commented on Oct 14, 2021

      @vstinner
      MemberAuthor

      I tried to remove the <string.h> include from Python.h:

      -#include <string.h> // memcpy()

      ... but it required me to modify 100+ files. I don't know the exact number, I gave up after 100 files. I was near half of C extensions built by setup.py, so the number total may be around 150 files which should include explicitly <string.h>.

      "#include <errno.h>" can now be removed from Python.h, but I'm not sure if it's a good idea. I started by removing <stdlib.h> and my plan is to see if it breaks third party C extensions. Right now, Cython is broken which prevents me to use my https://github.com/vstinner/pythonci tool :-(

    13. vstinner commented on Oct 14, 2021

      @vstinner
      MemberAuthor

      New changeset 2849947 by Victor Stinner in branch 'main':
      bpo-45434: Limited Python.h no longer includes stdio.h (GH-28960)
      2849947

    14. vstinner commented on Oct 14, 2021

      @vstinner
      MemberAuthor

      New changeset 7076bef by Victor Stinner in branch 'main':
      bpo-45434: Remove useless space in includes (GH-28963)
      7076bef

    15. vstinner commented on Oct 15, 2021

      @vstinner
      MemberAuthor

      New changeset 105582e by Victor Stinner in branch 'main':
      bpo-45434: Remove Include/eval.h header file (GH-28973)
      105582e

    16. corona10 commented on Oct 18, 2021

      @corona10
      Member

      New changeset fd03917 by Dong-hee Na in branch 'main':
      bpo-45434: Include stdlib.h for specialize stat (GH-29015)
      fd03917

    17. Yhg1s commented on Oct 18, 2021

      @Yhg1s
      Member

      Victor, what's the benefit of doing this work? Are there real problems this fixes? I'm worried about the churn in third-party extensions, examples, tutorials, etc, especially for audiences that upon seeing a compiler error won't immediately realise they need to include stdlib.h themselves. (Also, since Python.h sets things like _POSIX_C_SOURCE and _XOPEN_SOURCE, including them in the wrong order can produce even more confusing errors, or errors that only appear on some platforms.)

    18. vstinner commented on Oct 18, 2021

      @vstinner
      MemberAuthor

      I reopen the issue.

    19. vstinner commented on Oct 19, 2021

      @vstinner
      MemberAuthor

      New changeset 52af075 by Victor Stinner in branch 'main':
      bpo-45434: Only exclude <stdlib.h> in Python 3.11 limited C API (GH-29027)
      52af075

    20. transferred this issue fromon Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

    Metadata

    Metadata

    Assignees

    No one assigned

      Labels

      Projects

      No projects

        Milestone

        No milestone

        Relationships

        None yet

        Development

        No branches or pull requests

        Issue actions