Skip to content

FreeBSD buildbots cannot compile Python #90421

Closed
Closed
FreeBSD buildbots cannot compile Python#90421
Labels
3.10 (EOL)end of life3.11only security fixes3.9 (EOL)end of lifebuildThe build process and cross-build

Description

@pablogsal
pablogsal
opened on Jan 4, 2022
BPO 46263
Nosy @gvanrossum, @tiran, @ned-deily, @ambv, @ericsnowcurrently, @koobs, @zooba, @pablogsal, @miss-islington, @erlend-aasland
PRs
  • bpo-46263: Don't use MULTIARCH on FreeBSD #30410
  • [3.10] bpo-46263: Don't use MULTIARCH on FreeBSD (GH-30410) #30413
  • [3.9] bpo-46263: Don't use MULTIARCH on FreeBSD (GH-30410) #30414
  • bpo-46263: FreeBSD 14.0 jemalloc workaround for junk bytes of freed memory #30434
  • [3.10] bpo-46263: FreeBSD 14.0 jemalloc workaround for junk bytes of freed memory (GH-30434) #30436
  • [3.9] bpo-46263: FreeBSD 14.0 jemalloc workaround for junk bytes of freed memory (GH-30434) (GH-30437) #30437
  • bpo-46263: Do not ever expect "use_frozen_modules" to be -1. #30438
  • bpo-46263: Fix second location that needs MALLOC_CONF on FreeBSD #30440
  • [3.10] bpo-46263: Fix second location that needs MALLOC_CONF on FreeBSD (GH-30440) #30442
  • [3.9] bpo-46263: Fix second location that needs MALLOC_CONF on FreeBSD (GH-30440) #30443
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = <Date 2022-01-06.20:49:48.873>
created_at = <Date 2022-01-04.23:09:03.585>
labels = ['build', '3.9', '3.10', '3.11']
title = 'FreeBSD buildbots cannot compile Python'
updated_at = <Date 2022-04-08.10:04:22.749>
user = 'https://github.com/pablogsal'

    bugs.python.org fields:

    activity = <Date 2022-04-08.10:04:22.749>
actor = 'koobs'
assignee = 'none'
closed = True
closed_date = <Date 2022-01-06.20:49:48.873>
closer = 'erlendaasland'
components = ['Build']
creation = <Date 2022-01-04.23:09:03.585>
creator = 'pablogsal'
dependencies = []
files = []
hgrepos = []
issue_num = 46263
keywords = ['patch']
message_count = 41.0
messages = ['409719', '409722', '409727', '409731', '409741', '409744', '409746', '409747', '409765', '409768', '409775', '409824', '409828', '409831', '409832', '409833', '409834', '409850', '409851', '409858', '409863', '409865', '409873', '409875', '409876', '409877', '409878', '409880', '409882', '409885', '409888', '409889', '409891', '409892', '409896', '409897', '409900', '409906', '409910', '409913', '409924']
nosy_count = 10.0
nosy_names = ['gvanrossum', 'christian.heimes', 'ned.deily', 'lukasz.langa', 'eric.snow', 'koobs', 'steve.dower', 'pablogsal', 'miss-islington', 'erlendaasland']
pr_nums = ['30410', '30413', '30414', '30434', '30436', '30437', '30438', '30440', '30442', '30443']
priority = None
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'compile error'
url = 'https://bugs.python.org/issue46263'
versions = ['Python 3.9', 'Python 3.10', 'Python 3.11']

    Activity

    1. pablogsal commented on Jan 4, 2022

      @pablogsal
      pablogsal
      on Jan 4, 2022
      MemberAuthor

      Seems that the FreeBSD buildbots cannot compile Python 3.11 anymore:

      LD_LIBRARY_PATH=/usr/home/buildbot/python/3.x.koobs-freebsd-564d/build CC='cc -pthread' LDSHARED='cc -pthread -shared    ' OPT='-g -O0 -Wall'  _TCLTK_INCLUDES='' _TCLTK_LIBS=''  ./python -E ./setup.py  build
Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-564d/build/./setup.py", line 49, in <module>
    from distutils.command.build_ext import build_ext
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-564d/build/Lib/distutils/command/build_ext.py", line 13, in <module>
    from distutils.sysconfig import customize_compiler, get_python_version
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-564d/build/Lib/distutils/sysconfig.py", line 53, in <module>
    _config_vars = get_config_vars()
                   ^^^^^^^^^^^^^^^^^
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-564d/build/Lib/sysconfig.py", line 621, in get_config_vars
    _init_posix(_CONFIG_VARS)
    ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-564d/build/Lib/sysconfig.py", line 482, in _init_posix
    _temp = __import__(name, globals(), locals(), ['build_time_vars'], 0)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ModuleNotFoundError: No module named '_sysconfigdata_d_freebsd14_x86_64-unknown-freebsd14'
*** [sharedmods] Error code 1
make: stopped in /usr/home/buildbot/python/3.x.koobs-freebsd-564d/build
make: stopped in /usr/home/buildbot/python/3.x.koobs-freebsd-564d/build

      https://buildbot.python.org/all/#/builders/483/builds/1408

    2. pablogsal
      added
      release-blocker
      3.11only security fixes
      on Jan 4, 2022

    3. pablogsal commented on Jan 4, 2022

      @pablogsal
      pablogsal
      on Jan 4, 2022
      MemberAuthor

      There is also this other builder:

      https://buildbot.python.org/all/#/builders/172/builds/1093

      failing what what seems is a freeze problem:

      -  'use_frozen_modules': 1,
+  'use_frozen_modules': -1,
?                        +
   'use_hash_seed': 0,
   'user_site_directory': 1,
   'verbose': 0,
   'warn_default_encoding': 0,
   'warnoptions': [],
   'write_bytecode': 1,
   'xoptions': []}

      Ran 1 test in 0.152s
      FAILED (failures=1)
      test test_embed failed
      1 test failed again:
      test_embed

      Adding Eric and Guido for this particular problem to the nosy list

    4. ned-deily commented on Jan 5, 2022

      @ned-deily
      ned-deily
      on Jan 5, 2022
      Member

      Not to point out the obvious but this buildbot uses --enable-shared which is often a source of build / bootstrap issues, such as the compiler executable being built ends up dynamically loading a libpython from a previous installation or build.

    5. gvanrossum
      changed the title [-]FreeBSL buiildbots cannot compile Python[/-] [+]FreeBSD buildbots cannot compile Python[/+] on Jan 5, 2022
    6. gvanrossum
      changed the title [-]FreeBSL buiildbots cannot compile Python[/-] [+]FreeBSD buildbots cannot compile Python[/+] on Jan 5, 2022

    7. tiran commented on Jan 5, 2022

      @tiran
      tiran
      on Jan 5, 2022
      Member

      On my FreeBSD 13 VM the sysconfig data module is

      build/lib.freebsd-13.0-RELEASE-amd64-3.11-pydebug/sysconfigdata_d_freebsd13.py

      It does not have any multiarch suffix. The multiarch suffix is detected by configure.ac. Did FreeBSD 14.0's CC get support for "cc --print-multiarch"? I found a related FreeBSD bug https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=258377

    8. tiran commented on Jan 5, 2022

      @tiran
      tiran
      on Jan 5, 2022
      Member

      I have tested my PR on a FreeBSD VM. Python fails to compile on main and builds correctly with my patch.

    9. tiran
      added
      buildThe build process and cross-build
      3.9 (EOL)end of life
      3.10 (EOL)end of life
      on Jan 5, 2022

    10. 38 remaining items

    11. ericsnowcurrently commented on Jan 6, 2022

      @ericsnowcurrently
      ericsnowcurrently
      on Jan 6, 2022
      Member

      A FreeBSD builder where it's working:

      AMD64 FreeBSD Shared 3.x (koobs-freebsd-564d)
      OS: FreeBSD 14.0-CURRENT main-n244626-cb7cc72c546: GENERIC-NODEBUG (amd64) System: 2-core, 8Gb, VirtualBox (6.x) Guest Features: - Compiler: FreeBSD clang version 11.0.1 (llvmorg-11.0.1-0-g43ff75f2c3fe) - Clang "is" cc (default system compiler) - OpenSSL 1.1.1i-freebsd 8 Dec 2020

      Where it isn't:

      AMD64 FreeBSD Non-Debug 3.x (koobs-freebsd-9e36)
      Environment: * OS: FreeBSD 12.3-RELEASE r371126 GENERIC amd64 * System: 2-core, 2Gb, VirtualBox (6.x) Guest Features: * cc = FreeBSD clang version 10.0.1 (git@github.com:llvm/llvm-project.git llvmorg-10.0.1-0-gef32c611aa2) * DTRACE * ZFS * Posix Semaphores

      The main difference I see is the FreeBSD version.

    12. erlend-aasland commented on Jan 6, 2022

      @erlend-aasland
      erlend-aasland
      on Jan 6, 2022
      Contributor

      Also, AMD64 FreeBSD Shared 3.x is configured --with-pydebug and --enable-shared. I'd be surprised if those were the root cause of this though. Both buildbots are in-tree builds.

      FWIW, my FreeBSD 14.0-CURRENT main-n250453-7ac82c96fe7 VM also fails, as noted in previous posts. It's built with just ./configure && make.

    13. ericsnowcurrently commented on Jan 6, 2022

      @ericsnowcurrently
      ericsnowcurrently
      on Jan 6, 2022
      Member

      I just put up a PR that should fix the failure.

    14. ericsnowcurrently commented on Jan 6, 2022

      @ericsnowcurrently
      ericsnowcurrently
      on Jan 6, 2022
      Member

      New changeset 68c76d9 by Eric Snow in branch 'main':
      bpo-46263: Do not ever expect "use_frozen_modules" to be -1. (gh-30438)
      68c76d9

    15. erlend-aasland commented on Jan 6, 2022

      @erlend-aasland
      erlend-aasland
      on Jan 6, 2022
      Contributor

      Great. I tried the same trick on my VM earlier today, but I was unsure if it was the correct fix :) Thanks!

    16. ericsnowcurrently commented on Jan 6, 2022

      @ericsnowcurrently
      ericsnowcurrently
      on Jan 6, 2022
      Member

      test_pyobject_freed_is_freed is still failing on AMD64 FreeBSD Shared 3.x.

    17. miss-islington commented on Jan 6, 2022

      @miss-islington
      miss-islington
      on Jan 6, 2022
      Contributor

      New changeset c9137d4 by Christian Heimes in branch 'main':
      bpo-46263: Fix second location that needs MALLOC_CONF on FreeBSD (GH-30440)
      c9137d4

    18. miss-islington commented on Jan 6, 2022

      @miss-islington
      miss-islington
      on Jan 6, 2022
      Contributor

      New changeset 3cb64ed by Miss Islington (bot) in branch '3.10':
      bpo-46263: Fix second location that needs MALLOC_CONF on FreeBSD (GH-30440)
      3cb64ed

    19. miss-islington commented on Jan 6, 2022

      @miss-islington
      miss-islington
      on Jan 6, 2022
      Contributor

      New changeset b98730c by Miss Islington (bot) in branch '3.9':
      bpo-46263: Fix second location that needs MALLOC_CONF on FreeBSD (GH-30440)
      b98730c

    20. erlend-aasland commented on Jan 6, 2022

      @erlend-aasland
      erlend-aasland
      on Jan 6, 2022
      Contributor

      Both FreeBSD bots are green again. Thank you Christian and Eric!

    21. erlend-aasland
      closed this as completedon Jan 6, 2022
    22. erlend-aasland
      removed
      release-blocker
      on Jan 6, 2022
    23. erlend-aasland
      closed this as completedon Jan 6, 2022
    24. erlend-aasland
      removed
      release-blocker
      on Jan 6, 2022
    25. ezio-melotti
      transferred this issue fromon Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

    Metadata

    Metadata

    Assignees

    No one assigned

      Labels

      3.10 (EOL)end of life3.11only security fixes3.9 (EOL)end of lifebuildThe build process and cross-build

      Projects

      No projects

        Milestone

        No milestone

        Relationships

        None yet

        Development

        No branches or pull requests

        Issue actions