Repository navigation
sysconfig --generate-posix-vars creates wrong file when cross compiling #90067
Description
| BPO | 45909 |
|---|---|
| Nosy | @tiran, @moreati, @ethanhs |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = None
closed_at = None
created_at = <Date 2021-11-27.12:05:59.850>
labels = ['type-bug', 'build', '3.11']
title = 'sysconfig --generate-posix-vars creates wrong file when cross compiling'
updated_at = <Date 2021-11-27.19:03:41.450>
user = 'https://github.com/tiran'bugs.python.org fields:
activity = <Date 2021-11-27.19:03:41.450>
actor = 'christian.heimes'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Build', 'Cross-Build']
creation = <Date 2021-11-27.12:05:59.850>
creator = 'christian.heimes'
dependencies = []
files = []
hgrepos = []
issue_num = 45909
keywords = []
message_count = 2.0
messages = ['407141', '407163']
nosy_count = 3.0
nosy_names = ['christian.heimes', 'Alex.Willmer', 'ethan smith']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue45909'
versions = ['Python 3.11']
Activity
tiran commentedon Nov 27, 2021
"sysconfig --generate-posix-vars" creates pybuilddir.txt and a platform-specific sysconfig data file like build/lib.linux-x86_64-3.11/_sysconfigdata__linux_x86_64-linux-gnu.py
When creating a cross-compile build of Python, sysconfig mixes data from the cross compile build and the host build. It creates a pybuilddir.txt and build/lib directory with correct values (e.g. wasm32-unknown-emscripten) but sysconfigdata file with name values from the host Python PYTHON_FOR_BUILD (e.g x86_64-unknown-linux-gnu).
tiran commentedon Nov 27, 2021
I just realized that PYTHON_FOR_BUILD is far more complicated. Our WASM build system sets the env var to "$(pwd)/cpython/builddir/build/python". The configure script expects us to set several additional env vars.
The logic goes like this for cross compiling:
Since we don't set any of the variables _PYTHON_PROJECT_BASE, _PYTHON_HOST_PLATFORM, and _PYTHON_SYSCONFIGDATA_NAME, we get wrong sysconfig data file.
Osyotr commentedon Aug 21, 2022
I can't cross-compile python because of this.
Lib/sysconfig.pyusesos.nameandsys.platformalmost everywhere which are wrong in cross-compile scenario.For example, when building on windows for non-windows, the build will fail here:
cpython/Lib/sysconfig.py
Lines 590 to 593 in d8c7a11
_PYTHON_HOST_PLATFORMshould probably be used instead.Cmdline is like this:
_PYTHON_PROJECT_BASE=/d/vcpkg/buildtrees/python3/ntoarmv7-qnx-rel _PYTHON_HOST_PLATFORM=qnx-arm PYTHONPATH=./../src/v3.11.0rc1-bb53a96c2e.clean/Lib _PYTHON_SYSCONFIGDATA_NAME=_sysconfigdata__qnx_ C:/Users/Osyotr/Desktop/bb_gcc_test/out/vcpkg_installed/x64-windows/tools/python3/python.exe -S -m sysconfig --generate-posix-varstiran commentedon Aug 24, 2022
Cross compiling from Windows to a POSIX-like OS is uncharted territory. I suggest that you open a new ticket. Your use case needs feedback from core devs who are familiar with Windows build system, too.
FFY00 commentedon Nov 21, 2022
@tiran I don't know much about the WASM builds, but would it be possible for you to set the missing environment variables? Long term, I think we definitely want something better, but I think that requires standardizing and documenting cross-builds.
Do you have any alternative proposal?
manueljacob commentedon Apr 7, 2026
This very likely has the same root cause as #136267.