Repository navigation
Endianness not detected correctly due to AC_RUN_IFELSE macros #64541
Description
| BPO | 20342 |
|---|---|
| Nosy | @doko42, @gpshead, @mdickinson, @vstinner, @moreati |
| Files |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = None
closed_at = None
created_at = <Date 2014-01-22.08:44:31.459>
labels = ['interpreter-core', 'type-bug', 'build']
title = 'Endianness not detected correctly due to AC_RUN_IFELSE macros'
updated_at = <Date 2022-02-07.08:51:47.053>
user = 'https://bugs.python.org/zaytsev'bugs.python.org fields:
activity = <Date 2022-02-07.08:51:47.053>
actor = 'gregory.p.smith'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Interpreter Core', 'Cross-Build']
creation = <Date 2014-01-22.08:44:31.459>
creator = 'zaytsev'
dependencies = []
files = ['33613']
hgrepos = []
issue_num = 20342
keywords = []
message_count = 8.0
messages = ['208760', '208765', '208772', '208780', '208786', '412719', '412720', '412723']
nosy_count = 6.0
nosy_names = ['doko', 'gregory.p.smith', 'mark.dickinson', 'vstinner', 'zaytsev', 'Alex.Willmer']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue20342'
versions = ['Python 3.4']
Activity
zaytsev commentedon Jan 22, 2014
Hi,
I've managed to get the test suite of Python to run on Python 3.4.0b2 version cross-compiled for a specialized POWER chip. I get a number of failures that seem to have the same nature. I'm under impression, that actually the tests might need to be adjusted, but I'm not sure about it. I'm happy to provide any additional information to diagnose this issue. Please let me know if I've grouped them wrong and have to file a different issue for some tests. Thanks!
== CPython 3.4.0b2 (default, Jan 7 2014, 10:01:53) [GCC 4.4.6]
== Linux-2.6.32-358.11.1.bgq.el6.bgas_20131217.ppc64-ppc64-with-redhat-6.4-Santiago big-endian
== hash algorithm: fnv 64bit
== /tmp/test_python_11312
Testing with flags: sys.flags(debug=0, inspect=0, interactive=0, optimize=0, dont_write_bytecode=0, no_user_site=0, no_site=0, ignore_environment=0, verbose=0, bytes_warning=0, quiet=0, hash_randomization=1, isolated=0)
======================================================================
FAIL: fromhex (builtins.float)
Doctest: builtins.float.fromhex
----------------------------------------------------------------------
File "builtins", line ?, in builtins.float.fromhex
Failed example:
float.fromhex('-0x1p-1074')
Expected:
-5e-324
Got:
-4.9406564584124654e-324
----------------------------------------------------------------------
======================================================================
FAIL: test_float__format__ (test.test_types.TypesTests)
----------------------------------------------------------------------
Traceback (most recent call last): File "/XXX/opt/python-3.4.0b2-gcc-cnk/lib/python3.4/test/test_types.py", line 467, in test_float__format__ test(1e200, '+', '+1e+200') File "/XXX/opt/python-3.4.0b2-gcc-cnk/lib/python3.4/test/test_types.py", line 403, in test self.assertEqual(f.__format__(format_spec), result) AssertionError: '+9.9999999999999997e+199' != '+1e+200' - +9.9999999999999997e+199 + +1e+200 ----------------------------------------------------------------------======================================================================
FAIL: SequenceMatcher (difflib)
Doctest: difflib.SequenceMatcher
----------------------------------------------------------------------
File "/XXX/opt/python-3.4.0b2-gcc-cnk/lib/python3.4/difflib.py", line 83, in difflib.SequenceMatcher
Failed example:
print(round(s.ratio(), 3))
Expected:
0.866
Got:
0.86599999999999999
----------------------------------------------------------------------
======================================================================
FAIL: test_format_testfile (test.test_float.FormatTestCase)
----------------------------------------------------------------------
Traceback (most recent call last): File "/XXX/opt/python-3.4.0b2-gcc-cnk/lib/python3.4/test/test_float.py", line 592, in test_format_testfile self.assertEqual(fmt % float(arg), rhs) AssertionError: '0.029999999999999999' != '0.03' - 0.029999999999999999 + 0.03 ----------------------------------------------------------------------======================================================================
FAIL: test_non_ascii (test.test_format.FormatTest)
----------------------------------------------------------------------
Traceback (most recent call last): File "/XXX/opt/python-3.4.0b2-gcc-cnk/lib/python3.4/test/test_format.py", line 272, in test_non_ascii self.assertEqual(format(12.3, "\u2007<6"), "12.3\u2007\u2007") AssertionError: '12.300000000000001' != '12.3\u2007\u2007' - 12.300000000000001 + 12.3\u2007\u2007 ----------------------------------------------------------------------======================================================================
FAIL: test_float_default (test.test_optparse.TestExpandDefaults)
----------------------------------------------------------------------
Options:
-h, --help show this help message and exit
-p PROB, --prob=PROB blow up with probability PROB [default: 0.43]
"; got:
"Usage: test [options]
Options:
-h, --help show this help message and exit
-p PROB, --prob=PROB blow up with probability PROB [default:
0.42999999999999999]
"
----------------------------------------------------------------------
test_doc_tests (test.test_statistics.DocTests) ... **********************************************************************
File "/XXX/opt/python-3.4.0b2-gcc-cnk/lib/python3.4/statistics.py", line 290, in statistics.mean
Failed example:
mean([1, 2, 3, 4, 4])
Expected:
2.8
Got:
2.7999999999999998
File "/XXX/opt/python-3.4.0b2-gcc-cnk/lib/python3.4/statistics.py", line 380, in statistics.median_grouped
Failed example:
median_grouped([1, 2, 2, 3, 4, 4, 4, 4, 4, 5])
Expected:
3.7
Got:
3.7000000000000002
File "/XXX/opt/python-3.4.0b2-gcc-cnk/lib/python3.4/statistics.py", line 602, in statistics.pstdev
Failed example:
pstdev([1.5, 2.5, 2.5, 2.75, 3.25, 4.75])
Expected:
0.986893273527251
Got:
0.98689327352725098
3 items had failures:
1 of 5 in statistics.mean
1 of 4 in statistics.median_grouped
1 of 1 in statistics.pstdev
***Test Failed*** 3 failures.
zaytsev commentedon Jan 22, 2014
The CPU model is IBM PowerPC A2, the information about the cross-compiler is as follows:
vstinner commentedon Jan 22, 2014
According to what Yury told me on IRC, it's a cross-compilation issue. He wrote me that the following commands returns b'C?\xff\x01\x02\x03\x04\x05' which is the double big endian version, but DOUBLE_IS_BIG_ENDIAN_IEEE754 is not defined from pyconfig.h
./python -c 'import struct; print(ascii(struct.pack("d", 9006104071832581.0)))'
zaytsev commentedon Jan 22, 2014
I've re-named the issue to reflect the problem.
One possible solution that I can see is to use AC_C_BIGENDIAN macro, either exclusively, or only when cross-compiling.
In the latest autoconf sources, this macro seems to try to detect the endianness from various macros, and then by grepping the produced object files, so it's safe to use during cross-compilation.
Additionally, it allows to set the endianness by hand, if it can't be detected automatically.
Any thoughts?
[-]Float rounding issues on Red Hat 6.4 / PPC64 / GCC 4.4.6[/-][+]Endianness not detected correctly due to AC_RUN_IFELSE macros[/+]vstinner commentedon Jan 22, 2014
doko> One possible solution that I can see is to use AC_C_BIGENDIAN
macro, either exclusively, or only when cross-compiling.
This is wrong. The macro is not used to check the endian but to check
the IEEE 754 implementation. The check is important to decide if David
M. Gay's algorithm for "short float representation" can be used. See
pyport.h:
/* If we can't guarantee 53-bit precision, don't use the code
in Python/dtoa.c, but fall back to standard code. This
means that repr of a float will be long (17 sig digits).
Realistically, there are two things that could go wrong:
(1) doubles aren't IEEE 754 doubles, or
(2) we're on x86 with the rounding precision set to 64-bits
(extended precision), and we don't know how to change
the rounding precision.
*/
--
Python 3.4 already uses AC_C_BIGENDIAN in configure.ac, see how it is
used in pyport.h.
gpshead commentedon Feb 7, 2022
any idea if this is still relevant?
zaytsev commentedon Feb 7, 2022
Well, the machine that I had the problem on does no longer exist and I'm not sure whether this is a generic POWER issue.
The problem was that I had to cross-compile Python on one generation of POWER machines for another POWER chip using its own special toolchain. I can't tell whether Python build system was doing something wrong, or else whether it was a problem with the toolchain. In theory, both are valid options.
Also, I can't test whether the current version of Python has this problem or not with this particular chip / toolchain anymore. Maybe someone who has interest in POWER could try to cross-compile from AMD64 to POWER and check whether the result is still subtly broken... this should be much easier, and if there is a genuine problem with Python build system, it will probably expose it just as well.
gpshead commentedon Feb 7, 2022
I'll leave it open, nothing about this should be ppc64 vs ppc64el specific. Our cross compilation story has historically been not great.
Building for a target of one endianness from a build host of other endianness... sounds like exactly one of many kinds of thing that we might have a mess with until someone can confirm. (I'm not setup with a big endian toolchain to build for a qemu target right now so I can't easily confirm)