android compilation and link flags

@Yhg1s

BPO	26851
Nosy	@Yhg1s, @doko42, @vstinner, @xdegaye, @vadmium, @moreati, @yan12125
PRs	[Do Not Merge] Convert `Misc/NEWS` so that it is managed by towncrier #552
Files	build-flags.patch build-flags_2.patch build-flags_3.patch build-flags_3.patch: uploading twice the same file build-flags_4.patch build-flags_5.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/xdegaye'
closed_at = <Date 2017-01-04.20:58:21.080>
created_at = <Date 2016-04-26.12:20:11.511>
labels = ['type-feature', '3.7', 'build']
title = 'android compilation and link flags'
updated_at = <Date 2017-03-31.16:36:19.595>
user = 'https://github.com/xdegaye'

bugs.python.org fields:

activity = <Date 2017-03-31.16:36:19.595>
actor = 'dstufft'
assignee = 'xdegaye'
closed = True
closed_date = <Date 2017-01-04.20:58:21.080>
closer = 'xdegaye'
components = ['Cross-Build']
creation = <Date 2016-04-26.12:20:11.511>
creator = 'xdegaye'
dependencies = []
files = ['42601', '43868', '43875', '43878', '43893', '45101']
hgrepos = []
issue_num = 26851
keywords = ['patch']
message_count = 19.0
messages = ['264266', '271169', '271193', '271246', '271275', '271282', '271292', '271353', '271518', '271530', '271543', '271552', '278708', '278712', '284667', '284668', '284739', '284745', '284752']
nosy_count = 8.0
nosy_names = ['twouters', 'doko', 'vstinner', 'xdegaye', 'python-dev', 'martin.panter', 'Alex.Willmer', 'yan12125']
pr_nums = ['552']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue26851'
versions = ['Python 3.6', 'Python 3.7']

The attached patch:

Sets the recommended android compilation flags, see: http://developer.android.com/ndk/guides/standalone_toolchain.html#abi. Note that the android toolchains already set the -fpic flag, as shown with:

arm-linux-androideabi-gcc -v -S main.c 2>&1 | grep main.c
Sets the Position independent executables (PIE) flag which is mandatory starting at API level 21 and supported starting with API level 16.

The previous patch was using awkwardly, a 'host_cpu' configure argument to specify the arm ABI and missed setting LDFLAGS for the armv7 ABI.
This patch instead uses the __ARM_ARCH macro defined by each compiler of the Android toolchains:

    echo | ./clang -target armv7-none-linux-androideabi -dM -E - | grep ARM_ARCH
      #define __ARM_ARCH 7
      #define __ARM_ARCH_7A__ 1
    echo | ./clang -target armv5te-none-linux-androideabi -dM -E - | grep ARM_ARCH
      #define __ARM_ARCH 5
      #define __ARM_ARCH_5TE__ 1
    echo | ./arm-linux-androideabi-gcc -march=armv7-a -dM -E - | grep ARM_ARCH
      #define __ARM_ARCH 7
      #define __ARM_ARCH_7A__ 1
    echo | ./arm-linux-androideabi-gcc -dM -E - | grep ARM_ARCH
      #define __ARM_ARCH_5TE__ 1
      #define __ARM_ARCH 5

Python built with clang for the armv5te target crashes as reported in bpo-27606.

I left some suggestions and questions on the code review.

Thanks for the review and the suggestions Martin :)
New patch.

The first 'build-flags_3.patch' file that was uploaded did not get a review button because ssh://hg@hg.python.org/cpython was down at that time.

The second upload has been successfull !

Note: build-flags_3.patch includes configure.ac and configure. Usually, we suggest to not included generated files (configure) in patches, but push them when the patch is ready.

Thanks for the suggestion, I won't include them anymore in patches then.

New patch. The sed commands used to evaluate ANDROID_API_LEVEL and _arm_arch do not need to discard comment lines and empty lines in the output of the preprocessor.

All the bits that I understand look okay now. :)

I am still curious what configures the preprocessor to set __ARM_ARCH to 7 (I guess the clang -target argument?), and why we can’t set LDFLAGS at the same time or place. Is it just more convenient this way?

I am still curious what configures the preprocessor to set __ARM_ARCH to 7 (I guess the clang -target argument?)
Yes, the -target clang argument or the -march gcc argument.

and why we can’t set LDFLAGS at the same time or place. Is it just more convenient this way?
I don't understand the question. LDFLAGS is set at the time we know that __ARM_ARCH is 7, just after the preprocessing is done. Would you set it elsewhere ?
LDFLAGS is set in configure.ac with always this same idiom: LDFLAGS="$LDFLAGS new_options..." and this setting is done in what seems to be random places in configure.ac.
CCSHARED is set in one case statement.

Where is the code that sets the clang -target argument, or gcc -march? Is it hidden away somewhere in the autoconf code? Do you manually set it with ‘./configure CFLAGS="-target . . ." ’?

Yes, the packager must use appropriately either CFLAGS CPPFLAGS [1] and LDFLAGS, or CC. I am using:
CC="clang --sysroot=$(SYSROOT) -target $(TARGET) -gcc-toolchain $(GCC_TOOLCHAIN)".

[1] See bpo-27453, for the remaining problem upon using CPPFLAGS.

See also the clang cross compilation documentation at http://clang.llvm.org/docs/CrossCompilation.html.

This new patch fixes the following:

Use BASECFLAGS instead of CCSHARED.
'-march=armv7-a' is not set in BASECFLAGS anymore as this is redundant: this option is already set in the configure command line or is implicitly set by the the first component of the triple of the '-target' clang option.
Do not set '-mthumb' as this is optional and may be set for the armv7 arch as well. Also this option crashes Python on armv5te when built with clang, bpo-27606.

New changeset fa2bc63e64c6 by Xavier de Gaye in branch '3.6':
Issue bpo-26851: Set Android compilation and link flags.
https://hg.python.org/cpython/rev/fa2bc63e64c6

New changeset af363b5200ff by Xavier de Gaye in branch 'default':
Issue bpo-26851: Merge 3.6.
https://hg.python.org/cpython/rev/af363b5200ff

Latest patch committed with the following simplification: BASECFLAGS and LDFLAGS are now set for armv7 at the same place, as suggested by Martin in msg271518.

In the code review Victor asked the following question:

I'm a little bit surprised that you need to pass so much options which are specific to the platform. GCC doesn't have a generic option "hello, please compile for my architecture?"

ARM defines two incompatible floating point ABIS that specify the functions calling conventions, whether to use integer registers (soft floating point ABI) or the floating point registers (hard floating point ABI). The compiler must be told which one to use and whether the hardware FPU may be used if present when the soft floating point ABI has been chosen. This last case is set with the command line option '-mfloat-abi=softfp'. The following two documents describe this feature in details:
https://wiki.debian.org/ArmHardFloatPort/VfpComparison
https://wiki.debian.org/ArmHardFloatPort

The following command prints the command line options, including implicitly predefined ones of the clang compiler in android-ndk-r13b and shows that the default is '-mfloat-abi=soft' for armv7:

$ ./clang -x c < /dev/null -dM -E - -target armv7-none-linux-androideabi -###
Android clang version 3.8.256229  (based on LLVM 3.8.256229)
Target: armv7-none-linux-android
Thread model: posix
InstalledDir: /opt/android-ndk/toolchains/llvm/prebuilt/linux-x86_64/bin/.
 "/home/opt.symlink/android-ndk-r13b/toolchains/llvm/prebuilt/linux-x86_64/bin/clang" "-cc1" "-triple" "armv7-none-linux-android" "-E" "-disable-free" "-disable-llvm-verifier" "-main-file-name" "-" "-mrelocation-model" "pic" "-pic-level" "1" "-mthread-model" "posix" "-mdisable-fp-elim" "-fmath-errno" "-masm-verbose" "-mconstructor-aliases" "-fuse-init-array" "-target-cpu" "cortex-a8" "-target-feature" "+soft-float-abi" "-target-abi" "aapcs-linux" "-mfloat-abi" "soft" "-target-linker-version" "2.24" "-dwarf-column-info" "-debugger-tuning=gdb" "-resource-dir" "/home/opt.symlink/android-ndk-r13b/toolchains/llvm/prebuilt/linux-x86_64/bin/../lib64/clang/3.8.256229" "-internal-isystem" "/usr/local/include" "-internal-isystem" "/home/opt.symlink/android-ndk-r13b/toolchains/llvm/prebuilt/linux-x86_64/bin/../lib64/clang/3.8.256229/include" "-internal-externc-isystem" "/include" "-internal-externc-isystem" "/usr/include" "-fdebug-compilation-dir" "/opt/android-ndk/toolchains/llvm/prebuilt/linux-x86_64/bin" "-ferror-limit" "19" "-fmessage-length" "100" "-femulated-tls" "-fallow-half-arguments-and-returns" "-fno-signed-char" "-fobjc-runtime=gcc" "-fdiagnostics-show-option" "-fcolor-diagnostics" "-dM" "-o" "-" "-x" "c" "-"

Note that the system libffi must also be compiled with these same flags. The bundled libffi is deprecated in 3.6 by bpo-27976 and has been removed in 3.7 by bpo-27979.

Note that the system libffi must also be compiled with these same flags

Just tried. With my packaging scripts, CPython on ARM is compiled with -mfloat-abi=softfp -mfpu=vfpv3-d16 while libffi not. test_ctypes pass as usual. Maybe ctypes test suite is not complete?

And by the way, if all packages on a system should be compiled with the same set of flags, shouldn't those flags specified in build/package scripts rather than Makefile of individual packages?

Just tried. With my packaging scripts, CPython on ARM is compiled with -mfloat-abi=softfp -mfpu=vfpv3-d16 while libffi not. test_ctypes pass as usual.

This works as expected, '-mfloat-abi=softfp' and the default '-mfloat-abi=soft' use the same ABI [1]: "Function calls are generated to pass FP arguments (float, double) in integer registers (one for float, a pair of registers for double)".

[1] https://wiki.debian.org/ArmHardFloatPort/VfpComparison

Navigation Menu

Repository navigation

android compilation and link flags #71038

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

android compilation and link flags #71038

Description

Activity

xdegaye commented on Apr 26, 2016

xdegaye commented on Jul 24, 2016

vadmium commented on Jul 25, 2016

xdegaye commented on Jul 25, 2016

xdegaye commented on Jul 25, 2016

vstinner commented on Jul 25, 2016

xdegaye commented on Jul 25, 2016

xdegaye commented on Jul 26, 2016

vadmium commented on Jul 28, 2016

xdegaye commented on Jul 28, 2016

vadmium commented on Jul 28, 2016

xdegaye commented on Jul 28, 2016

xdegaye commented on Oct 15, 2016

xdegaye commented on Oct 15, 2016

python-dev commented on Jan 4, 2017

xdegaye commented on Jan 4, 2017

xdegaye commented on Jan 5, 2017

yan12125 commented on Jan 5, 2017

xdegaye commented on Jan 5, 2017

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions