Repository navigation
Decorate hot functions using __attribute__((hot)) to optimize Python #72804
Description
| BPO | 28618 |
|---|---|
| Nosy | @jcea, @pitrou, @vstinner, @methane, @serhiy-storchaka, @1st1 |
| Files |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = None
closed_at = <Date 2017-02-01.17:21:32.456>
created_at = <Date 2016-11-05.00:29:04.634>
labels = ['interpreter-core', '3.7', 'performance']
title = 'Decorate hot functions using __attribute__((hot)) to optimize Python'
updated_at = <Date 2017-05-18.00:42:46.111>
user = 'https://github.com/vstinner'bugs.python.org fields:
activity = <Date 2017-05-18.00:42:46.111>
actor = 'jcea'
assignee = 'none'
closed = True
closed_date = <Date 2017-02-01.17:21:32.456>
closer = 'vstinner'
components = ['Interpreter Core']
creation = <Date 2016-11-05.00:29:04.634>
creator = 'vstinner'
dependencies = []
files = ['45361', '45396', '45397', '45488']
hgrepos = []
issue_num = 28618
keywords = ['patch']
message_count = 34.0
messages = ['280097', '280105', '280106', '280108', '280115', '280116', '280125', '280126', '280350', '280556', '280557', '280568', '280606', '280607', '280675', '280679', '280748', '280764', '280831', '280832', '280844', '280845', '280846', '280849', '280853', '280859', '280860', '281459', '281463', '281466', '281467', '281473', '281477', '286662']
nosy_count = 7.0
nosy_names = ['jcea', 'pitrou', 'vstinner', 'methane', 'python-dev', 'serhiy.storchaka', 'yselivanov']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = None
status = 'closed'
superseder = None
type = 'performance'
url = 'https://bugs.python.org/issue28618'
versions = ['Python 3.7']
Activity
vstinner commentedon Nov 5, 2016
When analyzing results of Python performance benchmarks, I noticed that call_method was 70% slower (!) between revisions 83877018ef97 (Oct 18) and 3e073e7b4460 (Oct 22), including these revisions, on the speed-python server.
On these revisions, the CPU L1 instruction cache is less efficient: 8% cache misses, whereas it was only 0.06% before and after these revisions.
Since the two mentioned revisions have no obvious impact on the call_method() benchmark, I understand that the performance difference by a different layout of the machine code, maybe the exact location of functions.
IMO the best solution to such compilation issue is to use PGO compilation. Problem: PGO doesn't work on Ubuntu 14.04, the OS used by speed-python (the server runining benchmarks for http://speed.python.org/).
I propose to decorate manually the "hot" functions using the GCC __attribute__((hot)) decorator:
https://gcc.gnu.org/onlinedocs/gcc/Common-Function-Attributes.html#Common-Function-Attributes
(search for "hot")
Attached patch adds Py_HOT_FUNCTION and decorates the following functions:
These functions are the top 6 according to the Linux perf tool when running the call_simple benchmark of the performance project:
32,66%: _PyEval_EvalFrameDefault
13,09%: PyFrame_New
12,78%: call_function
12,24%: lookdict_unicode_nodummy
9,85%: _PyFunction_FastCall
8,47%: frame_dealloc
vstinner commentedon Nov 5, 2016
I ran benchmarks. Globally, it seems like the impact of the patch is positive. regex_v8 and call_simple are slower, but these benchmarks are microbenchmarks impacted by low level stuff like CPU L1 cache. Well, my patch was supposed to optimize CPython for call_simple :-/ I should maybe investigate a little bit more.
Performance comparison (performance 0.3.2):
haypo@smithers$ python3 -m perf compare_to orig.json hot.json -G
Slower (6):
Faster (32):
Benchmark hidden because not significant (26): 2to3, call_method, chaos, crypto_pyaes, float, hexiom, html5lib, json_loads, logging_silent, mako, meteor_contest, pathlib, pickle, python_startup_no_site, scimark_sparse_mat_mult, spectral_norm, sqlalchemy_imperative, sympy_sum, telco, tornado_http, unpack_sequence, unpickle, unpickle_list, unpickle_pure_python, xml_etree_generate, xml_etree_iterparse
--
More readable output, only display differences >= 5%:
haypo@smithers$ python3 -m perf compare_to orig.json hot.json -G --min-speed=5
Slower (1):
Faster (2):
Benchmark hidden because not significant (61): 2to3, call_method, call_method_slots, call_method_unknown, call_simple, chaos, crypto_pyaes, deltablue, django_template, dulwich_log, fannkuch, float, genshi_text, genshi_xml, go, hexiom, html5lib, json_dumps, json_loads, logging_format, logging_silent, logging_simple, mako, meteor_contest, nbody, nqueens, pathlib, pickle, pickle_dict, pickle_list, pickle_pure_python, pidigits, python_startup, python_startup_no_site, raytrace, regex_compile, regex_dna, regex_effbot, richards, scimark_fft, scimark_lu, scimark_sor, scimark_sparse_mat_mult, spectral_norm, sqlalchemy_declarative, sqlalchemy_imperative, sqlite_synth, sympy_expand, sympy_integrate, sympy_str, sympy_sum, telco, tornado_http, unpack_sequence, unpickle, unpickle_list, unpickle_pure_python, xml_etree_generate, xml_etree_iterparse, xml_etree_parse, xml_etree_process
vstinner commentedon Nov 5, 2016
Oh, I forgot to mention that I compiled Python with "./configure -C". The purpose of the patch is to optimize Python when LTO and/or PGO compilation are not explicitly used.
pitrou commentedon Nov 5, 2016
Can you compare against a PGO build? Ubuntu 14.04 is old, and I don't think this is something we should worry about.
Overall I think this manual approach is really the wrong way to look at it. Compilers can do better than us.
vstinner commentedon Nov 5, 2016
Antoine Pitrou added the comment:
Do you mean comparison between current Python with PGO and patched
Python without PGO?
The hot attribute is ignored by GCC when PGO compilation is used.
Well, it's a practical issue for me to run benchmarks for speed.python.org.
Moreover, I like the idea of getting a fast(er) Python even when no
advanced optimization techniques like LTO or PGO is used. At least,
it's common to build quickly Python using "./configure && make" for a
quick benchmark.
serhiy-storchaka commentedon Nov 5, 2016
Seconded.
pitrou commentedon Nov 5, 2016
Le 05/11/2016 à 16:37, STINNER Victor a écrit :
Yes.
Why isn't the OS updated on that machine?
vstinner commentedon Nov 5, 2016
Antoine Pitrou added the comment:
Oh ok, sure. I will try to run these 2 benchmarks.
I am not sure that I want to use PGO compilation to run benchmarks.
Last time I checked, I noticed performance differences between two
compilations. PGO compilation doesn't seem 100% deterministic.
Maybe PGO compilation is fine when you build Python to create a Linux
package. But to get reliable benchmarks, I'm not sure that it's a good
idea.
I'm still running benchmarks on Python recompiled many times using
different compiler options, to measure the impact of the compiler
options (especially LTO and/or PGO) on the benchmark stability.
vstinner commentedon Nov 8, 2016
Ok, here you have. As expected, PGO compilation is faster than default compilation with my patch. PGO implements more optimization than just __attribute__((hot)), it also optimizes branches for example.
haypo@smithers$ python3 -m perf compare_to pgo.json.gz patch.json.gz -G --min-speed=5
Slower (56):
Benchmark hidden because not significant (8): fannkuch, float, json_dumps, nbody, regex_v8, richards, scimark_monte_carlo, scimark_sparse_mat_mult
python-dev commentedon Nov 11, 2016
New changeset 59b91b4e9506 by Victor Stinner in branch 'default':
Issue bpo-28618: Make hot functions using __attribute__((hot))
https://hg.python.org/cpython/rev/59b91b4e9506
vstinner commentedon Nov 11, 2016
I tried different patches and ran many quick & dirty benchmarks.
I tried to use likely/unlikely macros (using GCC __builtin__expect): the effect is not significant on call_simple microbenchmark. I gave up on this part.
__attribute__((hot)) on a few Python core functions fixes the major slowdown on call_method on the revision 83877018ef97 (described in the initial message).
I noticed tiny differences when using __attribute__((hot)), speedup in most cases. I noticed sometimes slowdown, but very small (ex: 1%, but 1% on a microbenchmark doesn't mean anything).
I pushed my patch to try to keep stable performance when Python is not compiled with PGO.
If you would like to know more about the crazy effect of code placement in modern Intel CPUs, I suggest you to see the slides of this recent talk from an Intel engineer:
https://llvmdevelopersmeetingbay2016.sched.org/event/8YzY/causes-of-performance-instability-due-to-code-placement-in-x86
"Causes of Performance Swings Due to Code Placement in IA by Zia Ansari (Intel), November 2016"
--
About PGO or not PGO: this question is not simple, I suggest to discuss it in a different place to not flood this issue ;-)
For my use case, I'm not convinced yet that PGO with our current build system produce reliable performance.
Not all Linux distributions compile Python using PGO: Fedora and RHEL don't compile Python using PGO for example. Bugzilla for Fedora:
https://bugzilla.redhat.com/show_bug.cgi?id=613045
I guess that there also some developers running benchmarks on Python compiled with "./configure && make". I'm trying to enhance documentation and tools around Python benchmarks to advice developers to use LTO and/or PGO.
vstinner commentedon Nov 11, 2016
Final result on speed-python:
haypo@speed-python$ python3 -m perf compare_to json_8nov/2016-11-10_15-39-default-8ebaa546a033.json 2016-11-11_02-13-default-59b91b4e9506.json -G
Slower (12):
Faster (39):
Benchmark hidden because not significant (13): float, genshi_xml, pickle, pickle_dict, pidigits, scimark_fft, scimark_monte_carlo, sqlalchemy_imperative, sqlite_synth, sympy_expand, sympy_sum, unpickle, unpickle_list
9 remaining items
vstinner commentedon Nov 15, 2016
Ok, your benchmark results doens't look bad, so I marked the following functions as hot:
It's common to see these functions in the top 3 of "perf report".
vstinner commentedon Nov 15, 2016
hot3.patch: Mark additional functions as hot
I'm not sure about this patch. It's hard to get reliable benchmark results on microbenchmarks :-/ It's hard to understand if a speedup comes from the hot attribute, or if the compiler decided itself to change the code placement. Without the hot attribute, the code placement seems random.
vstinner commentedon Nov 15, 2016
I wrote hot3.patch when trying to make the following benchmarks more reliable:
15% slower than the average on 2016
cf7711887b4a (nov 7) takes 5.2 ms, rev 8ebaa546a033 (nov 10) takes 6.1
ms, etc.
serhiy-storchaka commentedon Nov 15, 2016
This doesn't look wise. This is specific to single extension module and perhaps to single particular benchmark. Most Python code don't use json at all.
What is the top of "perf report"? How this list intersects with the list of functions in .text.hot section of PGO build? Make several PGO builds (perhaps on different computers). Is .text.hot section stable?
vstinner commentedon Nov 15, 2016
Here are benchmark results on the speed-python server:
haypo@speed-python$ PYTHONPATH=~/perf python -m perf compare_to 2016-11-15_09-12-default-ac93d188ebd6.json 2016-11-15_15-13-default-cfc956f13ce2.json -G --min-speed=1
Slower (6):
Faster (26):
Benchmark hidden because not significant (32): 2to3, call_method, call_method_slots, call_method_unknown, crypto_pyaes, genshi_text, go, html5lib, logging_silent, meteor_contest, nqueens, pickle_dict, pickle_list, pickle_pure_python, pidigits, python_startup, python_startup_no_site, regex_compile, regex_dna, regex_v8, richards, scimark_fft, scimark_lu, scimark_sparse_mat_mult, sqlalchemy_declarative, sympy_integrate, sympy_sum, unpack_sequence, unpickle, unpickle_list, unpickle_pure_python, xml_etree_parse
vstinner commentedon Nov 15, 2016
Serhiy Storchaka:
Well, I tried different things to make these benchmarks more stable. I didn't say that we should merge hot3.patch as it is :-) It's just an attempt.
For json_loads, it's:
14.99% _json.cpython-37m-x86_64-linux-gnu.so scanstring_unicode
8.34% python _PyUnicode_FromUCS1
8.32% _json.cpython-37m-x86_64-linux-gnu.so scan_once_unicode
8.01% python lookdict_unicode_nodummy
6.72% python siphash24
4.45% python PyDict_SetItem
4.26% python _PyObject_Malloc
3.38% python _PyEval_EvalFrameDefault
3.16% python _Py_HashBytes
2.72% python PyUnicode_New
2.36% python PyLong_FromString
2.25% python _PyObject_Free
2.02% libc-2.19.so __memcpy_sse2_unaligned
1.61% python PyDict_GetItem
1.40% python dictresize
1.24% python unicode_hash
1.11% libc-2.19.so _int_malloc
1.07% python unicode_dealloc
1.00% python free_keys_object
Result produced with:
$ perf record ./python ~/performance/performance/benchmarks/bm_json_loads.py --worker -v -l 128 -w0 -n 100 $ perf reportI checked which functions are considered as "hot" by a PGO build: I found more than 2,000 functions... I'm not interested to tag so many functions with _Py_HOT_FUNCTIONS. I would prefer to only tag something like the top 10 or top 25 functions.
I don't know the recommandations to tag functions as hot. I guess that what matters is the total size of hot functions. Should I be smaller than the L2 cache? Smaller than the L3 cache? I'm talking about instructions, but data share also these caches...
In my experience PGO builds don't provide stable performances, but I was never able to write an article on that because of so many bugs :-)
vstinner commentedon Nov 22, 2016
FYI I wrote an article about this issue:
https://haypo.github.io/analysis-python-performance-issue.html
Sadly, it seems like I was just lucky when adding __attribute__((hot)) fixed the issue, because call_method is slow again!
methane commentedon Nov 22, 2016
Wow. It's sad that tagged version is accidentally slow...
I want to reproduce it and check
perf record -e L1-icache-load-misses.But IaaS (EC2, GCE, Azure VM) doesn't support CPU performance counter.
vstinner commentedon Nov 22, 2016
2016-11-22 12:07 GMT+01:00 INADA Naoki report@bugs.python.org:
You don't need to go that far to check performances: just run
call_method and check timings. You need to compare on multiple
revisions.
speed.python.org Timeline helps to track performances, to have an idea
of the "average performance" and detect spikes.
vstinner commentedon Nov 22, 2016
Naoki: "Wow. It's sad that tagged version is accidentally slow..."
If you use PGO compilation, for example use "./configure
--enable-optimizations" as suggested by configure if you don't enable
the option, you don't get the issue.
I hope that most Linux distribution use PGO compilation. I'm quite
sure that it's the case for Ubuntu. I don't know for Fedora.
methane commentedon Nov 22, 2016
I setup Ubuntu 14.04 on Azure, built python without neither PGO nor LTO.
But I failed to reproduce it.
@Haypo, would you give me two binaries?
vstinner commentedon Nov 22, 2016
Hey, performance issues with code placement is a mysterious secret :-)
Nobody understands it :-D
The server runner the benchmark is a Intel Xeon CPU of 2011. It seems
like code placement issues are more important on this CPU than my more
recent laptop or desktop PC.
vstinner commentedon Feb 1, 2017
Victor: "FYI I wrote an article about this issue: https://haypo.github.io/analysis-python-performance-issue.html Sadly, it seems like I was just lucky when adding __attribute__((hot)) fixed the issue, because call_method is slow again!"
I upgraded speed-python server (running benchmarks) to Ubuntu 16.04 LTS to support PGO compilation. I removed all old benchmark results and ran again benchmarks with LTO+PGO. It seems like benchmark results are much better now.
I'm not sure anymore that _Py_HOT_FUNCTION is really useful to get stable benchmarks, but it may help code placement a little bit. I don't think that it hurts, so I suggest to keep it. Since benchmarks were still unstable with _Py_HOT_FUNCTION, I'm not interested to continue to tag more functions with _Py_HOT_FUNCTION. I will now focus on LTO+PGO for stable benchmarks, and ignore small performance difference when PGO is not used.
I close this issue now.