Repository navigation
Performance regression for making bound methods #83298
Description
Activity
$ python3.9 -m timeit -r 11 -s 'class A: pass' -s 'A.m = lambda s: None' -s 'a = A()' 'a.m; a.m; a.m; a.m; a.m' 1000000 loops, best of 11: 230 nsec per loop $ python3.8 -m timeit -r 11 -s 'class A: pass' -s 'A.m = lambda s: None' -s 'a = A()' 'a.m; a.m; a.m; a.m; a.m' 2000000 loops, best of 11: 149 nsec per loop $ python3.7 -m timeit -r 11 -s 'class A: pass' -s 'A.m = lambda s: None' -s 'a = A()' 'a.m; a.m; a.m; a.m; a.m' 2000000 loops, best of 11: 159 nsec per loop $ python3.6 -m timeit -r 11 -s 'class A: pass' -s 'A.m = lambda s: None' -s 'a = A()' 'a.m; a.m; a.m; a.m; a.m' 10000000 loops, best of 11: 0.159 usec per loop
Timings made using the recent released python.org macOS 64-bit builds.
Is this regression is large enough to revive the free_list for bound methods?
This performance regression in still present in 3.9.0a6
Results from Tools/scripts/var_access_benchmark.py:
Python 3.8.2 Python 3.9.0a6 ------------ \--------------read_boundmethod 27.7 ns 47.1 ns
Is this regression is large enough to revive the free_list
for bound methods?Yes!
read_boundmethod 27.7 ns 47.1 ns
Extract of Tools/scripts/var_access_benchmark.py:
def read_boundmethod(trials=trials, a=A()): for t in trials: a.m; a.m; a.m; a.m; a.m a.m; a.m; a.m; a.m; a.m a.m; a.m; a.m; a.m; a.m a.m; a.m; a.m; a.m; a.m a.m; a.m; a.m; a.m; a.m
Which kind of code pattern is impacted by this performance regression, apart this micro-benchmark? Do you notice a significant slowdown in pyperformance?
When pyperformance was run before the change was merged, there was no significant difference:
https://bugs.python.org/issue37340#msg348425In bpo-37340, you wrote that sorted(data, key=str.upper) is 70% slower. Would you mind to provide the benchmark?
I compared sorted(data, key=str.upper) performance between before the removal of the free list (parent of commit 3e54b57) and the current master branch, I get:
Mean +- std dev: [master] 167 us +- 4 us -> [cache] 179 us +- 7 us: 1.07x slower (+7%)
I cannot reproduce the "70% slowdown".
Raymond: How did you compile Python? What is your OS? How did you run your benchmark?
---
It's a quick & dirty benchmark run. I didn't use LTO+PGO and I didn't isolate my CPU. Maybe the difference of 12 ns is just caused by the noise of my coarse benchmark procedure.
I used commands:
# master
git checkout master
git clean -fdx
./configure && make && ./python -m venv env && env/bin/python -m pip install pyperf
env/bin/python -m pyperf timeit -s 'import random; data = [f"x{i}" for i in range(1000)]; random.shuffle(data)' 'sorted(data, key=str.upper)' -o ../master.json# before removal of the free list
git checkout 3e54b57^
git clean -fdx
./configure && make && ./python -m venv env && env/bin/python -m pip install pyperf
env/bin/python -m pyperf timeit -s 'import random; data = [f"x{i}" for i in range(1000)]; random.shuffle(data)' 'sorted(data, key=str.upper)' -o ../master.jsonThat's a Fedora 31 using GCC -O3 (GCC 9.3.1).
--
The commit before the removal of the free list is:
commit 76b6451 (HEAD)
Date: Fri Jul 26 03:30:33 2019 +0200Well, maybe I did a mistake.
By the way, in sorted(data, key=str.upper): str.upper is an unbound method, no?
$ ./python Python 3.9.0a6+ (heads/master:d9a43e20fa, Apr 28 2020, 23:50:37) >>> type(str.upper) <class 'method_descriptor'>
The performance regression became a bit worse in 3.11 and improved a bit again in 3.14 (maybe because a freelist was added again #128692, although I did not measure).
(fiftyone) c:\projects\tmp>uv run -p 3.8 -m timeit -r 11 -s "class A: pass" -s "A.m = lambda s: None" -s "a = A()" "a.m; a.m; a.m; a.m; a.m" 5000000 loops, best of 11: 62 nsec per loop (fiftyone) c:\projects\tmp>uv run -p 3.9 -m timeit -r 11 -s "class A: pass" -s "A.m = lambda s: None" -s "a = A()" "a.m; a.m; a.m; a.m; a.m" 5000000 loops, best of 11: 86.3 nsec per loop (fiftyone) c:\projects\tmp>uv run -p 3.10 -m timeit -r 11 -s "class A: pass" -s "A.m = lambda s: None" -s "a = A()" "a.m; a.m; a.m; a.m; a.m" 5000000 loops, best of 11: 84.7 nsec per loop (fiftyone) c:\projects\tmp>uv run -p 3.11 -m timeit -r 11 -s "class A: pass" -s "A.m = lambda s: None" -s "a = A()" "a.m; a.m; a.m; a.m; a.m" 2000000 loops, best of 11: 105 nsec per loop (fiftyone) c:\projects\tmp>uv run -p 3.12 -m timeit -r 11 -s "class A: pass" -s "A.m = lambda s: None" -s "a = A()" "a.m; a.m; a.m; a.m; a.m" 2000000 loops, best of 11: 107 nsec per loop (fiftyone) c:\projects\tmp>uv run -p 3.13 -m timeit -r 11 -s "class A: pass" -s "A.m = lambda s: None" -s "a = A()" "a.m; a.m; a.m; a.m; a.m" 2000000 loops, best of 11: 117 nsec per loop (fiftyone) c:\projects\tmp>uv run -p 3.14 -m timeit -r 11 -s "class A: pass" -s "A.m = lambda s: None" -s "a = A()" "a.m; a.m; a.m; a.m; a.m" 2000000 loops, best of 11: 103 nsec per loopFor normal attributes the result is different:
(fiftyone) c:\projects\tmp>uv run -p 3.8 -m timeit -r 11 -s "class A: pass" -s "A.m = 10" -s "a = A()" "a.m; a.m; a.m; a.m; a.m" 10000000 loops, best of 11: 37.6 nsec per loop (fiftyone) c:\projects\tmp>uv run -p 3.9 -m timeit -r 11 -s "class A: pass" -s "A.m = 10" -s "a = A()" "a.m; a.m; a.m; a.m; a.m" 10000000 loops, best of 11: 38.1 nsec per loop (fiftyone) c:\projects\tmp>uv run -p 3.10 -m timeit -r 11 -s "class A: pass" -s "A.m = 10" -s "a = A()" "a.m; a.m; a.m; a.m; a.m" 5000000 loops, best of 11: 42.3 nsec per loop (fiftyone) c:\projects\tmp>uv run -p 3.11 -m timeit -r 11 -s "class A: pass" -s "A.m = 10" -s "a = A()" "a.m; a.m; a.m; a.m; a.m" 5000000 loops, best of 11: 52 nsec per loop (fiftyone) c:\projects\tmp>uv run -p 3.12 -m timeit -r 11 -s "class A: pass" -s "A.m = 10" -s "a = A()" "a.m; a.m; a.m; a.m; a.m" 5000000 loops, best of 11: 49.5 nsec per loop (fiftyone) c:\projects\tmp>uv run -p 3.13 -m timeit -r 11 -s "class A: pass" -s "A.m = 10" -s "a = A()" "a.m; a.m; a.m; a.m; a.m" 10000000 loops, best of 11: 28.8 nsec per loop (fiftyone) c:\projects\tmp>uv run -p 3.14 -m timeit -r 11 -s "class A: pass" -s "A.m = 10" -s "a = A()" "a.m; a.m; a.m; a.m; a.m" 10000000 loops, best of 11: 24.7 nsec per loopI investigated a bit more:
- Part of the slowdown is due to immortal objects (pep 683), but this only explains a small part
- Freelists have been removed and added later. On PGO builds the effect is also minor
- The method access
a.mis indeed slower. There are some changes that made it slower (but these changes made other paths faster):
- Due to managed dicts we have one more pointer indirection
- The LOAD_ATTR AND LOAD_METHOD have been merged (Merge
LOAD_METHODback intoLOAD_ATTRfor better specialization #93429) increasing inline cache and some more handling
We can restore performance with a dedicated opcode. For a microbenchmark (as in the OP)
a.mis about 1.34x faster. This does require extra tier1 opcodes though (or tier2 only, but that pending on a possible jit pep).
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields: