Repository navigation
Do not track immutable tuples in PyTuple_Pack #139389
Description
Feature or enhancement
Proposal:
When we use PyTuple_Pack all objects already well constructed. If we know that they immutable we can skip tracking it in GC, because GC will untrack them eventually.
I have a PR ready and benchmark results:
Geometric mean: 1.01x faster (Win11 x64, 11th Gen Intel(R) Core(TM) i5-11600K @ 3.90GHz, 48d0d0d)
All benchmarks:
+--------------------------+----------+------------------------+
| Benchmark | main | tuples |
+==========================+==========+========================+
| async_generators | 435 ms | 430 ms: 1.01x faster |
+--------------------------+----------+------------------------+
| asyncio_tcp | 750 ms | 756 ms: 1.01x slower |
+--------------------------+----------+------------------------+
| asyncio_tcp_ssl | 1.91 sec | 1.92 sec: 1.01x slower |
+--------------------------+----------+------------------------+
| comprehensions | 22.1 us | 21.8 us: 1.01x faster |
+--------------------------+----------+------------------------+
| bench_mp_pool | 104 ms | 103 ms: 1.01x faster |
+--------------------------+----------+------------------------+
| bench_thread_pool | 1.29 ms | 1.27 ms: 1.01x faster |
+--------------------------+----------+------------------------+
| coroutines | 28.2 ms | 27.7 ms: 1.02x faster |
+--------------------------+----------+------------------------+
| coverage | 88.5 ms | 86.3 ms: 1.02x faster |
+--------------------------+----------+------------------------+
| crypto_pyaes | 90.1 ms | 86.7 ms: 1.04x faster |
+--------------------------+----------+------------------------+
| deepcopy | 310 us | 307 us: 1.01x faster |
+--------------------------+----------+------------------------+
| deepcopy_memo | 36.4 us | 36.1 us: 1.01x faster |
+--------------------------+----------+------------------------+
| deltablue | 5.19 ms | 4.85 ms: 1.07x faster |
+--------------------------+----------+------------------------+
| django_template | 45.5 ms | 45.8 ms: 1.01x slower |
+--------------------------+----------+------------------------+
| docutils | 2.47 sec | 2.45 sec: 1.01x faster |
+--------------------------+----------+------------------------+
| dulwich_log | 86.2 ms | 86.9 ms: 1.01x slower |
+--------------------------+----------+------------------------+
| fannkuch | 449 ms | 441 ms: 1.02x faster |
+--------------------------+----------+------------------------+
| float | 85.3 ms | 82.5 ms: 1.03x faster |
+--------------------------+----------+------------------------+
| create_gc_cycles | 1.17 ms | 1.17 ms: 1.01x faster |
+--------------------------+----------+------------------------+
| gc_traversal | 2.97 ms | 2.88 ms: 1.03x faster |
+--------------------------+----------+------------------------+
| generators | 43.0 ms | 41.6 ms: 1.03x faster |
+--------------------------+----------+------------------------+
| genshi_text | 28.9 ms | 28.7 ms: 1.01x faster |
+--------------------------+----------+------------------------+
| go | 160 ms | 153 ms: 1.04x faster |
+--------------------------+----------+------------------------+
| hexiom | 8.39 ms | 8.13 ms: 1.03x faster |
+--------------------------+----------+------------------------+
| json_dumps | 8.62 ms | 8.69 ms: 1.01x slower |
+--------------------------+----------+------------------------+
| logging_format | 12.5 us | 12.2 us: 1.02x faster |
+--------------------------+----------+------------------------+
| logging_silent | 139 ns | 140 ns: 1.01x slower |
+--------------------------+----------+------------------------+
| logging_simple | 11.3 us | 11.1 us: 1.01x faster |
+--------------------------+----------+------------------------+
| mako | 14.2 ms | 14.4 ms: 1.01x slower |
+--------------------------+----------+------------------------+
| mdp | 1.47 sec | 1.50 sec: 1.02x slower |
+--------------------------+----------+------------------------+
| meteor_contest | 104 ms | 102 ms: 1.02x faster |
+--------------------------+----------+------------------------+
| nbody | 114 ms | 113 ms: 1.01x faster |
+--------------------------+----------+------------------------+
| pickle_pure_python | 439 us | 436 us: 1.01x faster |
+--------------------------+----------+------------------------+
| pprint_safe_repr | 953 ms | 916 ms: 1.04x faster |
+--------------------------+----------+------------------------+
| pprint_pformat | 1.95 sec | 1.88 sec: 1.04x faster |
+--------------------------+----------+------------------------+
| pyflate | 506 ms | 492 ms: 1.03x faster |
+--------------------------+----------+------------------------+
| python_startup | 28.5 ms | 27.4 ms: 1.04x faster |
+--------------------------+----------+------------------------+
| python_startup_no_site | 23.2 ms | 22.2 ms: 1.05x faster |
+--------------------------+----------+------------------------+
| raytrace | 361 ms | 345 ms: 1.05x faster |
+--------------------------+----------+------------------------+
| regex_compile | 146 ms | 146 ms: 1.01x faster |
+--------------------------+----------+------------------------+
| regex_effbot | 2.03 ms | 2.02 ms: 1.01x faster |
+--------------------------+----------+------------------------+
| regex_v8 | 23.9 ms | 22.7 ms: 1.06x faster |
+--------------------------+----------+------------------------+
| richards | 66.1 ms | 59.9 ms: 1.10x faster |
+--------------------------+----------+------------------------+
| richards_super | 71.6 ms | 68.7 ms: 1.04x faster |
+--------------------------+----------+------------------------+
| scimark_fft | 300 ms | 294 ms: 1.02x faster |
+--------------------------+----------+------------------------+
| scimark_lu | 135 ms | 131 ms: 1.03x faster |
+--------------------------+----------+------------------------+
| scimark_monte_carlo | 83.3 ms | 82.4 ms: 1.01x faster |
+--------------------------+----------+------------------------+
| scimark_sor | 157 ms | 150 ms: 1.05x faster |
+--------------------------+----------+------------------------+
| scimark_sparse_mat_mult | 4.27 ms | 4.35 ms: 1.02x slower |
+--------------------------+----------+------------------------+
| spectral_norm | 122 ms | 118 ms: 1.03x faster |
+--------------------------+----------+------------------------+
| sqlglot_optimize | 60.7 ms | 60.9 ms: 1.00x slower |
+--------------------------+----------+------------------------+
| sympy_expand | 501 ms | 503 ms: 1.00x slower |
+--------------------------+----------+------------------------+
| sympy_sum | 143 ms | 144 ms: 1.01x slower |
+--------------------------+----------+------------------------+
| sympy_str | 287 ms | 292 ms: 1.02x slower |
+--------------------------+----------+------------------------+
| telco | 7.26 ms | 7.33 ms: 1.01x slower |
+--------------------------+----------+------------------------+
| tomli_loads | 2.23 sec | 2.25 sec: 1.01x slower |
+--------------------------+----------+------------------------+
| typing_runtime_protocols | 189 us | 185 us: 1.02x faster |
+--------------------------+----------+------------------------+
| unpack_sequence | 65.4 ns | 68.7 ns: 1.05x slower |
+--------------------------+----------+------------------------+
| unpickle | 13.9 us | 14.1 us: 1.01x slower |
+--------------------------+----------+------------------------+
| unpickle_pure_python | 303 us | 300 us: 1.01x faster |
+--------------------------+----------+------------------------+
| xml_etree_parse | 130 ms | 130 ms: 1.01x slower |
+--------------------------+----------+------------------------+
| xml_etree_iterparse | 107 ms | 108 ms: 1.01x slower |
+--------------------------+----------+------------------------+
| xml_etree_process | 79.2 ms | 78.6 ms: 1.01x faster |
+--------------------------+----------+------------------------+
| Geometric mean | (ref) | 1.01x faster |
+--------------------------+----------+------------------------+
Benchmark hidden because not significant (20): 2to3, chaos, deepcopy_reduce, genshi_xml, html5lib, json_loads, nqueens, pathlib, pickle, pickle_dict, pickle_list, pidigits, regex_dna, sqlglot_normalize, sqlglot_parse, sqlglot_transpile, sqlite_synth, sympy_integrate, unpickle_list, xml_etree_generate
It doesn't hurt performance, but can decrease number of objects in GC to check and untrack.
Has this already been discussed elsewhere?
This is a minor feature, which does not need previous discussion elsewhere
Links to previous discussion of this feature:
No response
Activity
picnixz commentedon Sep 28, 2025
1.01 faster is not a realistic improvement IMO. In general, we want > 10% improvements. We should also have benchmarks on Linux machines with clang/gcc instead (and be sure that it's also using PGO+LTO).
But is it really important if it doesn't change the performance? by how much are we changing the number of items to track? is it a lot? if not, I don't think it's necessary.
sergey-miryanov commentedon Sep 28, 2025
I will try to calculate and compare.
Unfortunately, I don't have such machine :(
picnixz commentedon Sep 28, 2025
Also, could we have some micro-benchmark as well to check by how much
PyTuple_Packitself is affected? TiA.sergey-miryanov commentedon Sep 28, 2025
Got it.
Fidget-Spinner commentedon Sep 28, 2025
1.01x faster is the geometric mean, not a single benchmark. It is completely realistic for pyperformance. For context, one good specialization in the specializing interpreter gives 2-3% geomean on pyperformance. 10% is very unrealistic. The entire specializing interpreter gave a 25% speedup in its first iteration.
Edit: I removed NOT in call caps and replaced it with not, because I didn't want to seem shouty. Sorry if I did sound like it unintentionally, meant to use bold not caps!
picnixz commentedon Sep 28, 2025
Yes, but Victor usually asks for 10% improvements even for the geometric mean.
Ok, then in this case I'm fine with lowering the threshold. However, should we also consider losing 1% as important? For instance, unpacking sequences becomes 5% slower and unpickling became 1% slower as well, though other benchmarks seem fine.
I would still be interested in micro-benchmarks though.
Fidget-Spinner commentedon Sep 28, 2025
No my understanding is that he asks for 10% of geometric mean of microbenchmarks, which makes perfect sense. 10% on pyperformance geometric mean is different than 10% of microbenchmark geometric mean.
picnixz commentedon Sep 28, 2025
Sorry I wasn't clear here. I wanted to explain why I thought that the 10% threshold should have applied, but I wasn't aware that we only reached 1-2% max improvements on macro benchmarks in general. So if you're ok with a 1% improvements overall but with some specific tasks being slower, then I'm also ok.
Fidget-Spinner commentedon Sep 28, 2025
To be fair, I'm not too sure 1% is a good threshold on normal pyperformance nowadays too. Some benchmarks are noisy and 1% is within the range of noise for some systems.
serhiy-storchaka commentedon Sep 29, 2025
1% is within the noise range. I expect the difference (in one direction or another) to be several orders of magnitude smaller. Tuples creation is a small part of any code, and
PyTuple_Pack()is a tiny part of it. The GC will untrack such tuples first time it encounter them, so roughly the same code will be executed in any case.If there is a noticeable difference, there should be a microbenchmark that shows a significant (tens of percent) difference. Then you should show that such a case can actually happen in non-trivial amount of user code.
sergey-miryanov commentedon Sep 29, 2025
There are numbers of how reduced count of untracked tuples in
untrack_tuples:cnt_main is for 48d0d0d
cnt_pack is for fbb7342
cnt_all is for 04f0f66
pack_ratio = 100.0 * (cnt_main - cnt_pack) / cnt_mainpack_all = 100.0 * (cnt_main - cnt_all) / cnt_mainMost of the work done in the generation 1, for
packit is about 1% reduce count, forallit varies from 5% to 8%.Numbers are from pyperfomance benchmarks. Instrumentation for main made like this - https://github.com/python/cpython/pull/139390/files#diff-1c580282bd10a8157cc81dd4a4658d4bb47f75ea476cd433bc7435913b33eb77R137
markshannon commentedon Oct 9, 2025
Every tuple is created, but not every tuple is seen by the GC; many are dealloc before GC gets to see them.
For those tuples this adds overhead for no gain.
Do you any numbers for the ratio of tuples that reach GC for general applications?
Even if that ratio is high, why is it cheaper to check for tracking during construction than to check during GC?
markshannon commentedon Oct 9, 2025
@picnixz 1% is a fantastic improvement for a single, small PR.
70 such PRs that each created a 1% speedup would double the speed of CPython.
If all of my PRs had sped up CPython by 1%, it would be over a 1000 times than it is now 🙂
The challenge is determining whether the speedup is real, and that requires a more sophisticated approach than just running the benchmarks once.
sergey-miryanov commentedon Oct 9, 2025
Yeah, I agree. After careful consideration, I think that most of tuples die before garbage collection.
I have plans to collect total count of tuples that allocated and deallocated before GC.
Also, I plan to add microbenchmarks to measure impact of this micro-optimisation on PyTuple_* methods. My main job takes too much time :)
sergey-miryanov commentedon Oct 19, 2025
Here are some numbers for tracked and untracked tuples in GC. I previously collected the data and just processed it now (data for
pyperformance, code for counting total untracked tuples - ff81eb2, for tracked tuples - a79a283).Below is a plot of the total number of created, tracked, and untracked tuples (log scale):
As can be seen, the total number of tracked tuples is about 10-30% of the total number of created tuples. Total tracked tuples are tuples that have been seen by GC. Untracked tuples are those that were untracked by
untrack_tuples.The percentage of the tracked and untracked tuples to the total number:
About 2% of the total number of created tuples are kept in GC:
tracked_untracked_dataset.csv