Repository navigation
Increase fraction of code executed by tier 2. #118093
Description
According to stats and profiling only about 40% of bytecode instructions are executed by tier 2 and the remaining 60% by tier 1.
We the expected improvements to the JIT and tier 2 optimizer we expect tier 2 (with JIT) to have a significantly faster than tier 1.
It therefore make sense to get the fraction of instructions executed by tier 2 up from 40% to nearer 90%.
To do that we need to:
- Enter tier 2 at some function entry points as well as back edges. #118094
- Stay in tier 2 when exiting executors in most cases Trace stitching #112354
- Increase the number of micro-ops that we can handle in tier 2 #118095
Linked PRs
- GH-118093: Add tier two support to several instructions #121884
- GH-118093: Remove invalidated executors from side exits #121885
- GH-118093: Better handling of short and mid-loop traces #122252
- GH-118093: Add tier two support for
BINARY_OP_INPLACE_ADD_UNICODE#122253 - GH-118093: Add tier two support for
LOAD_ATTR_PROPERTY#122283 - GH-118093: Handle some polymorphism before requiring progress in tier two #122843
- GH-118093: Turn some
DEOPT_IFs intoEXIT_IFs #122998 - GH-118093: Specialize
CALL_KW#123006 - GH-118093: Specialize
CALL_FUNCTION_EX#123034 - GH-118093: Make
CALL_ALLOC_AND_ENTER_INITsuitable for tier 2. #123140 - GH-118093: Specialize calls to non-vectorcall classes as
CALL_NON_PY_GENERAL#123212 - GH-118093: Fix off-by-one errors in tier-up thresholds #124447
- GH-118093: Don't lose confidence when tracing through 100% biased branches #124813
Activity
brandtbucher commentedon Jul 17, 2024
By my count we're currently hovering around 54% of code executed in tier two (our benchmarks run about 266 billion tier one instructions on normal builds and 122 billion instructions on JIT builds). I've identified a few strategies for improving this (based on stats and tracing through how we execute a bunch of the benchmarks) and will start landing PRs soon. No magic bullets here, just chipping away at things:
CALL_KWandCALL_FUNCTION_EX. We probably don't need to do anything too crazy here early on in terms of optimizing calls... we can start by just adding a handful of specializations that allow us to trace through them instead of ending the trace.CALL_LIST_APPEND,IMPORT_NAME,LOAD_NAME,BUILD_SET,SEND_GEN, andIMPORT_FROM), and some are harder (LOAD_ATTR_PROPERTY,BINARY_SUBSCR_GETITEM,CALL_ALLOC_AND_ENTER_INIT,RAISE_VARARGS, andBINARY_OP_INPLACE_ADD_UNICODE).SEND_ASYNC_GEN_ANEXT, using a similar shim frame asCALL_ALLOC_AND_ENTER_INIT.DYNAMIC_EXIT) or statically (by using the current stack when projecting to infer callers). A more radical idea could be to start recording traces instead of projecting them. That simplifies a lot of things, but is a big rewrite of some pretty core stuff.DEOPT_IFs intoEXIT_IFs for better handling of control flow and polymorphism._FOR_ITER_TIER_TWOis an obvious candidate here, but there are others, too.My motivation for this is to make JIT improvements more pronounced. We currently spend less than 10% of our time in the JIT (vs ~25% of our time in tier one), which means that we need to improve the performance of JIT code by over 10% just to see a 1% improvement on the benchmarks. My (probably ambitious) goal is to get the fraction of code executed in tier two up to around 80% (meaning, in the neighborhood of 25%-30% of the total time spent running the benchmarks) in the next couple of weeks. Then the improvements can be easier to measure and iterate on.
brandtbucher commentedon Jul 17, 2024
It's also worth noting that our stats are currently broken on benchmarks that use C extensions or spawn subprocesses. So the actual numbers may vary a bit right now, but probably aren't heavily biased one way or another.
GH-118093: Add tier two support to several instructions (GH-121884)
GH-118093: Remove invalidated executors from side exits (GH-121885)
pythonGH-118093: Remove invalidated executors from side exits (python…
pythonGH-118093: Remove invalidated executors from side exits (python…
GH-118093: Add tier two support for LOAD_ATTR_PROPERTY (GH-122283)
GH-118093: Add tier two support for BINARY_OP_INPLACE_ADD_UNICODE (GH…
GH-118093: Improve handling of short and mid-loop traces (GH-122252)
GH-118093: Handle some polymorphism before requiring progress in tier…
GH-118093: Turn some DEOPT_IFs into EXIT_IFs (GH-122998)
GH-118093: Specialize `CALL_KW` (GH-123006)
pythonGH-118093: Specialize `CALL_KW` (pythonGH-123006)
GH-118093: Make `CALL_ALLOC_AND_ENTER_INIT` suitable for tier 2. (GH-…
pythonGH-118093: Handle some polymorphism before requiring progress i…
pythonGH-118093: Turn some DEOPT_IFs into EXIT_IFs (pythonGH-122998)
pythonGH-118093: Specialize `CALL_KW` (pythonGH-123006)
pythonGH-118093: Make `CALL_ALLOC_AND_ENTER_INIT` suitable for tier 2. (
GH-118093: Specialize calls to non-vectorcall classes as `CALL_NON_PY…
GH-118093: Fix off-by-one errors in tier-up thresholds (GH-124447)
GH-118093: Don't lose confidence when tracing through 100% biased bra…
markshannon commentedon Mar 10, 2026
There are still improvements that can be made, but those can be done in separate, smaller issues