Repository navigation
Make the JIT thread-safe #133171
Description
Activity
- added a commit that references this issue
on May 2, 2025 I discussed this with @brandtbucher.
Our first milestone for JIT × free-threading will be running JIT without any optimizations.
To achieve this, we need to complete the items marked as Phase 1 in the original issue description.Once the phase 1 is done, we can CI for JITxFree-threading without any optimization.
Reacted by Petr Andreevcc @colesbury @mpage
A little bit curious, is this PR open to people claiming different targets?
A little bit curious, is this PR open to people claiming different targets?
Can you elaborate what is diffrent target?
Can you elaborate what is diffrent target?
My bad, here's misunderstanding here. I mean is issue open to people who want to claiming the target we list above.(not different target)
My bad, here's misunderstanding here. I mean is issue open to people who want to claiming the target we list above.(not different target)
If you want to tackle a specific issue from the list, feel free to work on it.
But first, PR #137800 should be merged. See the explanation here: link to comment.Reacted by Nadeshiko ManjuSome design decisions with Brandt: we decided not to create executors per thread, since we don’t want duplicated traces across threads. A simple lock around the logic when inserting executors should be fine. If this causes performance issues, we can reconsider moving it to per-thread, but let’s see how it goes for now.
The next target is to investigate why this line fails when uop_optimize is executed.
Assertion failed: (PyObject_IS_GC(op)), function op_from_block, file gc_free_threading.c, line 367. /bin/sh: line 1: 217 Abort trap: 6 ./python.exe -E -S -m sysconfig --generate-posix-varsFWIW, I've found some
PyStackRefproblem withsysconfig, which I'm still investigating.
There is minimal example:-> % cat issue_lib.py X = "TEXT" -> % cat issue.py import issue_lib def f(): s = "TEXT" return s f() -> % ./python issue.pyIf
issue_lib.pyis not compiled yet, there is no problem.
But withpycfile compiled there is aPyStackRefborrow issue: somePyStackRefis closed before borrowedPyStackRefwas closed.Edited: It seems there's no real problem in
PyStackRefborrowing, some issue occurs with immortal string.Reacted by Donghee Na and Sergey MiryanovI have a different idea for
_Py_uop_analyze_and_optimize:We could what (I believe) ZJIT in Ruby does. Ruby also has the concept of free-threading, where multiple ractors may execute at the same time.
Phase 1:
For the first and simplest phase, we could add a watcher/callback to thread creation. When multiple threads are created, we simply invalidate the executor as _CHECK_VALIDITY fails.This means single threaded code gets all the benefits of JIT in the future, while multi-threaded code will lose all the benefits. However, both can coexist at the same time in the same build. We will finally get JIT + FT in a limited form.
Phase 2:
If we detect multiple threads are running, we turn off the non-thread safe optimizations, and redo the trace with those optimizations off. If we detect only a single thread, we follow phase 1 and run till we get invalidated. This allows single threaded code to run faster, while multi-threaded code will run a little slower. Note that all optimizations I've proposed for the JIT in the past 6 months and in the future are FT safe (in theory. whether they are implemented in practice is different).Reacted by Donghee Na, Nadeshiko Manju, Mikhail Efimov and Chris EiblThe next target is to investigate why this line fails when uop_optimize is executed.
Assertion failed: (PyObject_IS_GC(op)), function op_from_block, file gc_free_threading.c, line 367. /bin/sh: line 1: 217 Abort trap: 6 ./python.exe -E -S -m sysconfig --generate-posix-varsI cannot repro on main anymore. I think it fixed itself?
I created an alternative design here #141594. The PR turns on the entire JIT including optimizer on FT.
Currently,
--enable-experimental-jitand--disable-gilcan be configured together. However, the behavior in this configuration is probably surprising, and not what the user intends: the JIT is built, but never actually used.I've been seeing a lot of people configuring these "bleeding edge" experimental builds now, and we even have one in CI (we've been using it to make sure the JIT build step isn't broken). Until the free-threaded build supports the JIT, we should start by erroring during the configure step for these builds in 3.14. I'll open a PR to do this soon.
For 3.15, the JIT should be updated to work on the free-threaded build. This shouldn't be done in a way that pessimizes the JIT in the default build... it will probably start with a bunch of
#ifdef Py_GIL_DISABLED/#ifndef Py_GIL_DISABLEDblocks, in order to at least get something working.This isn't something that I am planning on working on personally. This isn't a good beginner issue either. If there is somebody with free-threading experience interested in learning more about the JIT and willing to take on this project, then that would be great. I can't claim deep free-threading knowledge, but I'm willing to walk you through how the JIT works, help identify what the potential thread-safety issues may be, and review/debug any PRs.
In my understanding, here's a very high-level triage of the current situation:
JUMP_BACKWARD: We need to re-enable specialization for this instruction. This shouldn't be too hard, since we already have idioms for specialization on the free-threaded build.bytecodes.c: There are several instructions that are tier-two (JIT) only. These need to be audited for thread-safety._PyOptimizer_Optimize: Finding space for and inserting executors into the code object needs to be done in a thread-safe way, which probably will involve locking the code object such that only one part of it can be compiled at a time._PyExecutorObject: These are objects that manage JIT code. They are joined into a giant doubly-linked list, and are unlinked as they are invalidated or no longer needed. This list needs to be made thread-safe.translate_bytecode_to_trace: This shouldn't be too bad, since each thread has its own copy of the bytecode and inline caches (so we don't need to worry about concurrent mutations during tracing). Function inlining andforloop headers may need some attention._PyJIT_Compile: This is probably fine, actually, which is a relief. The entire machine-code backend is stateless and self-contained. As long asmmap/mprotect/munmap/etc. are threadsafe, then we're okay. The generated code is immutable, and can be shared between threads without issue._Py_uop_analyze_and_optimizeis going to be where most of our time and ongoing effort is spent. Thankfully it's all optional (in theory), so we may just be able to turn it all off and selectively turn things back on. It consists of three passes:remove_globals: This has the potential to be annoying, but I think it may not be too bad (as long as watchers are already thread-safe?).optimize_uops: This is going to be the long tail, I think. This one is different, since we don't only care if theoptimize_uopsitself is thread-safe (easy), but also if our optimizations will be thread-safe at runtime (hard). We'll probably just end up#ifdef'ing out many of the bodies inoptimizer_bytecodes.cand figuring out how to turn them back on (if at all).remove_unneeded_uops: This already looks fine to me.If stuff is missing, please add it to the list.
It may even be reasonable to stop the world for JIT compilation initially (just to get something working), then later introduce finer-grained locking as needed.
Linked PRs
--disable-giland--enable-experimental-jit(for now) #133179