Repository navigation
Remove the _PyOptimizer APIs #126599
Description
We should rip the _PyOptimizer API out, and just keep all of the code that does that actual optimizing.
The API itself is unstable, constantly changing, poorly-defined, and undocumented. There's a bunch of infrastructure required just to test the API (not the actual optimizations we perform), and it introduces indirection and artificial boundaries into some pretty performance-sensitive stuff. Nobody is using it that we're aware of, nobody we've talked to is planning on using it, and frankly we don't want anyone to start using it. So let's remove it.
Activity
xuantengh commentedon Nov 9, 2024
Hi can I take and try this issue?
rruuaanng commentedon Nov 9, 2024
Don't rush, please wait for the discussion result :)
brandtbucher commentedon Nov 10, 2024
This has already been discussed quite a bit offline with the relevant parties. I think we can probably move ahead with it.
It's a pretty big project, but help is welcome if you really feel confident tackling it. It's not just deleting things, but also refactoring the code to remove the current API boundary between the interpreter and the optimizer. I'm happy to help out if you get stuck or need any advice, too.
rruuaanng commentedon Nov 11, 2024
@xuantengh Would you like to challenge?
If you don't have the confidence to give it to me, I will finish it when I have time.
xuantengh commentedon Nov 11, 2024
Hi I would like to give it a try.
xuantengh commentedon Nov 11, 2024
Hi @brandtbucher, after some preliminary investigation, this issue aims to remove the following 3 internal C APIs:
_PyOptimizer_NewCounter_PyOptimizer_NewUOpOptimizer_PyOptimizer_Optimizewhere the first two are only used for tests (
_testinternalcapi.c), and it's trival to rip them. For the last, it's used in instructionJUMP_BACKWARDas well as tier2 OPs_EXIT_TRACEand_DYNAMIC_EXIT. So we need to refactor the code to let these instruction/OPs directly conduct the optimizations?Please correct me if something wrong or missing.
rruuaanng commentedon Nov 14, 2024
@xuantengh Maybe you should submit a PR :)
GH-126599: Remove the "counter" optimizer/executor (GH-126853)
brandtbucher commentedon Jan 16, 2025
Thanks for getting the ball rolling with the counter optimizer/executor removal! I'll go ahead and rip out the rest.
GH-126599: Remove the PyOptimizer API (GH-129194)