Skip to content

Pluggable optimizer API #104584

Closed
Closed
Pluggable optimizer API#104584
Labels
3.13only security fixesinterpreter-core(Objects, Python, Grammar, and Parser dirs)performancePerformance or resource usage

Description

@markshannon
markshannon
opened on May 17, 2023

We need an API for optimizers to be plugged in to CPython.

The proposed model is that of client server, where the VM is the client and the optimizer is the server.
The optimizer registers with the VM, then VM calls the optimizer when hotspots are detected.

The API:

type struct {
    OBJECT_HEADER;
    _PyInterpreterFrame *(*execute)(PyExecutorObject *self, _PyInterpreterFrame *frame, PyObject **stack_pointer);
    /* Data needed by the executor goes here, but is opaque to the VM */
} PyExecutorObject;

/* This would be nicer as an enum, but C doesn't define the size of enums */
#define PY_OPTIMIZE_FUNCTION_ENTRY 1
#define PY_OPTIMIZE_RESUME_AFTER_YIELD 2
#define PY_OPTIMIZE_BACK_EDGE 4
typedef uint32_t PyOptimizerCapabilities;

type struct {
    OBJECT_HEADER;
    PyExecutorObject *(*compile)(PyOptimizerObject* self, PyCodeObject *code, int offset);
    PyOptimizerCapabilities capabilities;
    float optimization_cost;
    float run_cost;
    /* Data needed by the compiler goes here, but is opaque to the VM */
} PyOptimizerObject;

void _Py_Executor_Replace(PyCodeObject *code, int offset, PyExecutorObject *executor);

int _Py_Optimizer_Register(PyOptimizerObject* optimizer);

The semantics of a PyExecutorObject is that upon return from its execute function, the VM state will have advanced N instructions. Where N is a non-negative integer.

Full discussion here: faster-cpython/ideas#380

This is not a replacement for PEP 523. That will need a PEP. We should get this working first, before we consider replacing PEP 523.

Linked PRs

Activity

  1. markshannon
    added
    performancePerformance or resource usage
    3.13only security fixes
    on May 17, 2023

  2. markshannon commented on May 18, 2023

    @markshannon
    markshannon
    on May 18, 2023
    MemberAuthor

    Note that the above API is just the initial version to support our work on speeding up Python 3.13.
    It will probably need to be extended to support PyTorch Dynamo and other users of PEP 523 that cannot use PEP 669, but that is for another issue.

  3. markshannon
    added 2 commits that reference this issue on Jun 2, 2023

    GH-104584: Plugin optimizer API (GH-105100)

    4bfa01b

    GH-104584: Allow optimizers to opt out of optimizing. (GH-105244)

    e8ecb9e
  4. markshannon
    added a commit that references this issue on Jun 19, 2023

    GH-104584: Assorted fixes for the optimizer API. (GH-105683)

    5816199
  5. gvanrossum
    added 4 commits that reference this issue on Jun 27, 2023

    gh-104584: Baby steps towards generating and executing traces (#105924)

    51fc725

    gh-104584: Add #line directives to executor_cases.c.h (#106126)

    0762775

    gh-104584: Fix assert in DEOPT macro -- should fix buildbot (#106131)

    bb578a0

    gh-104584: Change DEOPT_IF in uops executor (#106146)

    6b5166f
  6. vstinner
    added 2 commits that reference this issue on Jun 28, 2023

    pythonGH-104584: Fix test_capi.test_counter_optimizer() when run twice

    24228b3

    GH-104584: Fix test_capi.test_counter_optimizer() when run twice (#10…

    adaacf2
  7. gvanrossum
    added a commit that references this issue on Jun 28, 2023

    gh-104584: Emit macro expansions to opcode_metadata.h (#106163)

    1173143
  8. vstinner
    added 2 commits that reference this issue on Jun 30, 2023

    pythonGH-104584: Add Include/cpython/optimizer.h to Makefile.pre.in

    df1f5e6

    GH-104584: Add Include/cpython/optimizer.h to Makefile.pre.in (#106277)

    319de0b

  9. 9 remaining items

  10. brandtbucher
    added a commit that references this issue on Jul 20, 2023

    GH-104584: Miscellaneous fixes for -Xuops (GH-106908)

    214a25d
  11. brandtbucher
    added a commit that references this issue on Jul 31, 2023

    GH-104584: Fix incorrect uoperands (GH-107513)

    5e584eb
  12. brandtbucher
    added 2 commits that reference this issue on Sep 6, 2023

    GH-104584: Restore frame->stacktop on optimizer error (GH-108953)

    6971e40

    GH-104584: Don't call executors from JUMP_BACKWARD (GH-109347)

    6c13e13
  13. brandtbucher
    added a commit that references this issue on Oct 10, 2023

    GH-104584: Fix refleak when tracing through calls (GH-110593)

    13380da
  14. iritkatriel
    added
    interpreter-core(Objects, Python, Grammar, and Parser dirs)
    on Nov 27, 2023

  15. encukou commented on Jun 20, 2024

    @encukou
    encukou
    on Jun 20, 2024
    Member

    Is documenting these planned for 3.13?

  16. markshannon commented on Jun 20, 2024

    @markshannon
    markshannon
    on Jun 20, 2024
    MemberAuthor

    Since we are removing them in 3.14, probably not.

  17. vstinner commented on Jun 20, 2024

    @vstinner
    vstinner
    on Jun 20, 2024
    Member

    Since we are removing them in 3.14, probably not.

    If possible, I would like to backport #120643 to Python 3.13, to solve a C99 compatibility issue.

  18. Glyphack
    added a commit that references this issue on Sep 2, 2024

    pythonGH-104584: Fix refleak when tracing through calls (pythonGH-110593

    9308192

  19. erlend-aasland commented on Oct 13, 2024

    @erlend-aasland
    erlend-aasland
    on Oct 13, 2024
    Contributor

    A bunch of PRs were merged; a quick glance at the PR list suggests that this issue can be closed as completed. Are there further work to be done?

  20. brandtbucher commented on Nov 7, 2024

    @brandtbucher
    brandtbucher
    on Nov 7, 2024
    Member

    We discussed the optimizer API offline. In short: we should rip the API itself out, and just keep all of the code that does that actual optimizing.

    The API itself is constantly changing, poorly-defined, and undocumented. There's a bunch of infrastructure required just to test the API (not the actual optimizations we perform), and it introduces indirection and artificial boundaries into some pretty performance-sensitive stuff. Nobody it using it that we're aware of, nobody we've talked to is planning on using it, and frankly we don't want anyone to start using it. So let's remove it.

  21. vstinner commented on Nov 8, 2024

    @vstinner
    vstinner
    on Nov 8, 2024
    Member

    Nitpick: I suggest opening a new issue to remove it.

  22. brandtbucher commented on Nov 8, 2024

    @brandtbucher
    brandtbucher
    on Nov 8, 2024
    Member

    GH-126599

  23. brandtbucher
    closed this as completedon Nov 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    3.13only security fixesinterpreter-core(Objects, Python, Grammar, and Parser dirs)performancePerformance or resource usage

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions