Skip to content

Support the use of the managed pre-header in builtin classes. #95707

Description

@markshannon

Currently Py_TPFLAGS_MANAGED_DICT is an internal-only flag, in fact setting in third-party code is likely to lead to a crash.
We would like to expose it, and a weakref equivalent Py_TPFLAGS_MANAGED_WEAKREFS to allow builtin classes to take advantage of compact object layout.
Compact layout uses less memory, performs better and allow more robust subclassing. So everyone should be able to use it.

  • Fix crashes when Py_TPFLAGS_MANAGED_DICT is used
  • Add Py_TPFLAGS_MANAGED_WEAKREFS
  • Document how to use these flags, and the upgrade path from tp_dictoffset and tp_weakreflist

Activity

  1. added
    type-featureA feature request or enhancement
    performancePerformance or resource usage
    interpreter-core(Objects, Python, Grammar, and Parser dirs)
    3.12only security fixes
    on Aug 5, 2022
  2. self-assigned this
    on Aug 5, 2022
  3. added a commit that references this issue on Aug 15, 2022
  4. tiran commented on Aug 15, 2022

    @tiran
    Member

    The changeset broke wasm32-emscripten builds.

    $ ./Tools/wasm/wasm_build.py emscripten-node-dl-debug
    $ cd builddir/emscripten-node-dl-debug/
    $ $ make test TESTOPTS="test_capi"
    ...
    0:00:03 load avg: 3.51 [1/1/1] test_capi crashed (Exit code 1)
    warning: unsupported syscall: __syscall_prlimit64
    
    exiting due to exception: RuntimeError: null function or function signature mismatch,RuntimeError: null function or function signature mismatch
        at gc_collect_main (wasm://wasm/02f6208e:wasm-function[4421]:0x321118)
        at _PyObject_GC_Link (wasm://wasm/02f6208e:wasm-function[4431]:0x321c49)
        at _PyObject_GC_New (wasm://wasm/02f6208e:wasm-function[4432]:0x321d0b)
        at wrapperdescr_get (wasm://wasm/02f6208e:wasm-function[1036]:0x1aeaa2)
        at super_getattro (wasm://wasm/02f6208e:wasm-function[2415]:0x2157fc)
        at PyObject_GetAttr (wasm://wasm/02f6208e:wasm-function[2060]:0x1fbb08)
        at _PyEval_EvalFrameDefault (wasm://wasm/02f6208e:wasm-function[3281]:0x2a7e37)
        at _PyEval_Vector (wasm://wasm/02f6208e:wasm-function[3282]:0x2a9185)
        at _PyFunction_Vectorcall (wasm://wasm/02f6208e:wasm-function[870]:0x1a2a57)
        at _PyObject_FastCallDictTstate (wasm://wasm/02f6208e:wasm-function[857]:0x1a1f22)
    
    == Tests result: FAILURE ==
    
  5. tiran commented on Aug 15, 2022

    @tiran
    Member

    The clear function signature is incorrect. The PR defines it as static void heapmanaged_clear but it must be static int heapmanaged_clear.

    I'm also getting warnings on X86_64:

    comparison of integer expressions of different signedness: ‘Py_ssize_t’ {aka ‘long int’} and ‘long unsigned int’ [-Wsign-compare]
    
  6. added 2 commits that reference this issue on Aug 15, 2022
  7. added a commit that references this issue on Aug 15, 2022
  8. gvanrossum commented on May 23, 2023

    @gvanrossum
    Member

    Ping @markshannon -- this still is not documented (there are some mentions of the flags but no explanations or documentation for the upgrade path). This came up in GH-104795.

  9. erlend-aasland commented on Sep 15, 2023

    @erlend-aasland
    Contributor

    @gvanrossum, see also #107073. Seems like Py_TPFLAGS_MANAGED_DICT depends on private API.

  10. gvanrossum commented on Sep 15, 2023

    @gvanrossum
    Member

    It’s public enough unless Victor moves it. :-)

  11. erlend-aasland commented on Sep 18, 2023

    @erlend-aasland
    Contributor

    It’s public enough unless Victor moves it. :-)

    Yes, it's included in Python.h, but most people (both users and core devs) assume an underscore-prefixed API is a private API, not a public API. Especially non-documented underscore-prefixed APIs ;) So, I'm not sure it is "public enough". At least it should be documented.

  12. gvanrossum commented on Sep 18, 2023

    @gvanrossum
    Member

    We can't change 3.12 so please document it. And feelings around leading _ are all over the place.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

3.12only security fixes3.13only security fixesdocsDocumentation in the Doc dirinterpreter-core(Objects, Python, Grammar, and Parser dirs)performancePerformance or resource usagetype-featureA feature request or enhancement

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions