Skip to content

[C API] Add PyObject_GetDict() function #139852

Description

@vstinner

Feature or enhancement

There is a private _PyObject_GetDictPtr() function which is used by 30 projects of the PyPI top 15,000 projects:

  • accesscontrol (7.2)
  • acquisition (6.1)
  • backports_datetime_fromisoformat (2.0.3)
  • casadi (3.7.2)
  • catboost (1.2.8)
  • cvxpy (1.7.2)
  • cvxpy_base (1.7.2)
  • cython (3.1.4)
  • extensionclass (6.0)
  • frozendict (2.4.6)
  • gamsapi (51.1.0)
  • gdal (3.11.4)
  • gevent (25.9.1)
  • guppy3 (3.1.5)
  • htseq (2.0.9)
  • hunter (3.9.0)
  • ihm (2.7)
  • jpype1 (1.6.0)
  • m2crypto (0.45.1)
  • mecab-ko (1.0.1)
  • mecab_python3 (1.0.10)
  • mupdf (1.21.1.20230112.1504)
  • passagemath_standard_no_symbolics (10.6.26)
  • persistence (5.1)
  • persistent (6.1.1)
  • pybind11 (3.0.1)
  • pygdal (3.6.4.11)
  • pygraphviz (1.14)
  • python_crfsuite (0.9.11)
  • sentencepiece (0.2.1)

I propose to make the function public with a different API to no longer ignore errors silently.

UPDATE: I now propose adding PyObject_GetDict() (PyObject **dict parameter), instead of PyObject_GetDictPtr() (PyObject ***dict_ptr parameter).

Linked PRs

Activity

  1. added a commit that references this issue on Oct 9, 2025
  2. markshannon commented on Oct 10, 2025

    @markshannon
    Member

    I don't think this is a good idea.

    There is reason this function is private. Using this function is potentially dangerous, the name is misleading and it is may be perpetuating a misunderstanding of how Python objects are implemented.

    It is dangerous, because assigning directly to the pointer returned by this function could result in the object and its __dict__ getting out of sync.
    Its name is misleading as it suggests that it is getting a pointer, without side effects, and that pointer can be written to.
    It also perpetuates the idea that all objects have a dictionary (at the C level), and that accessing that dictionary is cheap.

    Just because something is widely used does not mean we should make it public. We should be helping extension authors to write good code, not perpetuating their mistakes.

    Why are C extensions using this method anyway? If they are using it just to get the object's dictionary for legitimate reasons, then we should add a PyObject_GetDict function for that purpose.

    Also, we should probably change the docs for PyObject_GenericGetDict to make it clear that it is for implementing the __dict__ slot, not to be called as an API.

  3. kumaraditya303 commented on Oct 10, 2025

    @kumaraditya303
    Contributor

    This function is also broken on free-threading because free-threading stores and reads dict pointer atomically, it is not safe to read the dict pointer directly as returned by this function.

  4. vstinner commented on Oct 10, 2025

    @vstinner
    MemberAuthor

    Why are C extensions using this method anyway? If they are using it just to get the object's dictionary for legitimate reasons, then we should add a PyObject_GetDict function for that purpose.

    For various reasons, see for example #139854 (comment). Should we add a PyObject_SetDict() function as well?

  5. encukou commented on Oct 10, 2025

    @encukou
    Member

    Should we add a PyObject_SetDict() function as well?

    Not with the current evidence. I'd be interested to know if SWIG_PYTHON_SLOW_GETSET_THIS is helpful nowadays -- ideally PyObject_SetAttr would be all they need.

  6. da-woods commented on Oct 11, 2025

    @da-woods
    Contributor

    One of Cython's uses of this I think is duplicating PyType_Lookup so could probably go away if you provided that.

    I think in general PyObject_GetDict would be more useful than GetDictPtr.

    At least some of the time I'm using the dict rather than GetAttr/SetAttr because I can do PyDict_SetDefault and I know it'll be thread safe. That's very useful for lazy initialization, and is something you can't easily replicate without the dict.

  7. vstinner commented on Oct 11, 2025

    @vstinner
    MemberAuthor

    One of Cython's uses of this I think is duplicating PyType_Lookup so could probably go away if you provided that.

    I created #139847 to add PyType_Lookup() function.

  8. added 2 commits that reference this issue on Oct 11, 2025
  9. vstinner commented on Oct 11, 2025

    @vstinner
    MemberAuthor

    I think in general PyObject_GetDict would be more useful than GetDictPtr.

    Ok, I created #139968 to add PyObject_GetDict() function. It's similar to PyObject_GenericGetDict() but has a different API and doesn't raise an exception if the __dict__ doesn't exist.

  10. changed the title [-][C API] Add PyObject_GetDictPtr() function[/-] [+][C API] Add PyObject_GetDict() function[/+] on Oct 11, 2025
  11. vstinner commented on Nov 17, 2025

    @vstinner
    MemberAuthor

    I created capi-workgroup/decisions#82 for the C API Working Group.

    And the decision is to not add the PyObject_GetDict() function. So I close the issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions