Skip to content

[C API] Provide a replacement for removed private _Py_Identifier API, or consider reverting the removal #111484

Closed as not planned
Closed as not planned
[C API] Provide a replacement for removed private _Py_Identifier API, or consider reverting the removal#111484
Labels
topic-C-API

Description

@vstinner
vstinner
opened on Oct 30, 2023

In Python 3.13, the private _Py_Identifier API was removed: see PR #108593.

Replacements:

  • Use a literal bytes string: replace &PyId_abc with "abc", use "String" variant functions. Slower, but works on all Python versions. This solution can be used on code which is not performance sensitive.
  • Initialize a global or "static" variable to only create a Python str object once using PyUnicode_FromString(): less convenient, not compatible with multi-phase init API (PEP 489), not compatible with PEP 684 – A Per-Interpreter GIL.

If we want to add a public API for this, I would prefer to only add the bare minimum API, make it as small as possible:

  • Py_IDENTIFIER(name) macro.
  • PyUnicode_FromId(&PyId_name) function.
  • Py_Identifier structure with public members (needed by Py_IDENTIFIER() macro).
  • Maybe also: Py_static_string(name, "...") macro for strings which are not valid C identifiers.

It means that _PyObject_GetAttrId(obj, &PyId_abc) must be replaced with:

PyObject *name = PyUnicode_FromId(&Py_abc);
if (name == NULL) { ... handle error... }
attr = PyObject_GetAttr(obj, name);

Since functions like PyObject_GetAttr() don't accept NULL, so you cannot simply write:

attr = PyObject_GetAttr(obj, PyUnicode_FromId(&Py_abc));

See discussion:

Activity

  1. vstinner
    added
    topic-C-API
    on Oct 30, 2023

  2. encukou commented on Oct 30, 2023

    @encukou
    encukou
    on Oct 30, 2023
    Member

    Would these be interpreter-specific, or shared? Interned? Immortal?

  3. vstinner commented on Oct 30, 2023

    @vstinner
    vstinner
    on Oct 30, 2023
    MemberAuthor

    Would these be interpreter-specific, or shared? Interned? Immortal?

    If possible, it should work in all cases: multiple instances of an extension per interpreter and support multiple interpreters per process with one GIL per interpreter.

    Currently, _PyUnicode_FromId() interns strings, but interned strings are per interpreter which makes it problematic with Per-Interpreter GIL.

    Maybe it's easier to implement if strings are made immortal.

  4. vstinner commented on Dec 15, 2023

    @vstinner
    vstinner
    on Dec 15, 2023
    MemberAuthor

    The following private functions removed in Python 3.13 alpha1 were restored in Python 3.13 alpha2:

    • _PyObject_CallMethodId()
    • _PyObject_GetAttrId()
    • _PyUnicode_FromId()
    • _Py_IDENTIFIER()
    • _Py_static_string()
    • _Py_static_string_init()

  5. vstinner commented on Dec 20, 2023

    @vstinner
    vstinner
    on Dec 20, 2023
    MemberAuthor

    Since the private API is back and no one managed to propose a replacement so far, I suppose that we can keep this API a little bit longer until a replacement is designed (if needed).

  6. vstinner
    closed this as not plannedon Dec 20, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    topic-C-API

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions