Skip to content

Deprecate immortal interned strings: PyUnicode_InternImmortal() #85858

Closed
Closed
Deprecate immortal interned strings: PyUnicode_InternImmortal()#85858
Labels
3.10 (EOL)end of lifeinterpreter-core(Objects, Python, Grammar, and Parser dirs)

Description

@vstinner
vstinner
opened on Sep 2, 2020
BPO 41692
Nosy @vstinner, @methane, @serhiy-storchaka, @corona10, @shihai1991
PRs
  • bpo-41692: Deprecate PyUnicode_InternImmortal() #22486
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = <Date 2020-10-02.12:49:46.896>
created_at = <Date 2020-09-02.13:59:25.913>
labels = ['interpreter-core', '3.10']
title = 'Deprecate immortal interned strings: PyUnicode_InternImmortal()'
updated_at = <Date 2021-12-08.11:28:35.305>
user = 'https://github.com/vstinner'

    bugs.python.org fields:

    activity = <Date 2021-12-08.11:28:35.305>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2020-10-02.12:49:46.896>
closer = 'vstinner'
components = ['Interpreter Core']
creation = <Date 2020-09-02.13:59:25.913>
creator = 'vstinner'
dependencies = []
files = []
hgrepos = []
issue_num = 41692
keywords = ['patch']
message_count = 9.0
messages = ['376237', '376271', '376302', '376428', '377784', '377808', '377809', '393350', '408011']
nosy_count = 5.0
nosy_names = ['vstinner', 'methane', 'serhiy.storchaka', 'corona10', 'shihai1991']
pr_nums = ['22486']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue41692'
versions = ['Python 3.10']

    Activity

    1. vstinner commented on Sep 2, 2020

      @vstinner
      vstinner
      on Sep 2, 2020
      MemberAuthor

      Python has the concept of "immortal" interned strings: PyUnicode_InternImmortal().

      The feature was first introduced in the Python 2 "str" (bytes) type, bpo-576101 (commit 45ec02a). New PyString_InternImmortal() function.

      commit 45ec02a
      Author: Guido van Rossum <guido@python.org>
      Date: Mon Aug 19 21:43:18 2002 +0000

      SF patch 576101, by Oren Tirosh: alternative implementation of
interning.  I modified Oren's patch significantly, but the basic idea
and most of the implementation is unchanged.  Interned strings created
with PyString_InternInPlace() are now mortal, and you must keep a
reference to the resulting string around; use the new function
PyString_InternImmortal() to create immortal interned strings.

      Later, the feature was added to the PyUnicodeObject type, new PyUnicode_InternImmortal() function:

      commit 1680713
      Author: Walter Dörwald <walter@livinglogic.de>
      Date: Fri May 25 13:52:07 2007 +0000

      Add interning of unicode strings by copying the functionality from
stringobject.c.

Intern "True" and "False" in bool_repr() again as it was in the
8bit string era.

      Since Python 3.10, (mortal) interned strings are cleared at Python exit in Py_Finalize(). It avoids leaking memory when Python is embedded in an application: bpo-1635741.

      commit 666ecfb
      Author: Victor Stinner <vstinner@python.org>
      Date: Thu Jul 2 01:19:57 2020 +0200

      bpo-1635741: Release Unicode interned strings at exit (GH-21269)

* PyUnicode_InternInPlace() now ensures that interned strings are
  ready.
* Add _PyUnicode_ClearInterned().
* Py_Finalize() now releases Unicode interned strings:
  call _PyUnicode_ClearInterned().

      --

      PyUnicode_InternImmortal() is not used in the Python standard library. I propose to start deprecating the function and remove it in Python 3.12 (PEP-387 requires a deprecation for 2 releases). In Python 3.10, calling the function will emit a DeprecationWarning at runtime.

      Note: PyString_InternImmortal() (for bytes strings) has been removed from Python 3.0.

    2. vstinner
      added
      3.10 (EOL)end of life
      interpreter-core(Objects, Python, Grammar, and Parser dirs)
      on Sep 2, 2020

    3. methane commented on Sep 3, 2020

      @methane
      methane
      on Sep 3, 2020
      Member

      +1

    4. corona10 commented on Sep 3, 2020

      @corona10
      corona10
      on Sep 3, 2020
      Member

      +1

    5. shihai1991 commented on Sep 5, 2020

      @shihai1991
      shihai1991
      on Sep 5, 2020
      Member

      +1

    6. vstinner commented on Oct 1, 2020

      @vstinner
      vstinner
      on Oct 1, 2020
      MemberAuthor

      I proposed PR 22486 to deprecate the function.

    7. vstinner commented on Oct 2, 2020

      @vstinner
      vstinner
      on Oct 2, 2020
      MemberAuthor

      New changeset 583ee5a by Victor Stinner in branch 'master':
      bpo-41692: Deprecate PyUnicode_InternImmortal() (GH-22486)
      583ee5a

    8. vstinner commented on Oct 2, 2020

      @vstinner
      vstinner
      on Oct 2, 2020
      MemberAuthor

      The function is now deprecated. Thanks for the review INADA-san, I close the issue. Let's meet in Python 3.12 to remove it ;-)

    9. vstinner
      closed this as completedon Oct 2, 2020
    10. vstinner
      closed this as completedon Oct 2, 2020

    11. methane commented on May 10, 2021

      @methane
      methane
      on May 10, 2021
      Member

      For the record, I noticed PyUnicode_InternImmortal() is a stable ABI.

      We may need to keep the function to avoid dynamic link errors.
      But we can still change its implementation to just raise an exception.

    12. vstinner commented on Dec 8, 2021

      @vstinner
      vstinner
      on Dec 8, 2021
      MemberAuthor

      I cannot find "PyUnicode_InternImmortal" pattern in the source code of the PyPI top 5000 projects (December 1, 2021).

      I only found a false positive in frozendict-2.1.1:

      frozendict/src/3_10/cpython_src/Include/unicodeobject.h: // PyUnicode_InternImmortal() is deprecated since Python 3.10
      frozendict/src/3_10/cpython_src/Include/unicodeobject.h: Py_DEPRECATED(3.10) PyAPI_FUNC(void) PyUnicode_InternImmortal(PyObject **);
      frozendict/src/3_6/cpython_src/Include/unicodeobject.h: PyAPI_FUNC(void) PyUnicode_InternImmortal(PyObject **);
      frozendict/src/3_7/cpython_src/Include/unicodeobject.h: PyAPI_FUNC(void) PyUnicode_InternImmortal(PyObject **);
      frozendict/src/3_8/cpython_src/Include/unicodeobject.h: PyAPI_FUNC(void) PyUnicode_InternImmortal(PyObject **);
      frozendict/src/3_9/cpython_src/Include/unicodeobject.h: PyAPI_FUNC(void) PyUnicode_InternImmortal(PyObject **);

      These are copies of the Python unicodeobject.h header files, but the PyUnicode_InternImmortal() function is not called by frozendict.

      I used my download_pypi_top.py and search_pypi_top.py tools which can be found at:
      https://github.com/vstinner/misc/tree/main/cpython

    13. ezio-melotti
      transferred this issue fromon Apr 10, 2022
    14. vstinner
      added a commit that references this issue on May 13, 2022

      gh-85858: Remove PyUnicode_InternImmortal() function (#92579)

      059b5ba
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

    Metadata

    Metadata

    Assignees

    No one assigned

      Labels

      3.10 (EOL)end of lifeinterpreter-core(Objects, Python, Grammar, and Parser dirs)

      Projects

      No projects

        Milestone

        No milestone

        Relationships

        None yet

        Development

        No branches or pull requests

        Issue actions