Skip to content

gh-151722: Defer GC tracking of frozendict to end of construction #151740

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
corona10 merged 6 commits into from
Jun 22, 2026
Merged

gh-151722: Defer GC tracking of frozendict to end of construction #151740

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
cd1b1a4
gh-151722: Defer GC tracking of frozendict to end of construction
corona10 Jun 19, 2026
399d810
Address code review
corona10 Jun 20, 2026
286eafd
fix
corona10 Jun 20, 2026
d74be81
fix
corona10 Jun 20, 2026
897bdec
Update
corona10 Jun 20, 2026
69a6316
Address code review
corona10 Jun 22, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions Misc/NEWS.d/next/Core_and_Builtins/2026-06-20-00-30-47.gh-issue-151722.RPMPIY.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
Defer GC tracking of :class:`frozendict` to end of construction. Patch by
Donghee Na.
48 changes: 40 additions & 8 deletions Objects/dictobject.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -138,6 +138,7 @@ As a consequence of this, split keys have a maximum size of 16.
// Forward declarations
static PyObject* frozendict_new(PyTypeObject *type, PyObject *args,
PyObject *kwds);
static PyObject* frozendict_new_untracked(PyTypeObject *type);
static PyObject* dict_new(PyTypeObject *type, PyObject *args, PyObject *kwds);
static int dict_merge(PyObject *a, PyObject *b, int override, PyObject **dupkey);
static int dict_contains(PyObject *op, PyObject *key);
Expand Down Expand Up @@ -5242,7 +5243,7 @@ static PyNumberMethods dict_as_number = {
};

static PyObject *
dict_new(PyTypeObject *type, PyObject *args, PyObject *kwds)
dict_new_untracked(PyTypeObject *type)
{
assert(type != NULL);
assert(type->tp_alloc != NULL);
Expand All @@ -5262,8 +5263,18 @@ dict_new(PyTypeObject *type, PyObject *args, PyObject *kwds)
d->ma_keys = Py_EMPTY_KEYS;
d->ma_values = NULL;
ASSERT_CONSISTENT(d);
if (!_PyObject_GC_IS_TRACKED(d)) {
_PyObject_GC_TRACK(d);
return self;
}

static PyObject *
dict_new(PyTypeObject *type, PyObject *args, PyObject *kwds)

@markshannon markshannon Jun 20, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This function is strange. Why does it ignore its parameters?
Creating a dict with arguments should create a dict from those arguments.

This comment was marked as outdated.

Sign in to view

@corona10 corona10 Jun 20, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cpython/Objects/dictobject.c

Lines 5245 to 5269 in aa5b164

dict_new(PyTypeObject *type, PyObject *args, PyObject *kwds)
{
assert(type != NULL);
assert(type->tp_alloc != NULL);
// dict subclasses must implement the GC protocol
assert(_PyType_IS_GC(type));
PyObject *self = type->tp_alloc(type, 0);
if (self == NULL) {
return NULL;
}
PyDictObject *d = (PyDictObject *)self;
d->ma_used = 0;
d->_ma_watcher_tag = 0;
// We don't inc ref empty keys because they're immortal
assert((Py_EMPTY_KEYS)->dk_refcnt == _Py_DICT_IMMORTAL_INITIAL_REFCNT);
d->ma_keys = Py_EMPTY_KEYS;
d->ma_values = NULL;
ASSERT_CONSISTENT(d);
if (!_PyObject_GC_IS_TRACKED(d)) {
_PyObject_GC_TRACK(d);
}
return self;
}

The function already ignore those params, and it was done by dict_init

@markshannon markshannon Jun 20, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed:

>>> dict.__new__(dict, (), a=1,b=2)
{}

but frozendict is inconsistent:

>>> frozendict.__new__(frozendict, (), a=1,b=2)
frozendict({'a': 1, 'b': 2})

@corona10 corona10 Jun 20, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that than we need to modify frozenset also

>>> set.__new__(set, [1,2,3])
set()
>>> frozenset.__new__(frozenset, [1,2,3])
frozenset({1, 2, 3})

Let's handle this at separate issue.

@vstinner vstinner Jun 22, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see what's wrong here. For me, it works as expected:

>>> dict((), a=1,b=2)
{'a': 1, 'b': 2}
>>> frozendict((), a=1,b=2)
frozendict({'a': 1, 'b': 2})

The fact that arguments are used by __new__() or __init__() is just an implementation detail. I don't think that it matters for the "constructor" API.

{
PyObject *self = dict_new_untracked(type);
if (self == NULL) {
return NULL;
}
if (!_PyObject_GC_IS_TRACKED(self)) {

@methane methane Jun 20, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this if statement needed?

@corona10 corona10 Jun 20, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we don't need it

_PyObject_GC_TRACK(self);
}
return self;
}
Expand Down Expand Up @@ -5323,7 +5334,9 @@ frozendict_vectorcall(PyObject *type, PyObject * const*args,
return Py_NewRef(args[0]);
}

PyObject *self = frozendict_new(_PyType_CAST(type), NULL, NULL);
/* Keep the frozendict untracked until it is fully built, so a half-built
object is never reachable from another thread. */

@vstinner vstinner Jun 22, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
object is never reachable from another thread. */
object is never reachable from another thread (using the gc module). */

You may add a reference to the issue gh-151722.

PyObject *self = frozendict_new_untracked(_PyType_CAST(type));
if (self == NULL) {
return NULL;
}
Expand All @@ -5343,6 +5356,10 @@ frozendict_vectorcall(PyObject *type, PyObject * const*args,
}
}
}
/* Track only once fully built. */
if (!_PyObject_GC_IS_TRACKED(self)) {
_PyObject_GC_TRACK(self);
}
return self;
}

Expand Down Expand Up @@ -8361,17 +8378,28 @@ frozendict_hash(PyObject *op)
}


/* Allocate an empty, GC-untracked frozendict. Staying untracked while it is
filled keeps a half-built frozendict so another thread can't observe it
changing. Callers must GC-track it once fully built. */
static PyObject *
frozendict_new(PyTypeObject *type, PyObject *args, PyObject *kwds)
frozendict_new_untracked(PyTypeObject *type)
{
PyObject *d = dict_new(type, args, kwds);
PyObject *d = dict_new_untracked(type);
if (d == NULL) {
return NULL;
}
assert(can_modify_dict(_PyAnyDict_CAST(d)));
_PyFrozenDictObject_CAST(d)->ma_hash = -1;
return d;
}

PyFrozenDictObject *self = _PyFrozenDictObject_CAST(d);
self->ma_hash = -1;
static PyObject *
frozendict_new(PyTypeObject *type, PyObject *args, PyObject *kwds)
{
PyObject *d = frozendict_new_untracked(type);
if (d == NULL) {
return NULL;
}

if (args != NULL) {
if (dict_update_common(d, args, kwds, "frozendict") < 0) {
Expand All @@ -8383,6 +8411,10 @@ frozendict_new(PyTypeObject *type, PyObject *args, PyObject *kwds)
assert(kwds == NULL);
}

/* Track only once fully built. */
if (!_PyObject_GC_IS_TRACKED(d)) {
_PyObject_GC_TRACK(d);
}
return d;
}

Expand Down
Loading