Skip to content

gh-144356: Make set iterator __length_hint__ and iternext race-safe under no-gil #144357

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
hyongtao-code wants to merge 24 commits into
base: main
Choose a base branch
from
Open

gh-144356: Make set iterator __length_hint__ and iternext race-safe under no-gil #144357

Changes from 1 commit
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
47d75fe
Fix data race in setiter_len() under no-gil
hyongtao-code Jan 31, 2026
3e3785c
Add test case
hyongtao-code Jan 31, 2026
229ced3
📜🤖 Added by blurb_it.
blurb-it[bot] Jan 31, 2026
cdcf88a
Resovle comments
hyongtao-code Feb 1, 2026
21f1478
Update test case: used_race and exhaust_race
hyongtao-code Feb 1, 2026
a18c698
Add test case
hyongtao-code Feb 6, 2026
79b5fbc
Try to address the comments
hyongtao-code Feb 6, 2026
6ac15e0
Post fix
hyongtao-code Feb 6, 2026
3222eef
Address comments
hyongtao-code Feb 8, 2026
7fb39bc
Post fix
hyongtao-code Feb 8, 2026
78241a8
Update Lib/test/test_free_threading/test_set.py
hyongtao-code Feb 9, 2026
52285a2
Revert "Update Lib/test/test_free_threading/test_set.py"
hyongtao-code Feb 9, 2026
c9ece31
Address comments
hyongtao-code Feb 9, 2026
e41e852
Enhance the test case as test_itertools.py
hyongtao-code Feb 10, 2026
b32b5ec
Update NEWS
hyongtao-code Feb 10, 2026
d14856c
Address comment
hyongtao-code Feb 10, 2026
2a512a8
Address comments
hyongtao-code Feb 10, 2026
1ce898a
Update test cases
hyongtao-code Feb 11, 2026
c110ddd
Address comments: unused code
hyongtao-code Feb 13, 2026
6120aef
Update Lib/test/test_free_threading/test_set.py
hyongtao-code Feb 13, 2026
afea804
Update Objects/setobject.c
hyongtao-code Feb 13, 2026
dc4eaa6
Remove empyt line
hyongtao-code Feb 14, 2026
b2ef863
Address comments
hyongtao-code Mar 11, 2026
8e2af8b
Merge branch 'main' into fix-data-race
hyongtao-code May 19, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Post fix
  • Loading branch information
@hyongtao-code
hyongtao-code committed Feb 6, 2026
commit 6ac15e06a61ec4411c853f210e4785a293ea618c
46 changes: 29 additions & 17 deletions Objects/setobject.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1111,6 +1111,7 @@ static PyObject *setiter_iternext(PyObject *self)
Py_ssize_t i, mask;
setentry *entry;
PySetObject *so = si->si_set;
int exhausted = 0;

if (so == NULL)
return NULL;
Expand All @@ -1135,33 +1136,44 @@ static PyObject *setiter_iternext(PyObject *self)
}
#else
i = si->si_pos;
#endif
assert(i>=0);
entry = so->table;
mask = so->mask;
while (i <= mask && (entry[i].key == NULL || entry[i].key == dummy)) {
i++;
if (i < 0) {
/* iterator already exhausted */
exhausted = 1;
Comment thread
hyongtao-code marked this conversation as resolved.
Outdated
}
if (i <= mask) {
key = Py_NewRef(entry[i].key);
#endif

if (!exhausted) {
assert(i >= 0);
entry = so->table;
mask = so->mask;
while (i <= mask && (entry[i].key == NULL || entry[i].key == dummy)) {
i++;
}
if (i <= mask) {
key = Py_NewRef(entry[i].key);
#ifdef Py_GIL_DISABLED
Comment thread
hyongtao-code marked this conversation as resolved.
Outdated
FT_ATOMIC_STORE_SSIZE_RELAXED(si->si_pos, i + 1);
FT_ATOMIC_STORE_SSIZE_RELAXED(si->si_pos, i + 1);
#else
si->si_pos = i + 1;
si->si_pos = i + 1;
#endif
si->len--;
}
else {
si->len--;
}
else {
#ifdef Py_GIL_DISABLED
/* free-threaded: keep si_set; just mark exhausted */
FT_ATOMIC_STORE_SSIZE_RELAXED(si->si_pos, -1);
si->len = 0;
/* free-threaded: keep si_set; just mark exhausted */
FT_ATOMIC_STORE_SSIZE_RELAXED(si->si_pos, -1);
Comment thread
hyongtao-code marked this conversation as resolved.
Outdated
si->len = 0;
Comment thread
hyongtao-code marked this conversation as resolved.
Outdated
#else
si->si_set = NULL;
si->si_set = NULL;
#endif
}
}

#ifdef Py_GIL_DISABLED
done:
#endif
Py_END_CRITICAL_SECTION();

if (key == NULL) {
#ifndef Py_GIL_DISABLED
Comment thread
hyongtao-code marked this conversation as resolved.
Outdated
Py_DECREF(so);

@eendebakpt eendebakpt Feb 10, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you move the decref to the section

       /* exhausted */
        si->si_pos = -1;
        si->len = 0;

?

@hyongtao-code hyongtao-code Feb 13, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I’d prefer not to move the Py_DECREF into the Py_BEGIN_CRITICAL_SECTION block.

Expand Down
Loading