Skip to content
Merged
Changes from 1 commit
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
19ddd2e
Issue-124397: Add free-threading support for iterators.
rhettinger Apr 22, 2026
4aa242d
Add blurb
rhettinger Apr 23, 2026
e0c44be
More wordsmithing
rhettinger Apr 23, 2026
adcb718
Clarify use of the lock. Add message to the ValueError.
rhettinger Apr 23, 2026
4c2bad0
Include "threading" in the reference
rhettinger Apr 23, 2026
d9dde84
Support send(), throw(), and close() for generators.
rhettinger Apr 23, 2026
fcb9ee8
Tweak wording. Add doctest.
rhettinger Apr 23, 2026
314ec67
Merge branch 'main' into iterator_synchronization
rhettinger Apr 23, 2026
802c1e8
Adopt GPS suggestion to make the instance variables private.
rhettinger Apr 28, 2026
201bc76
Adopt GPS suggestion to add an example to the docstring
rhettinger Apr 28, 2026
2384f46
Add test for exceptions in next() calls
rhettinger Apr 28, 2026
8e291b4
Adopt suggestion for more iterator specific names
rhettinger Apr 30, 2026
adfad71
Test the code blocks
rhettinger Apr 30, 2026
b6601a1
Adopt Colesbury suggestion for a generator example
rhettinger Apr 30, 2026
7df1ef7
Use new name in class reference
rhettinger Apr 30, 2026
4efe3f5
Add whatsnew entry
rhettinger Apr 30, 2026
fa733cf
Adopt Peter's suggestion to not use a lazy import
rhettinger May 1, 2026
0e207fb
Note version added
rhettinger May 1, 2026
4802517
Harmonize the examples into parallel form for easy comparison
rhettinger May 1, 2026
06ed996
Update Lib/threading.py
rhettinger May 1, 2026
a81276a
Update Lib/threading.py
rhettinger May 1, 2026
0c0471f
Update Doc/library/threading.rst
rhettinger May 1, 2026
2563d26
Merge branch 'main' into iterator_synchronization
rhettinger May 1, 2026
ed5dd0f
Defer the import of functools
rhettinger May 1, 2026
fbae726
Adopt suggestion from Daniele to verify the worker threads actually s…
rhettinger May 1, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
More wordsmithing
  • Loading branch information
rhettinger committed Apr 23, 2026
commit e0c44be81c2ee8a90b635b25f1aced4a8918ac00
23 changes: 11 additions & 12 deletions Doc/library/threading.rst
Original file line number Diff line number Diff line change
Expand Up @@ -1441,22 +1441,22 @@ Currently, :class:`Lock`, :class:`RLock`, :class:`Condition`,
Iterator synchronization
------------------------

By default, Python iterators do not support concurrent use. Most iterators make
By default, Python iterators do not support concurrent access. Most iterators make
no guarantees when accessed simultaneously from multiple threads. Generator
iterators, for example, raise :exc:`ValueError` if one of their iterator methods
is called while the generator is already executing. The tools in this section
allow reliable concurrency support to be added to ordinary iterators and
iterator-producing callables.

Use :class:`serialize` when multiple threads should share a single iterator and
The :class:`serialize` wrapper lets multiple threads share a single iterator and
Comment thread
eendebakpt marked this conversation as resolved.
Outdated
take turns consuming from it. While one thread is running ``__next__()``, the
others block until the iterator becomes available. Each value produced by the
underlying iterator is delivered to exactly one caller.

Use :func:`concurrent_tee` when multiple threads should each receive the full
The :func:`concurrent_tee` function lets multiple threads each receive the full
stream of values from one underlying iterator. It creates independent iterators
that all draw from the same source. Values are buffered until every derived
iterator has received them.
that all draw from the same source. Values are buffered until consumed by all
of the derived iterators.

.. class:: serialize(iterable)

Expand All @@ -1466,8 +1466,8 @@ iterator has received them.
This makes it possible to share a single iterator, including a generator
iterator, between multiple threads. Calls are handled one at a time in
arrival order determined by lock acquisition. No values are duplicated or
skipped by the wrapper itself; each item produced by the underlying iterator
is returned to exactly one caller.
skipped by the wrapper itself. Each item produced by the underlying iterator
is given to exactly one caller.

This wrapper does not copy or buffer values. Threads that call
:func:`next` while another thread is already advancing the iterator will
Expand Down Expand Up @@ -1501,8 +1501,8 @@ iterator has received them.
Wrap an iterator-producing callable so that each iterator it returns is
automatically passed through :class:`serialize`.

This is especially useful as a decorator for generator functions that may be
consumed from multiple threads.
This is especially useful as a :term:`decorator` for generator functions,
allowing their generator-iterators to be consumed from multiple threads.

Example::

Expand Down Expand Up @@ -1541,8 +1541,7 @@ iterator has received them.
the same order.

Internally, values are buffered until every derived iterator has consumed
them. As a result, if one consumer falls far behind the others, the buffer
may grow without bound.
them.

The returned iterators share the same underlying synchronization lock. Each
individual derived iterator is intended to be consumed by one thread at a
Expand Down Expand Up @@ -1570,4 +1569,4 @@ iterator has received them.
t1.join()
t2.join()

Here, both threads see the full sequence ``0`` through ``4``.
In this example, both consumer threads see the full sequence ``0`` through ``4``.
Loading