Skip to content

Commit 8608d26

Browse files
committed
contextlib doc updates and refactoring
- explain single use, reusable and reentrant in docs - converted suppress to a reentrant class based impl - converted redirect_stdout to a reusable impl - moved both suppress and redirect_stdout behind a functional facade - added reentrancy tests for the updated suppress - added reusability tests for the updated redirect_stdio - slightly cleaned up an exception from contextmanager
1 parent e723622 commit 8608d26

4 files changed

Lines changed: 213 additions & 34 deletions

File tree

‎Doc/library/contextlib.rst‎

Lines changed: 116 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -128,6 +128,8 @@ Functions and classes provided:
128128
except FileNotFoundError:
129129
pass
130130

131+
This context manager is :ref:`reentrant <reentrant-cms>`.
132+
131133
.. versionadded:: 3.4
132134

133135

@@ -165,6 +167,8 @@ Functions and classes provided:
165167
applications. It also has no effect on the output of subprocesses.
166168
However, it is still a useful approach for many utility scripts.
167169

170+
This context manager is :ref:`reusable but not reentrant <reusable-cms>`.
171+
168172
.. versionadded:: 3.4
169173

170174

@@ -593,3 +597,115 @@ an explicit ``with`` statement.
593597
The specification, background, and examples for the Python :keyword:`with`
594598
statement.
595599

600+
601+
Reusable and reentrant context managers
602+
---------------------------------------
603+
604+
Most context managers are written in a way that means they can only be
605+
used effectively in a :keyword:`with` statement once. These single use
606+
context managers must be created afresh each time they're used -
607+
attempting to use them a second time will trigger an exception or
608+
otherwise not work correctly.
609+
610+
This common limitation means that it is generally advisable to create
611+
context managers directly in the header of the :keyword:`with` statement
612+
where they are used (as shown in all of the usage examples above).
613+
614+
Files are an example of effectively single use context managers, since
615+
the first :keyword:`with` statement will close the file, preventing any
616+
further IO operations using that file object.
617+
618+
Context managers created using :func:`contextmanager` are also single use
619+
context managers, and will complain about the underlying generator failing
620+
to yield if an attempt is made to use them a second time::
621+
622+
>>> from contextlib import contextmanager
623+
>>> @contextmanager
624+
... def singleuse():
625+
... print("Before")
626+
... yield
627+
... print("After")
628+
...
629+
>>> cm = singleuse()
630+
>>> with cm:
631+
... pass
632+
...
633+
Before
634+
After
635+
>>> with cm:
636+
... pass
637+
...
638+
Traceback (most recent call last):
639+
...
640+
RuntimeError: generator didn't yield
641+
642+
643+
.. _reentrant-cms:
644+
645+
Reentrant context managers
646+
^^^^^^^^^^^^^^^^^^^^^^^^^^
647+
648+
More sophisticated context managers may be "reentrant". These context
649+
managers can not only be used in multiple :keyword:`with` statements,
650+
but may also be used *inside* a :keyword:`with` statement that is already
651+
using the same context manager.
652+
653+
:class:`threading.RLock` is an example of a reentrant context manager, as is
654+
:func:`suppress`. Here's a toy example of reentrant use (real world
655+
examples of reentrancy are more likely to occur with objects like recursive
656+
locks and are likely to be far more complicated than this example)::
657+
658+
>>> from contextlib import suppress
659+
>>> ignore_raised_exception = suppress(ZeroDivisionError)
660+
>>> with ignore_raised_exception:
661+
... with ignore_raised_exception:
662+
... 1/0
663+
... print("This line runs")
664+
... 1/0
665+
... print("This is skipped")
666+
...
667+
This line runs
668+
>>> # The second exception is also suppressed
669+
670+
671+
.. _reusable-cms:
672+
673+
Reusable context managers
674+
^^^^^^^^^^^^^^^^^^^^^^^^^
675+
676+
Distinct from both single use and reentrant context managers are "reusable"
677+
context managers (or, to be completely explicit, "reusable, but not
678+
reentrant" context managers, since reentrant context managers are also
679+
reusable). These context managers support being used multiple times, but
680+
will fail (or otherwise not work correctly) if the specific context manager
681+
instance has already been used in a containing with statement.
682+
683+
An example of a reusable context manager is :func:`redirect_stdout`::
684+
685+
>>> from contextlib import redirect_stdout
686+
>>> from io import StringIO
687+
>>> f = StringIO()
688+
>>> collect_output = redirect_stdout(f)
689+
>>> with collect_output:
690+
... print("Collected")
691+
...
692+
>>> print("Not collected")
693+
Not collected
694+
>>> with collect_output:
695+
... print("Also collected")
696+
...
697+
>>> print(f.getvalue())
698+
Collected
699+
Also collected
700+
701+
However, this context manager is not reentrant, so attempting to reuse it
702+
within a containing with statement fails:
703+
704+
>>> with collect_output:
705+
... # Nested reuse is not permitted
706+
... with collect_output:
707+
... pass
708+
...
709+
Traceback (most recent call last):
710+
...
711+
RuntimeError: Cannot reenter <...>

‎Lib/contextlib.py‎

Lines changed: 52 additions & 29 deletions
Original file line numberDiff line numberDiff line change
@@ -48,7 +48,7 @@ def __enter__(self):
4848
try:
4949
return next(self.gen)
5050
except StopIteration:
51-
raise RuntimeError("generator didn't yield")
51+
raise RuntimeError("generator didn't yield") from None
5252

5353
def __exit__(self, type, value, traceback):
5454
if type is None:
@@ -117,6 +117,9 @@ def helper(*args, **kwds):
117117
return helper
118118

119119

120+
# Unfortunately, this was originally published as a class, so
121+
# backwards compatibility prevents the use of the wrapper function
122+
# approach used for the other classes
120123
class closing(object):
121124
"""Context to automatically close something at the end of a block.
122125
@@ -141,55 +144,75 @@ def __enter__(self):
141144
def __exit__(self, *exc_info):
142145
self.thing.close()
143146

144-
class redirect_stdout:
147+
class _RedirectStdout:
148+
"""Helper for redirect_stdout."""
149+
150+
def __init__(self, new_target):
151+
self._new_target = new_target
152+
self._old_target = self._sentinel = object()
153+
154+
def __enter__(self):
155+
if self._old_target is not self._sentinel:
156+
raise RuntimeError("Cannot reenter {!r}".format(self))
157+
self._old_target = sys.stdout
158+
sys.stdout = self._new_target
159+
return self._new_target
160+
161+
def __exit__(self, exctype, excinst, exctb):
162+
restore_stdout = self._old_target
163+
self._old_target = self._sentinel
164+
sys.stdout = restore_stdout
165+
166+
# Use a wrapper function since we don't care about supporting inheritance
167+
# and a function gives much cleaner output in help()
168+
def redirect_stdout(target):
145169
"""Context manager for temporarily redirecting stdout to another file
146170
147171
# How to send help() to stderr
148-
149172
with redirect_stdout(sys.stderr):
150173
help(dir)
151174
152175
# How to write help() to a file
153-
154176
with open('help.txt', 'w') as f:
155177
with redirect_stdout(f):
156178
help(pow)
157-
158-
# How to capture disassembly to a string
159-
160-
import dis
161-
import io
162-
163-
f = io.StringIO()
164-
with redirect_stdout(f):
165-
dis.dis('x**2 - y**2')
166-
s = f.getvalue()
167-
168179
"""
180+
return _RedirectStdout(target)
169181

170-
def __init__(self, new_target):
171-
self.new_target = new_target
182+
183+
class _SuppressExceptions:
184+
"""Helper for suppress."""
185+
def __init__(self, *exceptions):
186+
self._exceptions = exceptions
172187

173188
def __enter__(self):
174-
self.old_target = sys.stdout
175-
sys.stdout = self.new_target
176-
return self.new_target
189+
pass
177190

178191
def __exit__(self, exctype, excinst, exctb):
179-
sys.stdout = self.old_target
180-
181-
@contextmanager
192+
# Unlike isinstance and issubclass, exception handling only
193+
# looks at the concrete type heirarchy (ignoring the instance
194+
# and subclass checking hooks). However, all exceptions are
195+
# also required to be concrete subclasses of BaseException, so
196+
# if there's a discrepancy in behaviour, we currently consider it
197+
# the fault of the strange way the exception has been defined rather
198+
# than the fact that issubclass can be customised while the
199+
# exception checks can't.
200+
# See http://bugs.python.org/issue12029 for more details
201+
return exctype is not None and issubclass(exctype, self._exceptions)
202+
203+
# Use a wrapper function since we don't care about supporting inheritance
204+
# and a function gives much cleaner output in help()
182205
def suppress(*exceptions):
183206
"""Context manager to suppress specified exceptions
184207
185-
with suppress(OSError):
186-
os.remove(somefile)
208+
After the exception is suppressed, execution proceeds with the next
209+
statement following the with statement.
187210
211+
with suppress(FileNotFoundError):
212+
os.remove(somefile)
213+
# Execution still resumes here if the file was already removed
188214
"""
189-
try:
190-
yield
191-
except exceptions:
192-
pass
215+
return _SuppressExceptions(*exceptions)
193216

194217
# Inspired by discussions on http://bugs.python.org/issue13585
195218
class ExitStack(object):

‎Lib/test/test_contextlib.py‎

Lines changed: 44 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -641,27 +641,67 @@ def test_redirect_to_string_io(self):
641641
s = f.getvalue()
642642
self.assertIn('pow', s)
643643

644+
def test_enter_result_is_target(self):
645+
f = io.StringIO()
646+
with redirect_stdout(f) as enter_result:
647+
self.assertIs(enter_result, f)
648+
649+
def test_cm_is_reusable(self):
650+
f = io.StringIO()
651+
write_to_f = redirect_stdout(f)
652+
with write_to_f:
653+
print("Hello", end=" ")
654+
with write_to_f:
655+
print("World!")
656+
s = f.getvalue()
657+
self.assertEqual(s, "Hello World!\n")
658+
659+
# If this is ever made reentrant, update the reusable-but-not-reentrant
660+
# example at the end of the contextlib docs accordingly.
661+
def test_nested_reentry_fails(self):
662+
f = io.StringIO()
663+
write_to_f = redirect_stdout(f)
664+
with self.assertRaisesRegex(RuntimeError, "Cannot reenter"):
665+
with write_to_f:
666+
print("Hello", end=" ")
667+
with write_to_f:
668+
print("World!")
669+
670+
644671
class TestSuppress(unittest.TestCase):
645672

646-
def test_no_exception(self):
673+
def test_no_result_from_enter(self):
674+
with suppress(ValueError) as enter_result:
675+
self.assertIsNone(enter_result)
647676

677+
def test_no_exception(self):
648678
with suppress(ValueError):
649679
self.assertEqual(pow(2, 5), 32)
650680

651681
def test_exact_exception(self):
652-
653682
with suppress(TypeError):
654683
len(5)
655684

656685
def test_multiple_exception_args(self):
657-
686+
with suppress(ZeroDivisionError, TypeError):
687+
1/0
658688
with suppress(ZeroDivisionError, TypeError):
659689
len(5)
660690

661691
def test_exception_hierarchy(self):
662-
663692
with suppress(LookupError):
664693
'Hello'[50]
665694

695+
def test_cm_is_reentrant(self):
696+
ignore_exceptions = suppress(Exception)
697+
with ignore_exceptions:
698+
pass
699+
with ignore_exceptions:
700+
len(5)
701+
with ignore_exceptions:
702+
1/0
703+
with ignore_exceptions: # Check nested usage
704+
len(5)
705+
666706
if __name__ == "__main__":
667707
unittest.main()

‎Misc/NEWS‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -74,7 +74,7 @@ Library
7474
- Issue #19266: Rename the new-in-3.4 ``contextlib.ignore`` context manager
7575
to ``contextlib.suppress`` in order to be more consistent with existing
7676
descriptions of that operation elsewhere in the language and standard
77-
library documentation (Patch by Zero Piraeus)
77+
library documentation (Patch by Zero Piraeus).
7878

7979
- Issue #18891: Completed the new email package (provisional) API additions
8080
by adding new classes EmailMessage, MIMEPart, and ContentManager.

0 commit comments

Comments
 (0)