Skip to content
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
49 changes: 49 additions & 0 deletions Doc/library/typing.rst
Original file line number Diff line number Diff line change
Expand Up @@ -933,6 +933,55 @@ These can be used as types in annotations using ``[]``, each having a unique syn

.. versionadded:: 3.9


.. data:: TypeGuard

Special typing form used to annotate the return type of a user-defined
type guard function. ``TypeGuard`` only accepts a single type argument.
Comment thread
Fidget-Spinner marked this conversation as resolved.
Resolved

``TypeGuard`` aims to benefit *type narrowing* - a technique used by static
type checkers to determine a more precise type of an expression within a
program's code flow. Usually type narrowing is done by analyzing
conditional code flow and applying the narrowing to a block of code. The
conditional expression here is sometimes referred to as a "type guard".
Comment thread
Fidget-Spinner marked this conversation as resolved.
Resolved

Sometimes, a type guard can be a complex checking function.
Comment thread
Fidget-Spinner marked this conversation as resolved.
Resolved
These type guard functions require ``TypeGuard`` to narrow their input types
Comment thread
Fidget-Spinner marked this conversation as resolved.
Resolved
as the static type checker usually does not have enough information to
statically infer them.

A ``TypeGuard`` tells the static type checker that for a given function:
Comment thread
Fidget-Spinner marked this conversation as resolved.
Resolved
1. The return value is a boolean.
Comment thread
Fidget-Spinner marked this conversation as resolved.
Resolved
2. If the return value was "truthy", the type of the input to the

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's not get into truthiness here -- let's just say ``True``. Also, s/was/is/.

Separately, I wonder if we need to somehow distinguish more carefully between what happens in the static checker and at runtime. The checker makes the assumption that a True result implies that the expression used as the first argument has the type given in the TypeGuard[X] annotation. It behooves the implementation of the type guard function to honor this assumption at runtime, but neither the type checker nor the runtime has the ability to verify that it is so -- if the function "lies", that is a runtime bug for which the checker cannot take responsibility.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you mean that for the current PEP, things like:

def is_fake_str_list(val: List[object]) -> TypeGuard[List[str]]:
    '''Liar!'''
    return True

def func1(val: List[object]):
    if is_str_list(val):
        # Type of ``val`` is narrowed to List[str]
        print(" ".join(val))
    else:
        # Type of ``val`` remains as List[object]
        print("Not a list of strings!")

# Kaboom!
func([1,2,3])

Fail at runtime instead of being caught at "compile" time?

If so, I'm not sure if there's an easy way to fix that. One way would be for the static type checker to verify is_fake_str_list recursively. But this would bring about a crop of other problems, weird edge cases, and added implementation complexity for them.

Here's some comments from the PEP that I agree with (though it's talking about why there isn't strict narrowing, not runtime vs static checkers' responsibilities):

However, there are many ways a determined or uninformed developer can subvert type safety -- most commonly by using cast or Any. If a Python developer takes the time to learn about and implement user-defined type guards within their code, it is safe to assume that they are interested in type safety and will not write their type guard functions in a way that will undermine type safety or produce nonsensical results.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that's what I meant. And of course there's no easy fix for that. I just want the docs to not mislead the reader into thinking that that is somehow checked.

function is specified by the type inside ``TypeGuard``.
Comment thread
Fidget-Spinner marked this conversation as resolved.
Resolved

For example::

def is_str_list(val: List[object]) -> TypeGuard[List[str]]:
'''Determines whether all objects in the list are strings'''
return all(isinstance(x, str) for x in val)

def func1(val: List[object]):
if is_str_list(val):
# Type of ``val`` is narrowed to List[str]
print(" ".join(val))
else:
# Type of ``val`` remains as List[object]
print("Not a list of strings!")

In short, the form ``def foo(arg: TypeA) -> TypeGuard[TypeB]: ...``,
Means that if ``foo(arg)`` returned true, then ``arg`` narrows from
Comment thread
Fidget-Spinner marked this conversation as resolved.
Resolved
``TypeA`` to ``TypeB``.
Comment thread
Fidget-Spinner marked this conversation as resolved.
Resolved

Return statements within a type guard function should return ``bool``
values.
Comment thread
Fidget-Spinner marked this conversation as resolved.
Resolved

``TypeGuard`` also works with type variables. For more information, see
:pep:`647` (User-Defined Type Guards).

.. versionadded:: 3.10


Building generic types
""""""""""""""""""""""

Expand Down
9 changes: 9 additions & 0 deletions Doc/whatsnew/3.10.rst
Original file line number Diff line number Diff line change
Expand Up @@ -576,6 +576,15 @@ See :pep:`613` for more details.

(Contributed by Mikhail Golubev in :issue:`41923`.)

PEP 647: User-Defined Type Guards
---------------------------------

:data:`TypeGuard` has been added to the :mod:`typing` module to annotate
type guard functions and improve information provided to static type checkers
during type narrowing. For more information, please see :data:`TypeGuard`\ 's
documentation, and :pep:`647`.

(Contributed by Ken Jin in :issue:`43766`. PEP written by Eric Traut.)

Other Language Changes
======================
Expand Down
49 changes: 49 additions & 0 deletions Lib/test/test_typing.py
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@
from typing import Annotated, ForwardRef
from typing import TypeAlias
from typing import ParamSpec, Concatenate
from typing import TypeGuard
import abc
import typing
import weakref
Expand Down Expand Up @@ -4346,6 +4347,54 @@ def test_valid_uses(self):
self.assertEqual(C4.__parameters__, (T, P))


class TypeGuardTests(BaseTestCase):
def test_basics(self):
TypeGuard[int] # OK

def foo(arg) -> TypeGuard[int]: ...
self.assertEqual(gth(foo), {'return': TypeGuard[int]})

with self.assertRaises(TypeError):
TypeGuard[1]
with self.assertRaises(TypeError):
TypeGuard[int, str]

def test_repr(self):
self.assertEqual(repr(TypeGuard), 'typing.TypeGuard')
cv = TypeGuard[int]
self.assertEqual(repr(cv), 'typing.TypeGuard[int]')
cv = TypeGuard[Employee]
self.assertEqual(repr(cv), 'typing.TypeGuard[%s.Employee]' % __name__)
cv = TypeGuard[tuple[int]]
self.assertEqual(repr(cv), 'typing.TypeGuard[tuple[int]]')

def test_cannot_subclass(self):
with self.assertRaises(TypeError):
class C(type(TypeGuard)):
pass
with self.assertRaises(TypeError):
class C(type(TypeGuard[int])):
pass

def test_cannot_init(self):
with self.assertRaises(TypeError):
TypeGuard()
with self.assertRaises(TypeError):
type(TypeGuard)()
with self.assertRaises(TypeError):
type(TypeGuard[Optional[int]])()

def test_no_isinstance(self):
with self.assertRaises(TypeError):
isinstance(1, TypeGuard[int])
with self.assertRaises(TypeError):
issubclass(int, TypeGuard)

Comment thread
Fidget-Spinner marked this conversation as resolved.
Resolved
def test_final_unmodified(self):
def func(x): ...
self.assertIs(func, final(func))


class AllTests(BaseTestCase):
"""Tests for __all__."""

Expand Down
43 changes: 43 additions & 0 deletions Lib/typing.py
Original file line number Diff line number Diff line change
Expand Up @@ -118,6 +118,7 @@
'Text',
'TYPE_CHECKING',
'TypeAlias',
'TypeGuard',
]

# The pseudo-submodules 're' and 'io' are part of the public
Expand Down Expand Up @@ -566,6 +567,48 @@ def Concatenate(self, parameters):
return _ConcatenateGenericAlias(self, parameters)


@_SpecialForm
def TypeGuard(self, parameters):
"""Special typing form used to annotate the return type of a user-defined
type guard function. ``TypeGuard`` only accepts a single type argument.

``TypeGuard`` aims to benefit *type narrowing* - a technique used by static
type checkers to determine a more precise type of an expression within a
program's code flow.

A ``TypeGuard`` tells the static type checker that for a given function:
1. The return value is a boolean.
2. If the return value was "truthy", the type of the input to the
function is specified by the type inside ``TypeGuard``.
Comment thread
Fidget-Spinner marked this conversation as resolved.
Resolved

For example::

def is_str_list(val: List[object]) -> TypeGuard[List[str]]:
'''Determines whether all objects in the list are strings'''
return all(isinstance(x, str) for x in val)

def func1(val: List[object]):
if is_str_list(val):
# Type of ``val`` is narrowed to List[str]
print(" ".join(val))
else:
# Type of ``val`` remains as List[object]
print("Not a list of strings!")

In short, the form ``def foo(arg: TypeA) -> TypeGuard[TypeB]: ...``,
Means that if ``foo(arg)`` returned true, then ``arg`` narrows from
Comment thread
Fidget-Spinner marked this conversation as resolved.
Resolved
``TypeA`` to ``TypeB``.

Return statements within a type guard function should return ``bool``
values.
Comment thread
Fidget-Spinner marked this conversation as resolved.
Resolved

``TypeGuard`` also works with type variables. For more information, see
:pep:`647` (User-Defined Type Guards).
"""
item = _type_check(parameters, f'{self} accepts only single type.')
return _GenericAlias(self, (item,))


class ForwardRef(_Final, _root=True):
"""Internal wrapper to hold a forward reference."""

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
Implement :pep:`647` in the :mod:`typing` module by adding
:data:`TypeGuard`.