Repository navigation
Refactor typing._GenericAlias #84577
Description
Activity
typing._GenericAlias represents two different types: user defined (like List[int]) and special (like List). They have different behavior, and common methods contain special cases.
The proposed PR rewrites the implementation in more object-oriented paradigm: different classes for different behavior. _GenericAlias is split on three classes: user defined (it may be replaced with GenericAlias in future), special, and the base class for common methods. Its subclasses are also split on classes for special types Tuple and Callable and for parametrized Callable[] and Annotated[]. The work is not finished yet and the code is still complex.
The next PR removes __args__ and __parameters__ from _SpecialGenericAlias.
-
No existing test is failed. It is an evidence that these attributes are not intended, but a side effect of mixing two different kinds in one class.
-
get_args() ignores __args__ and returns () for _SpecialGenericAlias (after resolving bpo-40398).
-
Nested _SpecialGenericAlias is not considered a generic class containing type variables:
>>> from typing import * >>> T = TypeVar('T') >>> Dict[int, List[T]][str] typing.Dict[int, typing.List[str]] >>> Dict[int, List][str] Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/home/serhiy/py/cpython-release/Lib/typing.py", line 244, in inner return func(*args, **kwds) File "/home/serhiy/py/cpython-release/Lib/typing.py", line 695, in __getitem__ _check_generic(self, params) File "/home/serhiy/py/cpython-release/Lib/typing.py", line 194, in _check_generic raise TypeError(f"{cls} is not a generic class") TypeError: typing.Dict[int, typing.List] is not a generic class
It also fixes the following error:
>>> from typing import * >>> T = TypeVar('T') >>> Dict[T, List][str] Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/home/serhiy/py/cpython-release/Lib/typing.py", line 241, in inner return cached(*args, **kwds) File "/home/serhiy/py/cpython-release/Lib/typing.py", line 703, in __getitem__ subargs = tuple(subst[x] for x in arg.__parameters__) File "/home/serhiy/py/cpython-release/Lib/typing.py", line 703, in <genexpr> subargs = tuple(subst[x] for x in arg.__parameters__) KeyError: ~T
and allows to simplify __eq__ and __hash__ for _SpecialGenericAlias. Currently it has a weird behavior in corner case:
>>> from typing import * >>> from typing import T >>> List == List[T] True
-
The most recent change here caused a regression. The following file:
from typing import Generic, TypeVar, Union class CannotTransform(Exception): pass T = TypeVar("T") E = TypeVar("E", bound=Exception) class Ok(Generic[T]): pass class Err(Generic[E]): pass Result = Union[Ok[T], Err[E]] TResult = Result[T, CannotTransform] TMatchResult = TResult[int]Now fails with:
Traceback (most recent call last): File "/Users/jzijlstra-mpbt/py/cpython/Lib/../../black/black.py", line 22, in <module> TMatchResult = TResult[int] File "/Users/jzijlstra-mpbt/py/cpython/Lib/typing.py", line 244, in inner return func(*args, **kwds) File "/Users/jzijlstra-mpbt/py/cpython/Lib/typing.py", line 704, in __getitem__ arg = arg[subargs] File "/Users/jzijlstra-mpbt/py/cpython/Lib/typing.py", line 244, in inner return func(*args, **kwds) File "/Users/jzijlstra-mpbt/py/cpython/Lib/typing.py", line 695, in __getitem__ _check_generic(self, params) File "/Users/jzijlstra-mpbt/py/cpython/Lib/typing.py", line 194, in _check_generic raise TypeError(f"{cls} is not a generic class") TypeError: __main__.Err[__main__.CannotTransform] is not a generic classBefore commit c1c7d8e it was fine. This was found when we added 3.9-dev to CI for Black (psf/black#1393).
@serhiy can you look at this? A possible fix might be:
diff --git a/Lib/typing.py b/Lib/typing.py index 681ab6d21e..adcef1e82b 100644 --- a/Lib/typing.py +++ b/Lib/typing.py @@ -701,7 +701,8 @@ class _GenericAlias(_BaseGenericAlias, _root=True): arg = subst[arg] elif isinstance(arg, (_BaseGenericAlias, GenericAlias)): subargs = tuple(subst[x] for x in arg.__parameters__) - arg = arg[subargs] + if subargs: + arg = arg[subargs] new_args.append(arg) return self.copy_with(tuple(new_args))
The immediate cause of the problem seems to be that we call arg[subargs] with subargs being (). So arg is a _GenericAlias but its __parameters__ is (). I'm not sure what path was taken previously. :-(
I think I have a shorter repro, not involving unions:
from typing import * T = TypeVar("T") S = TypeVar("S") U = TypeVar("U") class A(Generic[T]): ... class B(Generic[T]): ... class C(Generic[T, S]): ... print(C[A[U], B[int]][str])
Fails in the same place, works in 3.8 (I didn't check just before the offending commit), and the same fix works.
Thank you Jelle for your report. There is even simpler example:
Dict[Tuple[T], List[int]][str]
Even simpler: Dict[T, List[int]][str].
Dict[T, list[int]][str] also fails. But dict[T, list[int]][str] works as expected (and there are tests).
Thanks Serhyi! I can confirm that the issue I posted is fixed.
Looks like there's nothing left to do here.
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields: