Add __parameters__ and __getitem__ in TypeVar and ParamSpec #88959

Activity

1. serhiy-storchaka commented on Jul 31, 2021

   

   serhiy-storchaka

   on Jul 31, 2021

   MemberAuthor

   More actions

   Adding __parameters__ and __getitem__ in TypeVar and ParamSpec allows to generalize and simplify the code (especially the C code) and allows to add more runtime checks. It may open ways for further simplification.

   Unfortunately it is not compatible with bpo-44098, so the latter changes should be reverted.
2. serhiy-storchaka

   added

   3.11only security fixesonly security fixes

   stdlibStandard Library Python modules in the Lib/ directoryStandard Library Python modules in the Lib/ directory

   type-featureA feature request or enhancementA feature request or enhancement

   on Jul 31, 2021
3. JelleZijlstra commented on Jan 25, 2022

   

   JelleZijlstra

   on Jan 25, 2022

   Member

   More actions

   The linked PR makes T = TypeVar("T"); T[int] work. That's not currently meaningful syntax to the type checker, but we may want it in the future for higher-kinded types. I would rather not make this change without a clear static typing use case in mind.
4. serhiy-storchaka commented on Feb 3, 2022

   

   serhiy-storchaka

   on Feb 3, 2022

   MemberAuthor

   More actions

   It does not have a use case of T[int] in mind. It is an implementation detail which simplifies the code, makes runtime checks more strict and makes types.GenericAlias more similar to typing._GenericAlias.

   For example, currently:

   >>> A = List[T]
   >>> B = list[T]
   >>> A[None]
   typing.List[NoneType]
   >>> B[None]
   list[None]
   >>> A['X']
   typing.List[ForwardRef('X')]
   >>> B['X']
   list['X']

   With the proposed change:

   >>> B[None]
   list[NoneType]
   >>> B['X']
   list[ForwardRef('X')]

   Meanless expressions like A[42], A[Final], A[Final[int]], A[ClassVar], A[ClassVar[int]], etc which are silently accepted will now be errors.

   The code simplification (especially for the C code) is my primary goal, and the rest is a nice side effect. It is possible to add more strict runtime checks and conversions without making TypeVar and ParamSpec subscriptable, but the code will not be so simple.
5. gvanrossum commented on Feb 3, 2022

   

   gvanrossum

   on Feb 3, 2022

   Member

   More actions

   I don't think that changing list[None] to list[NoneType] in the output is an improvement. I do appreciate the reduction in C code though!
6. JelleZijlstra commented on Feb 3, 2022

   

   JelleZijlstra

   on Feb 3, 2022

   Member

   More actions

   I'd also be wary about making changes that introduce additional runtime checks. As we've seen with the PEP-612 and 646 implementations, those can impede future iteration on typing.
7. serhiy-storchaka commented on Feb 5, 2022

   

   serhiy-storchaka

   on Feb 5, 2022

   MemberAuthor

   More actions

   I have created an alternative PR 31143 which adds special method __typing_subst__ in TypeVar and ParamSpec instead of __parameters__ and __getitem__. It has almost the same effect except making TypeVar and ParamSpec and subscriptable. The Python code in typing.py and _collections_abc.py is simplified, but the C code is little changed.

   But on other hand, the reasons for PR 27511:

   1. We already implemented __getitem__ for a Concatenate alias, although it is never documented. It is not even necessary, it is a pure implementation detail which makes the code for nested variables substitution simpler. Adding __parameters__ and __getitem__ in TypeVar and ParamSpec would have the same effect on simplification of the code.

   2. There is a use case for subscription of TypeVar. For example:

   if version < 2:
      RetType = T
   elif version < 3:
      RetType = Optional[T]
   else:
      RetType = Optional[T] | Literal[Unknown]
   
   def get_int(precision: int) -> RetType[int]: ...
   def get_str(encoding: str) -> RetType[str]: ...
   def get_any(converter: Callable[[str], T]) -> RetType[T]: ...
8. gvanrossum commented on Feb 7, 2022

   

   gvanrossum

   on Feb 7, 2022

   Member

   More actions

   Super subtle stuff. Tonight I do not have time to really dive into this, but I think Serhiy is on to something. KJ or Jelle, do you have the guts to dive in here?
9. serhiy-storchaka commented on Mar 11, 2022

   

   serhiy-storchaka

   on Mar 11, 2022

   MemberAuthor

   More actions

   New changeset b6a5d85 by Serhiy Storchaka in branch 'main':
   bpo-44796: Unify TypeVar and ParamSpec substitution (GH-31143)
   b6a5d85
10. serhiy-storchaka commented on Mar 12, 2022

    

    serhiy-storchaka

    on Mar 12, 2022

    MemberAuthor

    More actions

    PR 31143 simplified some code without changing the common API:

    Objects/genericaliasobject.c | 46 ++++++++++++++--------------------------------
    1 file changed, 14 insertions(+), 32 deletions(-)
    Lib/_collections_abc.py | 63 +++++++++------------------------------------------------------
    1 file changed, 9 insertions(+), 54 deletions(-)

    PR 27511 will simplify it even more:

    Objects/genericaliasobject.c | 60 ++++++++++++++++++------------------------------------------
    1 file changed, 18 insertions(+), 42 deletions(-)

    (Lib/typing.py will be simplified too, even if this does not reduce the number of lines).
11. ezio-melotti

    transferred this issue fromon Apr 10, 2022
12. AlexWaygood

    added

    topic-typing

    on Apr 13, 2022