Skip to content
Prev Previous commit
Next Next commit
Specification and Rejected Ideas sections
  • Loading branch information
jorenham committed Aug 2, 2021
commit 3cdd57e2847deb59ae108726845294fc32920ae0
113 changes: 88 additions & 25 deletions pep-9999.rst
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
PEP: 9999
Title: Variance operators for generic type variables: +T and -T
Title: Inline variance operators for generic type variables: ``+T`` and ``-T``
Author: Joren Hammudoglu <jhammudoglu at gmail.com>
Status: Active
Type: Standards Track
Expand Down Expand Up @@ -76,10 +76,6 @@ variables outside of generic classes.
results in up to 3x code duplication, especially for type variables
with constraints or an upper bound.

- Although the terms "covariant" "contravariant" are descriptive, they
are verbose, and challenging concepts for people new to typed
languages.


Proposal
========
Expand All @@ -92,33 +88,112 @@ one. These variance operators should only be used within the parameter
list of `typing.Generic` and `typing.Protocol` when subclassing them.

This syntax is practically identical to type parameter variance in the
Scala programming language [2]_.
Scala programming language [1]_.



Specification
=============

The new type variable variance syntax should be used within the generic
parameter list of `typing.Generic` or `typing.Protocol` base classes,
and only there. It is not allowed to be used on type variables that
are already marked as covariant or contravariant, either with
`covariant=True`, `contravariant=True` or with the new `+T` or `-T`
syntax.

Simplified Syntax
-----------------

Instead of specifying the variance on the typevar itself::

T_co = typing.TypeVar('T_co', covariant=True)
T_contra = typing.TypeVar('T_contra', contravariant=True)

class SupportsInvertOld(typing.Protocol[T_co]):
def __invert__(self) -> T_co: ...

class SupportsPartialOrderOld(typing.Protocol[T_contra]):
def __le__(self, other: T_contra): ...


The new syntax uses the `+` and `-` prefix operators to specify
covariant and contravariant generic types inline::

T = typing.TypeVar('T')

class SupportsInvert(typing.Protocol[+T]):
def __invert__(self) -> T: ...

class SupportsPartialOrder(typing.Protocol[-T]):
def __le__(self, other: T): ...


Differences with current syntax
-------------------------------

The new typevar operators return a transparent wrapper around the
original type variable, which can be accessed with the `__orig__`
attribute on the returned wrapper. e.g.::

(+T).__orig__ is T
(+T).__covariant__ == not T.__covariant
Comment thread
jorenham marked this conversation as resolved.
Outdated
(+T).__contravariant__ is False
(+T).__name__ == T.__name__
(+T).__constraints__ == T.__constraints__
(+T).__bound__ is T.__bound__


Thus, type variables defined with `covariant=True` and
`contravariant=True`, are not equivalent to `+T` and `-T`.


How to Teach This
=================
`+T` and `-T` are not valid type annotations, and should only be used
within the generic type parameter list of generic classes. e.g.::

class Spam(typing.Generic[+KT]): ...
class Eggs(typing.Protocol[KT, +VT]): ...
class OrderedSet(typing.AbstractSet[+T]): ...

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is invalid according to the specification, which says you can only put it in Generic or Protocol. I was actually going to ask whether this should be allowed.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah that's good to know. I can see how relaxing that restriction can be desirable, but I guess that that's out of scope here.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it a relaxing? This syntax is definitely legal right now without the +. If you allow it with your new syntax, you'll have to decide what it means. For example, maybe it should be possible to have an invariant subclass of a covariant class.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, thinking about it a little more I believe you should allow this, because invariant subclasses of covariant classes are commonplace. My suggestion would be that you have to specify the + again for every subclass, and if there's no + the subclass is invariant.

For example, we have this in the standard library:

class AbstractSet(Generic[+T]): ...  # covariant

class set(Sequence[T]): ...  # invariant

class frozenset(Sequence[+T]): ... # covariant

To express that, we need what I suggested above.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also agree with Jelle here, I think it should be allowed anywhere the the old forms are allowed.
On another note typing.AbstractSet is deprecated in favour of collections.abc.Set, so it might be a good idea to change that.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another edge case is this:

class WhatIsMyVariance(Generic[+T], Sequence[-T]): ...

That should probably be disallowed, but this should be spelled out.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with your suggestion @JelleZijlstra . But I think that In your example set and frozenset also need to subclass Generic, i.e.

class set(Sequence[T], Generic[T]): ...  # invariant

class frozenset(Sequence[T], Generic[+T]): ... # covariant

Then the restriction that +T and -T can only be used within the type parameter list of Generic and Protocol still holds. If you were to instead write class frozenset(Sequence[+T], Generic[+T]): ..., to me it looks as if Sequence itself is marked as covariant under T, whereas the covariance only applies to the specific definition of frozenset; the variance information does not need to be passed to the Sequence superclass.
This also avoids the possibility for edge cases, like in WhatIsMyVariance.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's not how things currently work, though: writing class set(Sequence[T]): is legal and accepted by typecheckers.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah I see, somehow I thought that in that case you couldn't do e.g. spam: set[str], and that it required an explicit Generic as additional superclass.

Anyway, I now indeed see how this behavior has to be explicitly defined. Requiring the same variance for the same type variable in the same generic class should do the trick. So no mixing; only allow e.g.

  • class LinkedList(Iterable[T], Container[T]): ...
  • class LinkedList(Iterable[+T], Container[+T]): ...
  • class LinkedList(Iterable[-T], Container[-T]): ...

This could be checked within typing.Generic.__init_subclass__.


are all valid uses.


All variance rules that apply to user-defined classes should apply
in the same way with the new syntax, as they do with the current syntax,
and vice-versa.

Suggested Sections
==================


Rejected Ideas
==============

For more detauls about discussions, see links below:

- `Discussion in python/typing <https://github.com/python/typing/issues/813>`_

1. Using `T_co = +TypeVar('T_co')` instead of `T_co = TypeVar('T_co', covariant=True)`
--------------------------------------------------------------------------------------

PROS:

- This requires minimal changes to the syntax
- Replaces the need to type `covariant=True` or `contravariant=True`
with a concise operator.


CONS:

- The `+` and `-` copy the type variable, but type variables should
be unique.
- It is not obvious what to do with the name of the type variable.
- Co- and contravariance are properties of the generic class, not of
the individual type variables.


References
==========

.. [1] mypy
http://mypy-lang.org/
.. [2] Scala Variance
.. [1] Scala Variance
https://docs.scala-lang.org/scala3/book/types-variance.html


Expand All @@ -136,15 +211,3 @@ This document is placed in the public domain or under the CC0-1.0-Universal lice
fill-column: 70
coding: utf-8
End:



Copyright
=========

This document is placed in the public domain or under the
CC0-1.0-Universal license, whichever is more permissive.


Various sections are found to be common across PEPs and are outlined in
PEP 1 [1]_. Those sections are provided here for convenience.