Skip to content

Protocols in typing extensions #464

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
gvanrossum merged 35 commits into from
Sep 14, 2017
Merged

Protocols in typing extensions #464

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
1ebd3f5
Adapt Protocol and runtime from PR #417
ilevkivskyi Aug 19, 2017
37b6a7b
An attempt to fix missing failure on some versions
ilevkivskyi Aug 19, 2017
bb3a8a9
Fix missing import
ilevkivskyi Aug 19, 2017
d8ae30e
Fix another missing import
ilevkivskyi Aug 19, 2017
a4e68ff
And another missing import
ilevkivskyi Aug 19, 2017
c8076ee
Abandon attempts to support protocols in 3.5.0 and 3.5.1, see #195
ilevkivskyi Aug 19, 2017
80e8014
Attempt to keep support of 3.5.2
ilevkivskyi Aug 19, 2017
a6a631a
Hopefully last backward compatibility nit
ilevkivskyi Aug 19, 2017
5913fe6
Skip some irrelevant tests
ilevkivskyi Aug 19, 2017
faf4ff0
Conditionally add Protocol and runtime to __all__
ilevkivskyi Aug 19, 2017
c7f6b68
Tweaks following Jelle's comments
ilevkivskyi Aug 20, 2017
8151355
Allow Protocol tests on 3.5.1
ilevkivskyi Sep 2, 2017
76a7ecd
Fix constannt NO_PROTOCOL definition
ilevkivskyi Sep 2, 2017
95faebe
Add super() argument
ilevkivskyi Sep 2, 2017
04f1c51
Fix typo
ilevkivskyi Sep 2, 2017
ce4e04f
Fix super()
ilevkivskyi Sep 2, 2017
a7cf2fb
Fix super() second attemt
ilevkivskyi Sep 2, 2017
8b50c60
Attempt an alternative solution
ilevkivskyi Sep 2, 2017
1dc1971
Fix _gorg in typing 3.5.1
ilevkivskyi Sep 2, 2017
e25077f
Disable some tests for old-style generics and add one new instead
ilevkivskyi Sep 2, 2017
1c1d9fe
Add some tests as proposed by Michael
ilevkivskyi Sep 2, 2017
9b22594
More tests per Michael's CR; remove redundant extra; drop old-style c…
ilevkivskyi Sep 2, 2017
458b232
Fix _gorg on Python 2; add 3.6.2 to Travis
ilevkivskyi Sep 2, 2017
dbc6def
First part of Jukka's CR
ilevkivskyi Sep 2, 2017
31cbe0d
Second part of Jukka's CR: docstrings and comments
ilevkivskyi Sep 2, 2017
171b1e5
Update one comment in test
ilevkivskyi Sep 3, 2017
fc0ec94
Don't support issubclass() for protocols with non-method members
ilevkivskyi Sep 9, 2017
f1f478a
Add more tests
ilevkivskyi Sep 9, 2017
8652ced
Only apply the None rule to callable attributes on class
ilevkivskyi Sep 9, 2017
8f0a778
Add more tests
ilevkivskyi Sep 9, 2017
23e80da
Backport latest changes and tests on Python 2
ilevkivskyi Sep 9, 2017
dc01948
Fix lint
ilevkivskyi Sep 9, 2017
733016d
Fix Python 3.6+ tests
ilevkivskyi Sep 9, 2017
8f12689
One more Python 3.6+ fix
ilevkivskyi Sep 9, 2017
a4d397e
Address CR
ilevkivskyi Sep 14, 2017
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Backport latest changes and tests on Python 2
  • Loading branch information
@ilevkivskyi
ilevkivskyi committed Sep 9, 2017
commit 23e80da6c1ae44f0c25c736c7200a8fe9dbe6679
63 changes: 57 additions & 6 deletions typing_extensions/src_py2/test_typing_extensions.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -344,16 +344,16 @@ def test_protocols_issubclass(self):
T = typing.TypeVar('T')
@runtime
class P(Protocol):
x = 1
def x(self): pass
@runtime
class PG(Protocol[T]):
x = 1
def x(self): pass
class BadP(Protocol):
x = 1
def x(self): pass
class BadPG(Protocol[T]):
x = 1
def x(self): pass
class C(object):
x = 1
def x(self): pass
self.assertIsSubclass(C, P)
self.assertIsSubclass(C, PG)
self.assertIsSubclass(BadP, PG)
Expand All @@ -373,6 +373,29 @@ class C(object):
with self.assertRaises(TypeError):
issubclass(PG, PG[int])

def test_protocols_issubclass_non_callable(self):
class C(object):
x = 1
@runtime
class PNonCall(Protocol):
x = 1
with self.assertRaises(TypeError):
issubclass(C, PNonCall)
self.assertIsInstance(C(), PNonCall)
PNonCall.register(C)
with self.assertRaises(TypeError):
issubclass(C, PNonCall)
self.assertIsInstance(C(), PNonCall)
# check that non-protocol subclasses are not affected
class D(PNonCall): pass
self.assertNotIsSubclass(C, D)
self.assertNotIsInstance(C(), D)
D.register(C)
self.assertIsSubclass(C, D)
self.assertIsInstance(C(), D)
with self.assertRaises(TypeError):
issubclass(D, PNonCall)

def test_protocols_isinstance(self):
T = typing.TypeVar('T')
@runtime
Expand Down Expand Up @@ -425,7 +448,7 @@ class C(object): pass
self.assertIsInstance(C(), P)
self.assertIsInstance(C(), D)

def test_none_blocks_implementation(self):
def test_none_on_non_callable_doesnt_block_implementation(self):
@runtime
class P(Protocol):
x = 1
Expand All @@ -436,6 +459,20 @@ class B(A):
class C(object):
def __init__(self):
self.x = None
self.assertIsInstance(B(), P)
self.assertIsInstance(C(), P)

def test_none_on_callable_blocks_implementation(self):
@runtime
class P(Protocol):
def x(self): pass
class A(object):
def x(self): pass
class B(A):
x = None
class C(object):
def __init__(self):
self.x = None
self.assertNotIsInstance(B(), P)
self.assertNotIsInstance(C(), P)

Expand Down Expand Up @@ -622,6 +659,20 @@ class P(Protocol):
x = None # type: int
class B(object): pass

@JukkaL JukkaL Sep 8, 2017

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you test the variants where x is defined for the instance of B (None / not None) or where x has value None / not None in the body of B?

@ilevkivskyi ilevkivskyi Sep 8, 2017

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I test only few variants/combinations. I will add more tests.

self.assertNotIsInstance(B(), P)
class C(object):
x = 1
class D(object):
x = None
self.assertIsInstance(C(), P)
self.assertIsInstance(D(), P)
class CI(object):
def __init__(self):
self.x = 1
class DI(object):
def __init__(self):
self.x = None
self.assertIsInstance(C(), P)
self.assertIsInstance(D(), P)

def test_protocols_pickleable(self):
global P, CP # pickle wants to reference the class by name
Expand Down
22 changes: 16 additions & 6 deletions typing_extensions/src_py2/typing_extensions.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -187,7 +187,8 @@ def __init__(cls, *args, **kwargs):
isinstance(base, GenericMeta) and base.__origin__ is Generic):
raise TypeError('Protocols can only inherit from other protocols,'
' got %r' % base)

cls._callable_members_only = all(callable(getattr(cls, attr))
for attr in cls._get_protocol_attrs())
def _no_init(self, *args, **kwargs):
if type(self)._is_protocol:
raise TypeError('Protocols cannot be instantiated')
Expand Down Expand Up @@ -219,12 +220,16 @@ def __instancecheck__(self, instance):
# It cannot support runtime protocol metaclasses, On Python 2 classes
# cannot be correctly inspected as instances of protocols.
return False
if issubclass(instance.__class__, self):
if ((not getattr(self, '_is_protocol', False) or
self._callable_members_only) and
issubclass(instance.__class__, self)):
return True
if self._is_protocol:
return all(hasattr(instance, attr) and getattr(instance, attr) is not None
for attr in self._get_protocol_attrs())
return False
if all(hasattr(instance, attr) and
(not callable(getattr(self, attr)) or getattr(instance, attr) is not None)
for attr in self._get_protocol_attrs()):
return True
return super(GenericMeta, self).__instancecheck__(instance)

def __subclasscheck__(self, cls):

@JukkaL JukkaL Sep 1, 2017

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This fails with class A has no attribute '__mro__':

from typing_extensions import Protocol, runtime

@runtime
class P(Protocol):
    def f(self): pass

class A: pass  # Note: no explicit object base class

print issubclass(A, P)  # Error

An explicit object base class works around the problem.

More generally, I'm not sure if subclass checks are well-defined if a protocol has non-method attributes, since they could well be initialized in __init__ and not visible in the type object. This could be fixed by not supporting issubclass with protocols, or only supporting it with protocols that only define methods, since it's rare that a method is defined in __init__ (though it's not completely unheard of).

@JelleZijlstra JelleZijlstra Sep 1, 2017

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's an old-style class in Python 2; I'm not sure we should care about support for old-style classes for Protocols in Python 2 (since they are deprecated and a lot of the class machinery works differently for them).

@ilevkivskyi ilevkivskyi Sep 1, 2017

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, old-stye classes are problematic, I will check if it is possible to do something reasonable with them, if not, then I will add a more meaningful error message like "Old style classes (not inheriting from object) are not supported with protocols".

if (self.__dict__.get('_is_protocol', None) and
Expand All @@ -233,6 +238,11 @@ def __subclasscheck__(self, cls):
return False
raise TypeError("Instance and class checks can only be used with"
" @runtime protocols")
if (self.__dict__.get('_is_runtime_protocol', None) and
not self._callable_members_only):
if sys._getframe(1).f_globals['__name__'] in ['abc', 'functools']:
return super(GenericMeta, self).__subclasscheck__(cls)
raise TypeError("Protocols with non-method members don't support issubclass()")
return super(_ProtocolMeta, self).__subclasscheck__(cls)

def _get_protocol_attrs(self):
Expand All @@ -250,7 +260,7 @@ def _get_protocol_attrs(self):
'__orig_bases__', '__extra__', '__tree_hash__',
'__doc__', '__subclasshook__', '__init__', '__new__',
'__module__', '_MutableMapping__marker',
'__metaclass__', '_gorg')):
'__metaclass__', '_gorg', '_callable_members_only')):
attrs.add(attr)
return attrs

Expand Down