Harmonize descriptor protocol documentation: direct call, super binding with Descriptor Howto docs

@rhettinger

BPO	20751
Nosy	@rhettinger, @vadmium, @Arthur-Milchior
PRs	bpo-20751: Replace method example with attribute example, matching the descriptor howto guide #29909 bpo-20751: Match variable name to the example. #29980 bpo-20751: Clarify descriptor #30271
Dependencies	bpo-12077: Harmonizing descriptor protocol documentation

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/rhettinger'
closed_at = <Date 2021-12-08.03:44:46.988>
created_at = <Date 2014-02-23.22:13:56.330>
labels = ['type-bug', '3.9', '3.10', '3.11', 'invalid', 'docs']
title = 'Harmonize descriptor protocol documentation: direct call, super binding with Descriptor Howto docs'
updated_at = <Date 2021-12-26.21:10:15.423>
user = 'https://bugs.python.org/zuo'

bugs.python.org fields:

activity = <Date 2021-12-26.21:10:15.423>
actor = 'Arthur-Milchior'
assignee = 'rhettinger'
closed = True
closed_date = <Date 2021-12-08.03:44:46.988>
closer = 'rhettinger'
components = ['Documentation']
creation = <Date 2014-02-23.22:13:56.330>
creator = 'zuo'
dependencies = ['12077']
files = []
hgrepos = []
issue_num = 20751
keywords = ['patch']
message_count = 13.0
messages = ['212035', '237709', '294023', '407617', '407627', '407965', '407966', '407970', '407995', '407996', '408410', '409204', '409208']
nosy_count = 5.0
nosy_names = ['rhettinger', 'zuo', 'docs@python', 'martin.panter', 'Arthur-Milchior']
pr_nums = ['29909', '29980', '30271']
priority = 'normal'
resolution = 'not a bug'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue20751'
versions = ['Python 3.9', 'Python 3.10', 'Python 3.11']

One misleading detail in the descriptor protocol documentation for super bindings is that the following fragment of the http://docs.python.org/reference/datamodel.html#invoking-descriptors page:

"""
Super Binding
If a is an instance of super, then the binding super(B, obj).m() searches obj.__class__.__mro__ for the base class A immediately preceding B and then invokes the descriptor with the call: A.__dict__['m'].__get__(obj, obj.__class__).
"""

...introduces the method *call* (".m()") which AFAIK has nothing to do with the actual matter of the description (attribute resolution).

Also, the "If *a* is an instance of super" fragment is strange, as *a* is not used in the following sentences at all.

I believe the description should be:

"""
Super Binding
If binding to a super instance, super(B, obj).x searches obj.__class__.__mro__ for the base class A immediately preceding B and then invokes the descriptor with the call: A.__dict__['x'].__get__(obj, obj.__class__).
"""

(using 'x' as the attribute name, as for the other kinds of binding).

Also, in some earlier fragment of the same page:

"""
Direct Call
The simplest and least common call is when user code directly invokes a descriptor method: x.__get__(a).
"""

The call x.__get__(a) without the second argument seems to be wrong if __get__ is implemented according to the specification "object.__get__(self, instance, owner)" from the same documentation page.

Who is best placed to comment on the suggested docs changes given in msg212035?

Lower-case “a” is defined at the top of the list: “The starting point . . . is ‘a.x’.” The last entry may fit in better if it was written “If binding to an instance of ‘super’ ”.

The problem with the m() call is also mentioned in bpo-25777, about the descriptor how-to.

bpo-12077 seems to be largely about the __get__ signature.

Regarding comment #1, The wording is correct and there was a reason for using a method. While super() can be used for attribute lookup, use cases are almost entirely dominated by method lookups. For many users, an attribute lookup with super() is unconventional, weird, hard to grok, awkward to demonstrate, and not well motivated by the way super() is actually used.

##############################################################
# Demonstration code for the example in text

    class A:
        def m(self):
            return 42

    class B(A):
        def m(obj):
            return super(B, obj).m()

    >>> b = B()
    >>> b.m()                             # Normal invocation 
    42
    >>> A.__dict__['m'].__get__(b, B)()   # Equivalent call
    42

That said, I will switch it to an attribute lookup for consistency with the other examples in the section and with the current version of the DescriptorHowto.

Regarding comment #2, the objtype argument is optional as shown in all of the examples. The call from object.__getattribute__() always passes in both parameters, even though only the first is required.

################################################################
\# Demonstration of \_\_get__() being called with one or two params

    class A:
        def __init__(self, x):
            self.x = x
        def m(self, y):
            return self.x * y

    >>> a = A(10)
    >>> a.m(5)
    50
    >>> vars(A)['m'].__get__(a)(5)     # objtype is not required
    50
    >>> vars(A)['m'].__get__(a, A)(5)  # objtype may be used
    50

################################################################
# Demonstration of object.__getattribute__ supplying both args

    class Desc:
        def __get__(self, *args):
            return args
      
    class B:
        z = Desc()
     
    >>> b = B()
    >>> b.z
    (<__main__.B object at 0x109156110>, <class '__main__.B'>)

New changeset 135ecc3 by Raymond Hettinger in branch 'main':
bpo-20751: Replace method example with attribute example, matching the descriptor howto (GH-29909)
135ecc3

So the current (after the aforementioned commit) form of the description is:

A dotted lookup such as super(A, a).x searches
obj.__class__.__mro__ for a base class B following A and then
returns B.__dict__['x'].__get__(a, A). If not a descriptor, x is
returned unchanged.

I guess here obj was supposed to be a.

But is the description correct when it comes to what class is used where?
I.e., shouldn't it be rather something along the lines of the following:

A dotted lookup such as super(A, obj).x (where obj is an
instance of A of some other subclass of A) searches
A.__mro__ for a base class B whose __dict__ contains name
"x" and then returns B.__dict__['x'].__get__(obj, type(obj)).
If B.__dict__['x'] is not a descriptor, it is returned unchanged.

Ad my comment #2 -- yes, it became groundless with time... Minor explanation: when I reported this issue in 2015, the signature of object.__get__ was documented just as "get(self, instance, owner)" (see: https://docs.python.org/3.5/reference/datamodel.html#implementing-descriptors); that's why I wrote about an inconsistency.

Sorry, a few mistakes distorted my proposal. It should be:

A dotted lookup such as super(A, obj).x (where obj is an
instance of A or of a subclass of A) searches A.__mro__
for a base class whose __dict__ contains name "x", and
then returns B.__dict__['x'].__get__(obj, type(obj)) (where
B is that base class). If B.__dict__['x'] is not a
descriptor, it is returned unchanged.

I am very sorry, I just noticed another mistake.

It should be:

A dotted lookup such as ``super(A, obj).x`` (where ``obj``
is an instance of ``A`` or of a subclass of ``A``) searches
``type(obj).__mro__`` for such a base class ``B`` that follows
``A`` and whose :attr:`__dict__` contains the name ``"x"``;
then ``B.__dict__['x'].__get__(obj, type(obj))`` is returned.
If not a descriptor, ``B.__dict__['x']`` is returned unchanged.

New changeset 4ccccb1 by Raymond Hettinger in branch 'main':
bpo-20751: Match variable name to the example. (GH-29980)
4ccccb1

I guess here obj was supposed to be a.

Okay, I updated the variable name to match the rest of the example.

But is the description correct when it comes to what class is used where?

It looks fine to me. It is harmonious with the other three entries in the section and every part of the wording is verified in the test code.

It should be:

Sorry, I disagree with the wordsmithing. IMO, the additional proposed wording mostly makes it harder to read. Also, this section is already a bit long. The details on super() are in the main docs for super(). Here we just want to show that super() is one of the four ways to invoke a descriptor.

Sure. But don't you think there should be .__get__(a, type(a)) rather than .__get__(a, A)? Then the whole statement would be true regardless of whether A is the actual type of a, or only a superclass of the type of a.

That would also be more consistent with the second point of the description, i.e., the one about Instance Binding (where we have type(a).__dict__['x'].__get__(a, type(a))).

Also, I believe that type(a).__mro__ would be more consistent (than a.__class__.mro) with that point.

Shouldn't those change be ported to 3.10 and maybe even earlier version? They are great, but hard to find if you read the current version manual.

I'm just surprised by the term "dotted lookup". The remaining of the documentation mention "attribute access". So I assumed that there was a reason to use "dotted lookup" instead of "attribute access". If there is such a reason, it's not clear right now. Otherwise, if they are equivalent, I'd suggest remaining consistent in the term used.

I must note that "dotted lookup" is also used in the howto about descriptor, but it does not explain the difference with "attribute access".

One last note, I find it a little bit strange that a was supposed to be the value on which the access is done, and suddenly it becomes a part of the value.

"a base class B following A" shouldn't it be "the base class"? . After all, there is at most one base class following A.

Also, I find it unclear what means "x is returned unchanged, since in this context x is not a value which exists by itself.

Navigation Menu

Repository navigation

Harmonize descriptor protocol documentation: direct call, super binding with Descriptor Howto docs #64950

1 remaining item

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Harmonize descriptor protocol documentation: direct call, super binding with Descriptor Howto docs #64950

Description

Activity

zuo commented on Feb 23, 2014

BreamoreBoy commented on Mar 9, 2015

vadmium commented on May 20, 2017

1 remaining item

rhettinger commented on Dec 3, 2021

rhettinger commented on Dec 4, 2021

zuo commented on Dec 7, 2021

zuo commented on Dec 7, 2021

zuo commented on Dec 7, 2021

rhettinger commented on Dec 8, 2021

rhettinger commented on Dec 8, 2021

zuo commented on Dec 12, 2021

Arthur-Milchior commented on Dec 26, 2021

Arthur-Milchior commented on Dec 26, 2021

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions