Idle: Use inspect.signature for calltips

@terryjreedy

BPO	19903
Nosy	@terryjreedy, @serhiy-storchaka, @vedgar, @mlouielu
PRs	bpo-19903: IDLE: Change to inspect.signature for calltips #1382 bpo-19903: IDLE: Calltips changed to use inspect.signature #2822 [3.6] bpo-19903: IDLE: Calltips changed to use inspect.signature (GH-2822) #3053
Dependencies	bpo-20122: Move CallTips tests to idle_tests bpo-20223: inspect.signature does not support new functools.partialmethod bpo-20401: inspect.signature removes initial starred method params (bug)

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/terryjreedy'
closed_at = <Date 2017-08-10.04:59:32.044>
created_at = <Date 2013-12-06.00:21:46.168>
labels = ['expert-IDLE', 'type-bug', '3.7']
title = 'Idle: Use inspect.signature for calltips'
updated_at = <Date 2017-08-10.06:15:00.611>
user = 'https://github.com/terryjreedy'

bugs.python.org fields:

activity = <Date 2017-08-10.06:15:00.611>
actor = 'terry.reedy'
assignee = 'terry.reedy'
closed = True
closed_date = <Date 2017-08-10.04:59:32.044>
closer = 'terry.reedy'
components = ['IDLE']
creation = <Date 2013-12-06.00:21:46.168>
creator = 'terry.reedy'
dependencies = ['20122', '20223', '20401']
files = []
hgrepos = []
issue_num = 19903
keywords = []
message_count = 37.0
messages = ['205339', '208757', '213878', '219203', '235154', '235156', '235157', '288586', '292636', '292712', '292719', '292721', '292741', '292787', '292864', '292964', '292966', '292969', '292985', '293008', '293010', '293011', '293012', '293013', '293019', '293020', '293024', '293042', '293052', '293055', '293083', '300041', '300042', '300045', '300047', '300048', '300049']
nosy_count = 4.0
nosy_names = ['terry.reedy', 'serhiy.storchaka', 'veky', 'louielu']
pr_nums = ['1382', '2822', '3053']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue19903'
versions = ['Python 3.6', 'Python 3.7']

Change idlelib.CallTips.get_argspec to use inspect.signature, new in 3.3, instead of inspect.getfullargspec and inspect.formatargspec. One thing it handles better is a namedtuple class, which has a C-coded __init__ inherited from object a Python-coded __new__ written by namedtuple. Signature() will also handle C-coded functions if and when Argument Clinic (new in 3.4) adds signature information.

from collections import namedtuple
from inspect import signature

Point = namedtuple('Point', 'x y')
print(signature(Point.__new__))
# '(_cls, x, y)'
print(signature(Point))
# '(x, y)'

Note that str (called by print) is needed to get the desired string form.

There are tests in CallTips.py, but they should be converted to unittest, moved to idle_test/test_calltips.py, changed to match new output detail, and expanded to include new examples.

*17481 is about changing getargspec to use signature, at least as a backup. But that would not fix the example here. The calltips have been moved in 20122.

I have not decided yet whether to apply to 3.4.

My understanding is that when a builtin is converted, a) the docstring signature disappears, and b) getfullargspec calls str(signature). I need to check what now happens with .getfullargspec version .signature for uncoverted builtins and if any changes should be made now.

.signature also handles wrapped functions better.

>>> def deco(f):
        import functools
        @functools.wraps(f)
        def ret(*args, **kw):
                return f(*args, **kw)
        return ret

>>> @deco
def f(n, k):
        return n + k*2

>>> import inspect as ip
>>> isf=ip.signature(f)
>>> print(isf)
(n, k)
>>> ip.formatargspec(*ip.getfullargspec(f))
'(*args, **kw)'

The above suggests that the change might be applied to 3.4. The difference surprises me because bpo-17481 patched getfullargspec() to 'use' signature() 'behind the scenes', so I expected the result to be the same. It seems that 'use' does not mean what I thought. My hesitation is that only some builtins have been converted to use Argument Clinic, and Idle currently treats builtins special to get the signature from the docstring. Treating just some builting special will be slightly awkward.

An example should make my concern clearer. str(signature(int)) is '()' because int (and all other builtins) has not been converted whereas the Idle calltip is
int(x=0) -> integer
int(x, base=10) -> integer
I do not want to simply add a line with '()' to the top of the calltip, so a little thought is needed to improve the calltip for the rare cases above without degrading them for existing, more common cases.

Several builtin functions have recently gotten the Arg clinic treatment. So I think it is now time to switch. Before closing bpo-29653 as a duplicate, I discovered that inspect.signature also handles functools.partials correctly, while the other inspect functions do not.

I'm now testing to change getfullargspect to signature. It came out that signature can't accept bound method with only _VAR_KEYWORD. This test case will gave a ValueError:

    >>> class C:
        def m2(**kw):
            pass
    >>> c = C()
    >>> ip.signature(c.m2)
   Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.6/inspect.py", line 3002, in signature
    return Signature.from_callable(obj, follow_wrapped=follow_wrapped)
  File "/usr/lib/python3.6/inspect.py", line 2752, in from_callable
    follow_wrapper_chains=follow_wrapped)
  File "/usr/lib/python3.6/inspect.py", line 2169, in _signature_from_callable
    return _signature_bound_method(sig)
  File "/usr/lib/python3.6/inspect.py", line 1759, in _signature_bound_method
    raise ValueError('invalid method signature')
ValueError: invalid method signature

Thanks for the report. As Yuri pointed out in msg292701, and I verified, c.m2() raises "TypeError: meth() takes 0 positional arguments but 1 was given".

The purpose of get_argspec is to tell the user how to call the function without getting such a TypeError. But this is not possible, at least in this case. When signature raises "ValueError: invalid method signature", get_argspec should catch ValueError and put, for instance, "Function has an invalid method signature" in the box, instead of a signature that does not work, so users will know that the function cannot be called properly. (Should message be red?)

An initial patch could just comment out this test. I would like to see whether everything else passes. Or have you, Louie, already done that?
Signature can raise "TypeError if that type of object is not supported". Does .signature return for all the other test cases? I expect it should if only called after current guard.

try:
    ob_call = ob.__call__
except BaseException:
    return argspec

Before committing a patch using .signature, the known possible exception (ValueError) must be caught. This test should then be changed to verify the new behavior.

Possible future enhancement. Current nothing happens after code like "1(" or "''(". Should a box pop up saying (in red?) something like "Non-callable object before ("?

Terry, only this case was failed, others in unittest work very well.

(I presume'ordinal' meant 'ordinary'.) I don't know where signature finds the info on built-in type objects Methods like .append have .__text_signature__. List does not, and list.__call__.__text_signature is the generic '($self, /, *args, **kwargs)'. That signature finds it somewhere is a reason for the switch. There is no longer a signature in the first lines of the docstring. So currently, 'list(' only displays "Built-in mutable sequence."

Louie, I verified that for python-coded classes, signature itself gets the info from the __init__ method, so we don't need 'fob = ob.__init__' for python classes either.

>>> list.__text_signature__
'(iterable=(), /)'

But IDLE shouldn't use __text_signature__ directly, it should use the inspect module.

Interesting dir(list.append) and the IDLE completion box show __text_signature__. dir(list) and the box do not.

As well as __name__, __qualname__, __module__, __bases__, __call__, mro, etc.

But somehow in this case, List have no __text_signature__:

>>> str(i.signature(list))
'(iterable=(), /)'
>>> list.__text_signature__
'(iterable=(), /)'
>>> str(i.signature(List))
'(iterable=(), /)'
>>> List.__text_signature__
>>>

And, another case, the __text_signature__ and i.signature() is similar, but not the same:

>>> range.__init__.__text_signature__
'($self, /, *args, **kwargs)'
  ^
>>> str(i.signature(range.__init__))
'(self, /, *args, **kwargs)'

So we can't simply compare-and-eliminate this case.

What is List? Not a builtin.

I am guessing that $self means to remove for bound methods. In any case, __text_signature__ is effectively private to inspect.signature. We only care what the latter produces.

Am I right in assuming this will also fix the bug mentioned in this comment

https://emptysqua.re/blog/copying-a-python-functions-signature/#comment-1816090904

? If yes, I'm glad I didn't have to report it. :-)

New changeset 646f6c3 by Terry Jan Reedy in branch '3.6':
[3.6] bpo-19903: IDLE: Calltips changed to use inspect.signature (GH-2822) (bpo-3053)
646f6c3

Although my freshly opened window says PR2822 is open, it was merged with
3b0f620
4 hours ago, and then backported. Louie, thanks for the patch and the persistence.

This issue is about IDLE getting further out of the business of calculating signatures and depending instead on the newest (and tested) inspect function.

In 2012, the blogger used getargspec, which was deprecated for 3.x a few years before, in 3.0. I suspect that the blog is about 2.x, which is no longer relevant for IDLE development.

Inspect.getargspec was superceded by getfullargspec, which was superceded by signature. Read more at
https://docs.python.org/3/library/inspect.html#inspect.getfullargspec
https://docs.python.org/3/library/inspect.html#introspecting-callables-with-the-signature-object

Signature() has a keyword-only option follow_wrapped=True. The new patch uses this default. I suspect that this is relevant to the blogger's concern, but I don't know which value the blogger would want. Either way, I would not switch unless convinced that False is the best choice for beginning users. Any such discussion would be a new issue.

Hm... now I see that link is very misleading... it opens a blog post, and then only a few seconds later jumps to the comment. So now I'll paste the comment here, to avoid misunderstandings:

Python3.6.0 (now in 2017). I start IDLE, and write

>>> def deco(f):
        import functools
        @functools.wraps(f)
        def ret(*args, **kw):
                return f(*args, **kw)
        return ret

>>> @deco
def f(n, k):
        return n + k*2

>> f(

And get a tooltip (*args, **kw) instead of (n, k). I guess this is a bug.

This is much clearer. What you got is a limitation of getfullargspec relative to signature (see the links). In this case, you want follow_wrapped=True, to get the specific signature of the wrapped function instead of the generic signature of the wrapper*. With the patch, the tip in 3.6 and 3.7 is '(n, k)'. So yes, this is one of the changes consequent on the switch.

>>> str(inspect.signature(f,follow_wrapped=False))
'(*args, **kw)'

Navigation Menu

Repository navigation

Idle: Use inspect.signature for calltips #64102

13 remaining items

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Idle: Use inspect.signature for calltips #64102

Description

Activity

terryjreedy commented on Dec 6, 2013

terryjreedy commented on Jan 22, 2014

terryjreedy commented on Mar 17, 2014

terryjreedy commented on May 27, 2014

terryjreedy commented on Feb 1, 2015

terryjreedy commented on Feb 1, 2015

terryjreedy commented on Feb 1, 2015

terryjreedy commented on Feb 25, 2017

mlouielu commented on Apr 30, 2017

terryjreedy commented on May 2, 2017

mlouielu commented on May 2, 2017

13 remaining items

terryjreedy commented on May 4, 2017

serhiy-storchaka commented on May 4, 2017

terryjreedy commented on May 4, 2017

serhiy-storchaka commented on May 5, 2017

mlouielu commented on May 5, 2017

mlouielu commented on May 5, 2017

terryjreedy commented on May 5, 2017

vedgar commented on Aug 10, 2017

terryjreedy commented on Aug 10, 2017

terryjreedy commented on Aug 10, 2017

terryjreedy commented on Aug 10, 2017

vedgar commented on Aug 10, 2017

terryjreedy commented on Aug 10, 2017

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions