Improve help() by making typing.overload() information accessible at runtime

@gvanrossum

BPO	45100
Nosy	@gvanrossum, @rhettinger, @ronaldoussoren, @JelleZijlstra, @TeamSpen210, @sobolevn, @Fidget-Spinner, @AlexWaygood, @DiddiLeija
PRs	gh-89263: Add typing.get_overloads #31716

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2021-09-04.18:03:28.637>
labels = ['3.11']
title = 'Improve help() by making typing.overload() information accessible at runtime'
updated_at = <Date 2022-03-28.22:36:21.459>
user = 'https://github.com/rhettinger'

bugs.python.org fields:

activity = <Date 2022-03-28.22:36:21.459>
actor = 'gvanrossum'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = []
creation = <Date 2021-09-04.18:03:28.637>
creator = 'rhettinger'
dependencies = []
files = []
hgrepos = []
issue_num = 45100
keywords = ['patch']
message_count = 14.0
messages = ['401052', '401074', '401080', '401094', '401097', '413677', '413678', '413684', '413685', '413695', '413751', '414561', '414566', '416226']
nosy_count = 9.0
nosy_names = ['gvanrossum', 'rhettinger', 'ronaldoussoren', 'JelleZijlstra', 'Spencer Brown', 'sobolevn', 'kj', 'AlexWaygood', 'DiddiLeija']
pr_nums = ['31716']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = None
url = 'https://bugs.python.org/issue45100'
versions = ['Python 3.11']

@overload

We could make my proposed overload registry more reusable by putting it in a different module, probably functools. (Another candidate is inspect, but inspect.py imports functools.py, so that would make it difficult to use the registry for functools.singledispatch.)

We could then bill it as a "variant registry", with an API like this:

def register_variant(key: str, variant: Callable) -> None: ...
def get_variants(key: str) -> list[Callable]: ...
def get_key_for_callable(callable: Callable) -> str | None: ...

@overload could then call register_variant() to register each overload, and code that wants a list of overloads (pydoc, inspect.signature, runtime type checkers) could call get_variants().

get_key_for_callable() essentially does f"{callable.__qualname__}.{callable.__name__}", but returns None for objects it can't handle. It will also support at least classmethods and staticmethods.

I will prepare a PR implementing this idea.

The latest plan sounds good to me. I have some Thoughts on the proposed API, but it will be easier to express those as part of a PR review. Looking forward to seeing the PR!

Looks like there may be a new plan where we solve a smaller problem (overloads) in the context of typing only.

After #31716 is merged, I'd like to have a stab at writing a PR to have overloads shown in the output of help(). For now, it will probably be easiest to do that without tinkering with inspect.signature(), and instead only making changes to pydoc -- I have a few ideas of how to do that.

gh-89263: Add typing.get_overloads (GH-31716)

I was just playing around with the new get_overloads function, and it appears that it doesn't work if for functions defined in the interactive shell. This seems unfortunate -- it certainly took me by surprise.

C:\Users\alexw\coding\cpython>python
Running Debug|x64 interpreter...
Python 3.11.0a7+ (main, Apr 14 2022, 10:41:31) [MSC v.1931 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from typing import *
>>> @overload
... def foo(arg: int) -> str: ...
...
>>> @overload
... def foo(arg: str) -> int: ...
...
>>> def foo(arg: int | str) -> str | int: ...
...
>>> get_overloads(foo)
[<function foo at 0x000001D030B0C9F0>]
>>> import inspect
>>> inspect.signature(_[0])
<Signature (arg: str) -> int>

This is because all functions defined in the interactive shell have the same line-number set on their code object, and the new overloads registry uses the co_firstlineno attribute of the code attribute to register overloads.

>>> def foo(): ...
...
>>> def bar(): ...
...
>>> foo.__code__.co_firstlineno == bar.__code__.co_firstlineno == 1
True

I initially thought that this would cause problems for using get_overloads to improve the output of help(). Maybe it's not so much of a problem, though, since most people won't be calling help() on functions defined in the interactive shell -- they'll be calling it on functions that they import from within the interactive shell. Moreover, using @overload for a function defined in the interactive shell is pretty weird, since a type checker isn't much use to you in the interactive shell.

It's a little unfortunate because it makes it harder for people to experiment with get_overloads() in the REPL. But I agree that it's not a big problem for people using help() to introspect library functions.

Since @overload is a static typing feature, why would you be typing it in the REPL in the first place?

Since @overload is a static typing feature, why would you be typing it in the REPL in the first place?

If you read about get_overloads() in the What's New and want to try it out for yourself, you might write some overloads in the REPL to see it in action. Looks like that's essentially what happened to Alex.

Since @overload is a static typing feature, why would you be typing it in the REPL in the first place?

Yeah, this is why I think it might actuallly not be that much of a big deal, as I said above.

If you read about get_overloads() in the What's New and want to try it out for yourself, you might write some overloads in the REPL to see it in action. Looks like that's essentially what happened to Alex.

Yup. Experimenting with stuff in the REPL is generally a part of my process when I write patches for CPython. I was tinkering with modifying pydoc to use get_overloads, and was trying things out in the REPL. I was surprised when things didn't work as expected, and it took me a minute or two to figure out that the reason for the unexpected behaviour was that I was in the interactive shell.

py-typing-extensions: Update to 4.3.0

@JelleZijlstra

@JelleZijlstra The issue seems to be solved. We can close the issue.

We added typing.get_overloads, but I don't think we ended up making help() look at this information yet.

py-typing-extensions: Update to 4.3.0

Navigation Menu

Repository navigation

Improve help() by making typing.overload() information accessible at runtime #89263

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Improve help() by making typing.overload() information accessible at runtime #89263

Description

Activity

JelleZijlstra commented on Mar 5, 2022

AlexWaygood commented on Mar 5, 2022

gvanrossum commented on Mar 28, 2022

AlexWaygood commented on Apr 16, 2022

AlexWaygood commented on Apr 17, 2022

AlexWaygood commented on Apr 17, 2022

JelleZijlstra commented on Apr 17, 2022

gvanrossum commented on Apr 17, 2022

JelleZijlstra commented on Apr 17, 2022

AlexWaygood commented on Apr 17, 2022

furkanonder commented on May 9, 2023

JelleZijlstra commented on May 9, 2023

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions