Repository navigation
typing.get_type_hints with TYPE_CHECKING imports / getting hints for single argument #87629
Description
Activity
The-Compiler commented
on Mar 10, 2021 The-CompilermannequinMannequinAuthorConsider a file such as:
# from __future__ import annotations from typing import TYPE_CHECKING, Union, get_type_hints if TYPE_CHECKING: import types def fun(a: 'types.SimpleNamespace', b: Union[int, str]): pass print(fun.__annotations__) print(get_type_hints(fun))
When running this, typing.get_type_hints fails (as you would expect):
Traceback (most recent call last): File "/home/florian/tmp/x.py", line 11, in <module> print(get_type_hints(fun)) File "/usr/lib/python3.9/typing.py", line 1449, in get_type_hints value = _eval_type(value, globalns, localns) File "/usr/lib/python3.9/typing.py", line 283, in _eval_type return t._evaluate(globalns, localns, recursive_guard) File "/usr/lib/python3.9/typing.py", line 539, in _evaluate eval(self.__forward_code__, globalns, localns), File "<string>", line 1, in <module> NameError: name 'types' is not definedHowever, in my case I'm not actually interested in the type of 'a', I only need the type for 'b'. Before Python 3.10 (or the __future__ import), I can do so by getting it from __annotations__ directly.
With Python 3.10 (or the __future__ import), this doesn't seem to be possible anymore - I'd need to either evaluate the 'Union[int, str]' annotation manually (perhaps calling into private typing.py functions), or maybe work around the issue by passing some magical dict-like object as local/globals which ignores the NameError. Both of those seem suboptimal.
Thus, I'd like a way to either:
- Ignore exceptions in get_type_hints and instead get something like a typing.Unresolvable['types.SimpleNamespace'] back
- Have something like a typing.get_argument_type_hints(fun, 'b') instead, allowing me to get the arguments one by one rather than resolving the whole thing
- Have a public API to resolve a string type annotation (i.e. the equivalent of
typing._eval_type)
Maybe return the original string?
The-Compiler commented
on Mar 12, 2021 The-CompilermannequinMannequinAuthorI suppose returning the string unchanged would work as well. However, "Errors should never pass silently." :)
Perhaps that with a ignore_nameerror=True or switch or so would work?
If we add a new flag to ignore errors it's difficult to write code that
works in 3.9 as well. And given the use case I have doubts that "Errors
should never pass silently" is really the right Zen line to focus on. I'd
rather go for "Simple is better than complex" or "practicality beats
purity".The-Compiler commented
on Mar 15, 2021 The-CompilermannequinMannequinAuthorFair points.
As an aside, I'm also wondering how inspect.Parameter.annotation should interact with the changes in Python 3.10? That used to be the canonical way (as far as I'm aware) of getting a single argument's type annotation (other than getting it from __annotations__ manually), but with PEP-563 now would always return a (probably not very useful?) string.
Please open a separate issue for that.
IIUC inspect.signature auto-resolves string annotations to typing.ForwardRef internally from 3.10 onwards. It's mentioned in the what's new for PEP-563 https://docs.python.org/3.10/whatsnew/3.10.html#pep-563-postponed-evaluation-of-annotations-becomes-default
If it fails, it will just give the string. So the only place where inspect.signature might start giving you different output is if you previously defined a function like so:
def foo(a: 'MyType'): ...
And you expected inspect.signature.paramters to be
[<Parameter "a: 'MyType'">](the string). However if MyType is in globals()/locals(), you'll instead get[<Parameter "a: MyType">]in 3.10.FWIW someone already reported that in bpo-43355.
The-Compiler commented
on Mar 15, 2021 The-CompilermannequinMannequinAuthorAh, I wasn't aware of that, thanks for the pointer! So what inspect does internally is:
def _get_type_hints(func, **kwargs): try: return typing.get_type_hints(func, **kwargs) except Exception: # First, try to use the get_type_hints to resolve # annotations. But for keeping the behavior intact # if there was a problem with that (like the namespace # can't resolve some annotation) continue to use # string annotations return func.__annotations__
Which means there's even some "prior art" there already falling back to a string when the annotation couldn't be resolved. Doing so in typing.get_type_hints on a per-argument basis would thus also make inspect more consistent:
Right now,
print(repr(inspect.signature(fun).parameters['b'].annotation))in my example returns a string, but when changing the annotation for
a, the returned annotation forbis now magically atyping.Unionobject.(I personally would indeed expect inspect to resolve those annotations, but yeah, let's keep that in bpo-43355.)
Hey Larry, it would seem that PEP-649 as currently specified would make it impossible to access annotations via the inspect module in cases where x.__annotations__ raises (because one of the annotations references an undefined variable).
I really think that we need *some* way of accessing partial annotations. Even just leaving the failing key out of __annotations__ (but keeping other keys if their annotation works) would be better than failing to return an __annotations__ dict at all.
Hey Larry, it would seem that PEP-649 as currently specified would make it impossible to access annotations via the inspect module in cases where x.__annotations__ raises (because one of the annotations references an undefined variable).
That's true. If PEP-649 is accepted, inspect.signature() might want to catch NameError when examining __annotations__ on the object. Though I'm not sure what it should do when the exception is raised.
I really think that we need *some* way of accessing partial annotations. Even just leaving the failing key out of __annotations__ (but keeping other keys if their annotation works) would be better than failing to return an __annotations__ dict at all.
Unfortunately I don't agree--"errors should never pass silently." Silently omitting the failed annotation seems like it would be a bad experience. What if the value you needed from the annotation was the one that was omitted? Now you have a mystery obscuring your existing problem.
There is a PR against PEP-649 specifically to suppress NameErrors:
https://github.com/larryhastings/co_annotations/pull/3I haven't merged the PR as I don't agree with it.
If the NameError were to leave behind a marker wrapping the string, that might allow the best of both worlds -- errors don't pass *silently* (an error value is returned) and you can still inspect the other annotations.
It would also cause the code generated for the annotations function to balloon--the calculation of every value would have to be wrapped in a try/except, and for what I assume is an obscure use case. I'm already getting some pushback on the code generated with PEP-649 as it is.
My goal in designing PEP-649 was to take stock semantics and time-shift the evaluation of the annotations, adding as little opinion as possible. I think catching NameErrors is too opinionated, much less catching every plausible error.
Still, the approach is mentioned in PEP-649. I assume the Steering Committee rules is aware this option exists. If they rule in favor of adding PEP-649, but stipulate that this feature is necessary, I will of course implement it.
I thought Jelle’s solution would avoid code bloat — a dedicated opcode. It can do whatever is decided.
I admit I hadn't looked that closely at Jelle's PR. You're right, its effects on code size should be minimal.
If I'm reading it correctly, Jelle's PR would suppress NameError by replacing the failed value with a new "AnnotationName" object. It wouldn't handle any other exceptions (e.g. ValueError).
Also, as currently implemented it doesn't permit getting attributes or indexing into the AnnotationName--if "o" is the object we failed to look up, "o.value" and "o[1]" would raise exceptions. The AnnotationName could be extended with __getattr__ and __getitem__, but this makes it even more opinionated.
Also, the PR only adds LOAD_ANNOTATION_GLOBAL; it would presumably also need LOAD_ANNOTATION_NAME and LOAD_ANNOTATION_CLASSDEREF. (I don't think we'd need LOAD_ANNOTATION_LOCAL.)
I'll file these comments on the PR.
antonagestam commented
on Mar 20, 2022 antonagestammannequinMannequinAs a consumer of
get_type_hints()I think it'd be valuable to even have partially resolved types. My use case is that I provide anAnnotatedalias with a marker, and all I care about when inspecting user type hints is whether or not the arguments of anAnnotatedtype contains my marker object. So ideally the fallback to an unresolved string or a sentinel object such as the proposedtyping.Unresolvableshould happen at the "lowest resolvable level" so that what can be resolved isn't lost.By example, I'm saying that I think that this code:
marker = object() def dec(cls): print(get_type_hints(cls)) return cls @dec class A(abc.ABC): forward: Annotated[B, marker] class B: ...
Should produce:
{"forward": Annotated[Unresolvable["B"], marker]}I guess this would apply in situations where for instance a part of a union isn't resolvable too. If we have a union A|B where A is resolvable and B isn't, it should be resolved to:
A | Unresolvable["B"]And not to:
Unresolvable["A | B"](I think for this perspective it's irrelevant whether unresolved types have a sentinel type or are just represented as strings).
(Here's the library that's my use case for the curious: https://github.com/antonagestam/abcattrs)
Reacted by ebc-conscia
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields: