singledispatchmethod raises an error when relying on a forward declaration

@gvanrossum

BPO	41987
Nosy	@gvanrossum, @glyph, @ambv, @ethanhs, @joernheissler, @isidentical, @mental32, @ryansobol, @AlexWaygood
PRs	bpo-41987: Fix unnecessary evaluation of return type forward declaratons in singledispatch. #23216

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2020-10-09.20:11:43.528>
labels = ['type-bug', 'library', '3.9', '3.10']
title = 'singledispatchmethod raises an error when relying on a forward declaration'
updated_at = <Date 2022-01-11.09:46:19.220>
user = 'https://github.com/glyph'

bugs.python.org fields:

activity = <Date 2022-01-11.09:46:19.220>
actor = 'AlexWaygood'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2020-10-09.20:11:43.528>
creator = 'glyph'
dependencies = []
files = []
hgrepos = []
issue_num = 41987
keywords = ['patch']
message_count = 16.0
messages = ['378346', '378350', '378353', '378361', '379896', '379911', '380797', '380799', '380800', '380803', '380832', '380843', '380845', '380852', '409572', '410269']
nosy_count = 9.0
nosy_names = ['gvanrossum', 'glyph', 'lukasz.langa', 'ethan smith', 'joernheissler', 'BTaskaya', 'mental', 'ryansobol', 'AlexWaygood']
pr_nums = ['23216']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue41987'
versions = ['Python 3.9', 'Python 3.10']

I'm not an expert on singledispatch. It seems the get_type_hints() call is present in 3.8 as well.

Could you investigate and find a root cause? Then maybe we can fix it. (If you come up with a PR that would be very much appreciated.)

I spent some time debugging this looking for the root cause.

I think it looks like the recursion check in ForwardRef._evaluate() fails to trigger. At some point recursive_guard is a frozen set containing "'Integer'" (i.e. a string whose first and last character are single quotes, while self.__forward_arg__ is 'Integer' (i.e. a string that does not contain quotes).

I'm running out of time for the rest of the investigation, so feel free to confirm this and go down the rabbit hole from there...

Interesting! thanks for the hint guido I'll try to investigate further.

This sounds like a regression (to me at least), in that case should a separate issue & patch PR be opened on the bpo or should this issue be used instead?

Keep this issue.

FWIW here's a minimal demo:

from __future__ import annotations
from typing import get_type_hints

class C:
    def func(self, a: "C"):
        pass

    print(get_type_hints(func))

In 3.8 this prints

{'a': ForwardRef('C')}

while in 3.9 it raises NameError:

Traceback (most recent call last):
  File "C:\Users\gvanrossum\cpython\t.py", line 4, in <module>
    class C:
  File "C:\Users\gvanrossum\cpython\t.py", line 8, in C
    print(get_type_hints(func))
  File "C:\Python39\lib\typing.py", line 1386, in get_type_hints
    value = _eval_type(value, globalns, localns)
  File "C:\Python39\lib\typing.py", line 254, in _eval_type
    return t._evaluate(globalns, localns, recursive_guard)
  File "C:\Python39\lib\typing.py", line 497, in _evaluate
    self.__forward_value__ = _eval_type(
  File "C:\Python39\lib\typing.py", line 254, in _eval_type
    return t._evaluate(globalns, localns, recursive_guard)
  File "C:\Python39\lib\typing.py", line 493, in _evaluate
    eval(self.__forward_code__, globalns, localns),
  File "<string>", line 1, in <module>
NameError: name 'C' is not defined

So the biggest difference I see is that ForwardRef._evaluate() has grown a recursive_guard argument in 3.9. This makes me think that in 3.8, only one level of evaluation was happening, and in 3.8, we keep evaluating until we don't see a string or ForwardRef.

The specific examples all happen at a point where the forward ref "C" cannot be resolved at all yet (since they're happening *in the class body*).

Possibly the best way out is to treat unresolved references differently, and just return the ForwardRef to the caller -- after all this is what the example does in 3.8.

Does anyone know why the treatment of unresolved references was changed in 3.9?

Also, I'm a bit puzzled about something from the previously mentioned Integer class and its use of __future__.annotations.

Why is it possible to declare an Integer return type for the add() method, but only possible to declare an "Integer" forward reference for the _() method?

Does anyone know why the treatment of unresolved references was changed in 3.9?

Probably to prepare for 3.10, where from _future__ import annotations is the default.

Also, I'm a bit puzzled about something from the previously mentioned Integer class and its use of __future__.annotations.

Why is it possible to declare an Integer return type for the add() method, but only possible to declare an "Integer" forward reference for the _() method?

I don't know -- you might want to look through the source code of singledispatch. Maybe the flow through the initial decorator is different than the flow through register().

Hello!

git-bisect points at
https://bugs.python.org/issue41341
#21553

It breaks both the examples from
https://bugs.python.org/issue41987#msg379896 and
https://bugs.python.org/issue41987#msg380803

Thanks for the bisection. It's not surprising that that's the culprit, and in other situations that's the right thing to do. I'm not sure how to address this without breaking other stuff -- maybe leave the ForwardRef if evaluating it doesn't work? But that's likely to have other subtle side effects -- we still want simple typos (or other reasons why a reference is legitimately broken) to go unchecked. Maybe singledispatch can catch the error and fall back on looking at bare __annotations__?

pythonGH-86153: get_type_hints now parses forward reference properly

pythonGH-86153: Created a test for get_type_hints based on GVR's exam…

This can probably be closed as it's not an issue in Py 3.10+

bintime: Work around python/cpython#86153

Add bintime submodule (#73)

@isidentical

This can probably be closed as it's not an issue in Py 3.10+

I can still reproduce the issue, so I can't confirm this. What exactly changed in Python 3.10?

First, regarding the problem discussed in #86153 (comment) and further on: this remained unfixed in 3.9, 3.10, 3.11, 3.12 and 3.13. It was fixed by PEP 749, precisely by 7b7b90d (GH-119891):

t.py:

from __future__ import annotations
from functools import singledispatchmethod

class Integer:
    @singledispatchmethod
    def add(self, other: object) -> Integer:
        raise NotImplementedError(f"Unsupported type {type(other)}")

    @add.register
    def _(self, other: int) -> "Integer":
        ...

print("ok!")

❯ uvx every-python run 7b7b90d~ python t.py |& tail -n 1
NameError: name 'Integer' is not defined

❯ uvx every-python run 7b7b90d python t.py |& tail -n 1
ok!

Therefore, this is no longer a bug to fix.

The original error reported by the OP still reproduces on 3.15:

from __future__ import annotations
from functools import singledispatchmethod

class Comparable:
    @singledispatchmethod
    def compare(self, arg: object):
        raise NotImplementedError("what")

    @compare.register
    def _(self, arg: Comparable):
        return "somewhat similar"

with a lovely error message:

Traceback (most recent call last):
  File "/home/bswck/Python/cpython/t.py", line 33, in <module>
    class Comparable:
    ...<6 lines>...
            return "somewhat similar"
  File "/home/bswck/Python/cpython/t.py", line 38, in Comparable
    @compare.register
     ^^^^^^^^^^^^^^^^
  File "/home/bswck/Python/cpython/Lib/functools.py", line 1030, in register
    return self.dispatcher.register(cls, func=method)
           ~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
  File "/home/bswck/Python/cpython/Lib/functools.py", line 974, in register
    raise TypeError(
    ...<2 lines>...
    )
TypeError: Invalid annotation for 'arg'. ForwardRef('Comparable') is an unresolved forward reference.

(Without from __future__ import annotations on top we also get nearly the same error).

It's not a bug report, but a feature request, as previously noted by @isidentical in #86153 (comment) and by @gvanrossum in #86153 (comment). I'm relabeling this issue accordingly.

As @isidentical noted, the reason why this didn't work in the first place is that @register evaluates the annotation immediately, and at the time of evaluation the class body is still executing to complete the class namespace; the final class object is not available in any way yet. There is no way to get to the class object, because it simply doesn't exist at this time.

One simple solution that comes to my mind is delaying registrations entirely by moving them to __set_name__ of singledispatchmethod, when the class object is ready.
singledispatchmethod.register() would simply queue registrations and __set_name__ would flush it to the singledispatch registry in the context of the finalized class.

Doing this to all registrations could break .dispatch calls inside the class body, but that seems like poor programming to me.
If that is a concern though, we could only queue unresolvable registrations, but that gives us a complicated flow to reason about + implementation complexity.

Revisiting this, I think it's not worth it.

This is easy and works:

from __future__ import annotations
from functools import singledispatchmethod

class Comparable:
    @singledispatchmethod
    def compare(self, arg: object):
        raise NotImplementedError("what")

    def callback(self, arg: Comparable):
        return "somewhat similar"
    
Comparable.compare.register(Comparable.callback)

The cost of changing the assumptions that singledispatch was built on is IMO too high to support this one edge case.

@johnslavik

@johnslavik the suggested workaround:

from __future__ import annotations
from functools import singledispatchmethod

class Comparable:
    @singledispatchmethod
    def compare(self, arg: object):
        raise NotImplementedError("what")

    def callback(self, arg: Comparable):
        return "somewhat similar"
    
Comparable.compare.register(Comparable.callback)

does work at runtime, but results in a mypy error: error: "Callable[..., Any]" has no attribute "register" [attr-defined]

Navigation Menu

Repository navigation

singledispatchmethod raises an error when relying on a forward declaration #86153

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

singledispatchmethod raises an error when relying on a forward declaration #86153

Description

Activity

gvanrossum commented on Oct 30, 2020

gvanrossum commented on Nov 11, 2020

mental32 commented on Nov 12, 2020

gvanrossum commented on Nov 12, 2020

gvanrossum commented on Nov 12, 2020

gvanrossum commented on Nov 12, 2020

ryansobol commented on Nov 12, 2020

ryansobol commented on Nov 12, 2020

gvanrossum commented on Nov 12, 2020

joernheissler commented on Jan 3, 2022

gvanrossum commented on Jan 11, 2022

ashb commented on Jan 13, 2025

johnslavik commented on Jan 6, 2026

johnslavik commented on Jan 20, 2026

kyleaoman commented on May 20, 2026

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions