functools._lru_cache_wrapper should be a descriptor class, not a callable

I'm not entirely sure this is a bug in the type stub. It depends on the interpretation of ParamSpec when used with methods.

This is related to the discussion here and this bug report filed in the pyright issue tracker.

Could this be a use case for Concatenate?

Leave the stub for _lru_cache_wrapper as it is:

from typing import Generic, TypeVar, ParamSpec, Callable

_P = ParamSpec("_P")
_T = TypeVar("_T")

class _lru_cache_wrapper(Generic[_P, _T]):  # type: ignore
    __wrapped__: Callable[_P, _T]  # type: ignore
    def __call__(self, *args: _P.args, **kwargs: _P.kwargs) -> _T: ...  # type: ignore

But then change the stub for lru_cache/cache to the following:

import sys
from typing import overload, Concatenate, Any

if sys.version_info >= (3, 8):
    @overload
    def lru_cache(maxsize: int | None = ..., typed: bool = ...) -> Callable[[Callable[Concatenate[Any, _P], _T]], _lru_cache_wrapper[_P, _T]]: ...  # type: ignore
    @overload
    def lru_cache(maxsize: Callable[Concatenate[Any, _P], _T], typed: bool = ...) -> _lru_cache_wrapper[_P, _T]: ...  # type: ignore

else:
    def lru_cache(maxsize: int | None = ..., typed: bool = ...) -> Callable[[Callable[Concatenate[Any, _P], _T]], _lru_cache_wrapper[_P, _T]]: ...  # type: ignore

if sys.version_info >= (3, 9):
    def cache(__user_function: Callable[Concatenate[Any, _P], _T]) -> _lru_cache_wrapper[_P, _T]: ...  # type: ignore

I think cache and lru_cache can be used with either methods or non-method functions, right? Wouldn't that approach break for non-methods?

Wouldn't that approach break for non-methods?

Good point! Are you able to flesh out the rewrite-it-as-a-descriptor proposal a little?

I was hoping that someone who is more familiar with how these calls are meant to work could suggest a solution.

How about this?

import sys
from typing import Generic, TypeVar, ParamSpec, Callable, Concatenate, overload, Type

_P = ParamSpec("_P")
_T = TypeVar("_T")
_S = TypeVar("_S")

class _lru_cache_wrapper(Generic[_S, _P, _T]):  # type: ignore
    __wrapped__: Callable[Concatenate[_S, _P], _T]  # type: ignore
    def __call__(self, __obj: _S, *args: _P.args, **kwargs: _P.kwargs) -> _T: ...  # type: ignore
    def __get__(self, __obj: _S, __type: Type[_S] | None = ...) -> Callable[_P, _T]: ...

if sys.version_info >= (3, 8):
    @overload
    def lru_cache(maxsize: int | None = ..., typed: bool = ...) -> Callable[[Callable[Concatenate[_S, _P], _T]], _lru_cache_wrapper[_S, _P, _T]]: ...  # type: ignore
    @overload
    def lru_cache(maxsize: Callable[Concatenate[_S, _P], _T], typed: bool = ...) -> _lru_cache_wrapper[_S, _P, _T]: ...  # type: ignore

else:
    def lru_cache(maxsize: int | None = ..., typed: bool = ...) -> Callable[[Callable[Concatenate[_S, _P], _T]], _lru_cache_wrapper[_S, _P, _T]]: ...  # type: ignore

if sys.version_info >= (3, 9):
    def cache(__user_function: Callable[Concatenate[_S, _P], _T]) -> _lru_cache_wrapper[_S, _P, _T]: ...  # type: ignore

@mrkmndz

I don't think that works. This doesn't work with non-method functions that take no parameters.

@cache
def func() -> int:
    return 0

Maybe this isn't the right approach?

Perhaps someone who contributed to PEP 612 could weigh in? @mrkmndz

The more we discuss this, the more I agree that this might not be something that can be solved in the stubs (and yes, I'm obviously out of my depth here, so someone more qualified should weigh in). But, well, here's a third attempt at typing lru_cache/cache:

import sys
from typing import Generic, TypeVar, ParamSpec, Callable, Concatenate, overload, NoReturn, Protocol

_P1 = ParamSpec("_P1")
_P2 = ParamSpec("_P2")
_R = TypeVar("_R")
_S = TypeVar("_S")

class _WrapperReturnBase(Protocol[_P1, _R]):
    __wrapped__: Callable[_P1, _R]
    def__call__(self, *args: _P1.args, **kwargs: _P1.kwargs) -> _R: ...
    def cache_info(self) -> _CacheInfo : ...
    def cache_clear(self) -> None: ...

class _WrapperReturnNoArgs(_WrapperReturnBase[[], _R], Protocol[_R]):
    def __get__(self, __obj: _S, __type: Type[_S] | None = ...)  -> Callable[..., NoReturn]: ...

class _WrapperReturnOneArg(_WrapperReturnBase[[_S], _R], Protocol[_S, _R]):
    def __get__(self, __obj: _S, __type: Type[_S] | None = ...) -> _WrapperReturnNoArgs[_R]: ...

class _WrapperReturnMultipleArgs(_WrapperReturnBase[Concatenate[_S, _P2], _R], Protocol[_S, _P2, _R]):
    def __get__(self, __obj: _S, __type: Type[_S] | None = ...) -> _WrapperReturnMultipleArgs[_S, _P2, _R] | _WrapperReturnOneArg[_S, _R]: ...

class _lru_cache_wrapper(Generic[_S, _P, _R]):
    @overload
    def __call__(self, __func: Callable[[], _R]) -> WrapperReturnNoArgs[_R]: ...
    @overload
    def __call__(self, __func: Callable[[_S], _R]) -> _WrapperReturnOneArg[_S, _R]: ...
    @overload
    def __call__(self, __func: Callable[Concatenate[_S, _P2], _R]) -> _WrapperReturnMultipleArgs[_S, _P2, _R]: ...

if sys.version_info >= (3, 8):
    @overload
    def lru_cache(maxsize: int | None = ..., typed: bool = ...) -> _lru_cache_wrapper[_S, _P, _R]: ...
    @overload
    def lru_cache(maxsize: Callable[[], _R], typed: bool = ...) -> _WrapperReturnNoArgs[_R]: ...
    @overload
    def lru_cache(maxsize: Callable[[_S], _R], typed: bool = ...) -> _WrapperReturnOneArg[_S, _R]: ...
    @overload
    def lru_cache(maxsize: Callable[Concatenate[_S, _P2], _R], typed: bool = ...) -> _WrapperReturnMultipleArgs[_S, _P2, _R]: ...

else:
    def lru_cache(maxsize: int | None = ..., typed: bool = ...) -> _lru_cache_wrapper[_S, _P, _R]: ...

if sys.version_info >= (3, 9):
    @overload
    def cache(__user_function: Callable[[], _R]) -> _WrapperReturnNoArgs[_R]: ...
    @overload
    def cache(__user_function: Callable[[_S], _R]) ->  _WrapperReturnOneArg[_S, _R]: ...
    @overload
    def cache(__user_function: Callable[Concatenate[_S, _P2], _R]) -> _WrapperReturnMultipleArgs[_S, _P2, _R]: ...

Revert "lru_cache preserves signature of wrapped function (python#6221)"

I also encountered this issue when adding ParamSpec support to mypy. I'm proposing to revert (#6356) the original change until we have a solution that works with all ParamSpec implementations or we at least have clarity about how PEP 612 should be interpreted here.

Revert "lru_cache preserves signature of wrapped function (#6221)" (#…

Basic support for ParamSpec type checking (#11594)

Basic support for ParamSpec type checking (python#11594)

I came up with a proposal that aims to obviate the need for using descriptor classes for this, by providing a new mechanism to declare function types with attributes that will then be bound as methods by the usual mechanism if they are defined inside a class: https://mail.python.org/archives/list/typing-sig@python.org/thread/35FTOYUG2IPCRIIH3MQKEVV7XW3V7ASB/

What's the latest on this?

I am going to close this as "inexpressible" for now. There have been multiple tries to fix this, but none have been successful. It seems that we need changes to Python's type system before we can solve this.

Navigation Menu

Repository navigation

functools._lru_cache_wrapper should be a descriptor class, not a callable #6347

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

functools._lru_cache_wrapper should be a descriptor class, not a callable #6347

Description

Activity

AlexWaygood commented on Nov 20, 2021

erictraut commented on Nov 20, 2021

AlexWaygood commented on Nov 20, 2021

erictraut commented on Nov 20, 2021

AlexWaygood commented on Nov 20, 2021

erictraut commented on Nov 22, 2021

AlexWaygood commented on Nov 22, 2021

JukkaL commented on Nov 22, 2021

tmke8 commented on Mar 4, 2023

tamird commented on Feb 23, 2024

srittau commented on Sep 23, 2025

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions