:mod:`!functools` --- Higher-order functions and operations on callable objects

.. module:: functools

:synopsis: Higher-order functions and operations on callable objects.

**Source code:** :source:`Lib/functools.py`

.. testsetup:: default

import functools

from functools import *

The :mod:`!functools` module is for higher-order functions: functions that act on

or return other functions. In general, any callable object can be treated as a

function for the purposes of this module.

The :mod:`!functools` module defines the following functions:

.. decorator:: cache(user_function)

Simple lightweight unbounded function cache. Sometimes called

`"memoize" <https://en.wikipedia.org/wiki/Memoization>`_.

Returns the same as ``lru_cache(maxsize=None)``, creating a thin

wrapper around a dictionary lookup for the function arguments. Because it

never needs to evict old values, this is smaller and faster than

:deco:`lru_cache` with a size limit.

For example::

The cache is threadsafe so that the wrapped function can be used in

multiple threads. This means that the underlying data structure will

remain coherent during concurrent updates.

It is possible for the wrapped function to be called more than once if

another thread makes an additional call before the initial call has been

completed and cached.

Call-once behavior is not guaranteed because locks are not held during the

function call. Potentially another call with the same arguments could

occur while the first call is still running.

.. versionadded:: 3.9

.. decorator:: cached_property(func)

Transform a method of a class into a property whose value is computed once

and then cached as a normal attribute for the life of the instance. Similar

to :deco:`property`, with the addition of caching. Useful for expensive

computed properties of instances that are otherwise effectively immutable.

Example::

The mechanics of :deco:`cached_property` are somewhat different from

:deco:`property`. A regular property blocks attribute writes unless a

setter is defined. In contrast, a *cached_property* allows writes.

The *cached_property* decorator only runs on lookups and only when an

attribute of the same name doesn't exist. When it does run, the

*cached_property* writes to the attribute with the same name. Subsequent

attribute reads and writes take precedence over the *cached_property*

method and it works like a normal attribute.

The cached value can be cleared by deleting the attribute. This

allows the *cached_property* method to run again.

The *cached_property* does not prevent a possible race condition in

multi-threaded usage. The getter function could run more than once on the

same instance, with the latest run setting the cached value. If the cached

property is idempotent or otherwise not harmful to run more than once on an

instance, this is fine. If synchronization is needed, implement the necessary

locking inside the decorated getter function or around the cached property

access.

Note, this decorator interferes with the operation of :pep:`412`

key-sharing dictionaries. This means that instance dictionaries

can take more space than usual.

Also, this decorator requires that the ``__dict__`` attribute on each instance

be a mutable mapping. This means it will not work with some types, such as

metaclasses (since the ``__dict__`` attributes on type instances are

read-only proxies for the class namespace), and those that specify

``__slots__`` without including ``__dict__`` as one of the defined slots

(as such classes don't provide a ``__dict__`` attribute at all).

If a mutable mapping is not available or if space-efficient key sharing is

desired, an effect similar to :deco:`cached_property` can also be achieved by

stacking :deco:`property` on top of :deco:`lru_cache`. See

:ref:`faq-cache-method-calls` for more details on how this differs from :deco:`cached_property`.

.. versionadded:: 3.8

.. versionchanged:: 3.12

Prior to Python 3.12, :deco:`!cached_property` included an undocumented lock to

ensure that in multi-threaded usage the getter function was guaranteed to

run only once per instance. However, the lock was per-property, not

per-instance, which could result in unacceptably high lock contention. In

Python 3.12+ this locking is removed.

.. function:: cmp_to_key(func)

Transform an old-style comparison function to a :term:`key function`. Used

with tools that accept key functions (such as :func:`sorted`, :func:`min`,

:func:`max`, :func:`heapq.nlargest`, :func:`heapq.nsmallest`,

:func:`itertools.groupby`). This function is primarily used as a transition

tool for programs being converted from Python 2 which supported the use of

comparison functions.

A comparison function is any callable that accepts two arguments, compares them,

and returns a negative number for less-than, zero for equality, or a positive

number for greater-than. A key function is a callable that accepts one

argument and returns another value to be used as the sort key.

Example::

For sorting examples and a brief sorting tutorial, see :ref:`sortinghowto`.

.. versionadded:: 3.2

.. decorator:: lru_cache(user_function)

lru_cache(maxsize=128, typed=False)

Decorator to wrap a function with a memoizing callable that saves up to the

*maxsize* most recent calls. It can save time when an expensive or I/O bound

function is periodically called with the same arguments.

The cache is threadsafe so that the wrapped function can be used in

multiple threads. This means that the underlying data structure will

remain coherent during concurrent updates.

It is possible for the wrapped function to be called more than once if

another thread makes an additional call before the initial call has been

completed and cached.

Since a dictionary is used to cache results, the positional and keyword

arguments to the function must be :term:`hashable`.

Distinct argument patterns may be considered to be distinct calls with

separate cache entries. For example, ``f(a=1, b=2)`` and ``f(b=2, a=1)``

differ in their keyword argument order and may have two separate cache

entries.

If *user_function* is specified, it must be a callable. This allows the

*lru_cache* decorator to be applied directly to a user function, leaving

the *maxsize* at its default value of 128::

If *maxsize* is set to ``None``, the LRU feature is disabled and the cache can

grow without bound.

If *typed* is set to true, function arguments of different types will be

cached separately. If *typed* is false, the implementation will usually

regard them as equivalent calls and only cache a single result. (Some

types such as *str* and *int* may be cached separately even when *typed*

is false.)

Note, type specificity applies only to the function's immediate arguments

rather than their contents. The scalar arguments, ``Decimal(42)`` and

``Fraction(42)`` are treated as distinct calls with distinct results.

In contrast, the tuple arguments ``('answer', Decimal(42))`` and

``('answer', Fraction(42))`` are treated as equivalent.

The wrapped function is instrumented with a :func:`!cache_parameters`

function that returns a new :class:`dict` showing the values for *maxsize*

and *typed*. This is for information purposes only. Mutating the values

has no effect.

.. method:: lru_cache.cache_info()

:no-typesetting:

To help measure the effectiveness of the cache and tune the *maxsize*

parameter, the wrapped function is instrumented with a :func:`!cache_info`

function that returns a :term:`named tuple` showing *hits*, *misses*,

*maxsize* and *currsize*.

.. method:: lru_cache.cache_clear()

:no-typesetting:

The decorator also provides a :func:`!cache_clear` function for clearing or

invalidating the cache.

The original underlying function is accessible through the

:attr:`__wrapped__` attribute. This is useful for introspection, for

bypassing the cache, or for rewrapping the function with a different cache.

The cache keeps references to the arguments and return values until they age

out of the cache or until the cache is cleared.

If a method is cached, the ``self`` instance argument is included in the

cache. See :ref:`faq-cache-method-calls`

An `LRU (least recently used) cache

<https://en.wikipedia.org/wiki/Cache_replacement_policies#Least_Recently_Used_(LRU)>`_

works best when the most recent calls are the best predictors of upcoming

calls (for example, the most popular articles on a news server tend to

change each day). The cache's size limit assures that the cache does not

grow without bound on long-running processes such as web servers.

In general, the LRU cache should only be used when you want to reuse

previously computed values. Accordingly, it doesn't make sense to cache

functions with side-effects, functions that need to create

distinct mutable objects on each call (such as generators and async functions),

or impure functions such as time() or random().

Example of an LRU cache for static web content::

Example of efficiently computing

`Fibonacci numbers <https://en.wikipedia.org/wiki/Fibonacci_number>`_

using a cache to implement a

`dynamic programming <https://en.wikipedia.org/wiki/Dynamic_programming>`_

technique::

.. versionadded:: 3.2

.. versionchanged:: 3.3

Added the *typed* option.

.. versionchanged:: 3.8

Added the *user_function* option.

.. versionchanged:: 3.9

Added the function :func:`!cache_parameters`

.. decorator:: total_ordering

Given a class defining one or more rich comparison ordering methods, this

class decorator supplies the rest. This simplifies the effort involved

in specifying all of the possible rich comparison operations:

The class must define one of :meth:`~object.__lt__`, :meth:`~object.__le__`,

:meth:`~object.__gt__`, or :meth:`~object.__ge__`.

In addition, the class should supply an :meth:`~object.__eq__` method.

For example::

.. note::

While this decorator makes it easy to create well behaved totally

ordered types, it *does* come at the cost of slower execution and

more complex stack traces for the derived comparison methods. If

performance benchmarking indicates this is a bottleneck for a given

application, implementing all six rich comparison methods instead is

likely to provide an easy speed boost.

.. note::

This decorator makes no attempt to override methods that have been

declared in the class *or its superclasses*. Meaning that if a

superclass defines a comparison operator, *total_ordering* will not

implement it again, even if the original method is abstract.

.. versionadded:: 3.2

.. versionchanged:: 3.4

Returning ``NotImplemented`` from the underlying comparison function for

unrecognised types is now supported.

.. data:: Placeholder

A singleton object used as a sentinel to reserve a place

for positional arguments when calling :func:`partial`

and :func:`partialmethod`.

.. versionadded:: 3.14

.. function:: partial(func, /, *args, **keywords)

Return a new :ref:`partial object<partial-objects>` which when called

will behave like *func* called with the positional arguments *args*

and keyword arguments *keywords*. If more arguments are supplied to the

call, they are appended to *args*. If additional keyword arguments are

supplied, they extend and override *keywords*.

Roughly equivalent to::

The :func:`!partial` function is used for partial function application which "freezes"

some portion of a function's arguments and/or keywords resulting in a new object

with a simplified signature. For example, :func:`partial` can be used to create

a callable that behaves like the :func:`int` function where the *base* argument

defaults to ``2``:

.. doctest::

>>> basetwo = partial(int, base=2)

>>> basetwo.__doc__ = 'Convert base 2 string to an int.'

>>> basetwo('10010')

18

If :data:`Placeholder` sentinels are present in *args*, they will be filled first

when :func:`!partial` is called. This makes it possible to pre-fill any positional

argument with a call to :func:`!partial`; without :data:`!Placeholder`,

only the chosen number of leading positional arguments can be pre-filled.

If any :data:`!Placeholder` sentinels are present, all must be filled at call time:

.. doctest::

>>> say_to_world = partial(print, Placeholder, Placeholder, "world!")

>>> say_to_world('Hello', 'dear')

Hello dear world!

Calling ``say_to_world('Hello')`` raises a :exc:`TypeError`, because

only one positional argument is provided, but there are two placeholders

that must be filled in.

If :func:`!partial` is applied to an existing

:ref:`partial object <partial-objects>`, :data:`!Placeholder` sentinels of the

input object are filled in with new positional arguments.

A placeholder can be retained by inserting a new

:data:`!Placeholder` sentinel to the place held by a previous :data:`!Placeholder`:

.. doctest::

>>> from functools import partial, Placeholder as _

>>> remove = partial(str.replace, _, _, '')

>>> message = 'Hello, dear dear world!'

>>> remove(message, ' dear')

'Hello, world!'

>>> remove_dear = partial(remove, _, ' dear')

>>> remove_dear(message)

'Hello, world!'

>>> remove_first_dear = partial(remove_dear, _, 1)

>>> remove_first_dear(message)

'Hello, dear world!'

:data:`!Placeholder` cannot be passed to :func:`!partial` as a keyword argument.

.. versionchanged:: 3.14

Added support for :data:`Placeholder` in positional arguments.

.. class:: partialmethod(func, /, *args, **keywords)

Return a new :class:`partialmethod` descriptor which behaves

like :class:`partial` except that it is designed to be used as a method

definition rather than being directly callable.

*func* must be a :term:`descriptor` or a callable (objects which are both,

like normal functions, are handled as descriptors).

When *func* is a descriptor (such as a normal Python function,

:func:`classmethod`, :func:`staticmethod`, :func:`~abc.abstractmethod` or

another instance of :class:`partialmethod`), calls to ``__get__`` are

delegated to the underlying descriptor, and an appropriate

:ref:`partial object<partial-objects>` returned as the result.

When *func* is a non-descriptor callable, an appropriate bound method is

created dynamically. This behaves like a normal Python function when

used as a method: the *self* argument will be inserted as the first

positional argument, even before the *args* and *keywords* supplied to

the :class:`partialmethod` constructor.

Example::

.. versionadded:: 3.4

.. function:: reduce(function, iterable, /[, initial])

Apply *function* of two arguments cumulatively to the items of *iterable*, from

left to right, so as to reduce the iterable to a single value. For example,

``reduce(lambda x, y: x+y, [1, 2, 3, 4, 5])`` calculates ``((((1+2)+3)+4)+5)``.

The left argument, *x*, is the accumulated value and the right argument, *y*, is

the update value from the *iterable*. If the optional *initial* is present,

it is placed before the items of the iterable in the calculation, and serves as

a default when the iterable is empty. If *initial* is not given and

*iterable* contains only one item, the first item is returned.

Roughly equivalent to::

See :func:`itertools.accumulate` for an iterator that yields all intermediate

values.

.. versionchanged:: 3.14

*initial* is now supported as a keyword argument.

.. decorator:: singledispatch

Transform a function into a :term:`single-dispatch <single

dispatch>` :term:`generic function`.

To define a generic function, decorate it with the ``@singledispatch``

decorator. When defining a function using ``@singledispatch``, note that the

dispatch happens on the type of the first argument::

.. method:: singledispatch.register()

:no-typesetting:

To add overloaded implementations to the function, use the :func:`!register`

attribute of the generic function, which can be used as a decorator. For

functions annotated with types, the decorator will infer the type of the

first argument automatically::

:class:`typing.Union` can also be used::

For code which doesn't use type annotations, the appropriate type

argument can be passed explicitly to the decorator itself::

For code that dispatches on a collections type (e.g., ``list``), but wants

to typehint the items of the collection (e.g., ``list[int]``), the

dispatch type should be passed explicitly to the decorator itself with the

typehint going into the function definition::

.. note::

At runtime the function will dispatch on an instance of a list regardless

of the type contained within the list i.e. ``[1,2,3]`` will be

dispatched the same as ``["foo", "bar", "baz"]``. The annotation

provided in this example is for static type checkers only and has no

runtime impact.

To enable registering :term:`lambdas<lambda>` and pre-existing functions,

the :func:`~singledispatch.register` attribute can also be used in a functional form::

The :func:`~singledispatch.register` attribute returns the undecorated function. This

enables decorator stacking, :mod:`pickling<pickle>`, and the creation

of unit tests for each variant independently::

When called, the generic function dispatches on the type of the first

argument::

Where there is no registered implementation for a specific type, its

method resolution order is used to find a more generic implementation.

The original function decorated with ``@singledispatch`` is registered

for the base :class:`object` type, which means it is used if no better

implementation is found.

If an implementation is registered to an :term:`abstract base class`,

virtual subclasses of the base class will be dispatched to that

implementation::

To check which implementation the generic function will choose for

a given type, use the ``dispatch()`` attribute::

To access all registered implementations, use the read-only ``registry``

attribute::

.. versionadded:: 3.4

.. versionchanged:: 3.7

The :func:`~singledispatch.register` attribute now supports using type annotations.

.. versionchanged:: 3.11

The :func:`~singledispatch.register` attribute now supports

:class:`typing.Union` as a type annotation.

.. class:: singledispatchmethod(func)

Transform a method into a :term:`single-dispatch <single

dispatch>` :term:`generic function`.

To define a generic method, decorate it with the ``@singledispatchmethod``

decorator. When defining a method using ``@singledispatchmethod``, note

that the dispatch happens on the type of the first non-*self* or non-*cls*

argument::

``@singledispatchmethod`` supports nesting with other decorators such as

:deco:`classmethod`. Note that to allow for

``dispatcher.register``, ``singledispatchmethod`` must be the *outer most*

decorator. Here is the ``Negator`` class with the ``neg`` methods bound to

the class, rather than an instance of the class::

The same pattern can be used for other similar decorators:

:deco:`staticmethod`, :deco:`~abc.abstractmethod`, and others.

.. versionadded:: 3.8

.. versionchanged:: 3.15

Added support of non-:term:`descriptor` callables.

.. function:: update_wrapper(wrapper, wrapped, assigned=WRAPPER_ASSIGNMENTS, updated=WRAPPER_UPDATES)

Update a *wrapper* function to look like the *wrapped* function. The optional

arguments are tuples to specify which attributes of the original function are

assigned directly to the matching attributes on the wrapper function and which

attributes of the wrapper function are updated with the corresponding attributes

from the original function. The default values for these arguments are the

module level constants ``WRAPPER_ASSIGNMENTS`` (which assigns to the wrapper

function's :attr:`~function.__module__`, :attr:`~function.__name__`,

:attr:`~function.__qualname__`, :attr:`~function.__annotations__`,

:attr:`~function.__type_params__`, and :attr:`~function.__doc__`, the

documentation string) and ``WRAPPER_UPDATES`` (which updates the wrapper

function's :attr:`~function.__dict__`, i.e. the instance dictionary).

To allow access to the original function for introspection and other purposes

(e.g. bypassing a caching decorator such as :deco:`lru_cache`), this function

automatically adds a ``__wrapped__`` attribute to the wrapper that refers to

the function being wrapped.

The main intended use for this function is in :term:`decorator` functions which

wrap the decorated function and return the wrapper. If the wrapper function is

not updated, the metadata of the returned function will reflect the wrapper

definition rather than the original function definition, which is typically less

than helpful.

:func:`update_wrapper` may be used with callables other than functions. Any

attributes named in *assigned* or *updated* that are missing from the object

being wrapped are ignored (i.e. this function will not attempt to set them

on the wrapper function). :exc:`AttributeError` is still raised if the

wrapper function itself is missing any attributes named in *updated*.

.. versionchanged:: 3.2

The ``__wrapped__`` attribute is now automatically added.

The :attr:`~function.__annotations__` attribute is now copied by default.

Missing attributes no longer trigger an :exc:`AttributeError`.

.. versionchanged:: 3.4

The ``__wrapped__`` attribute now always refers to the wrapped

function, even if that function defined a ``__wrapped__`` attribute.

(see :issue:`17482`)

.. versionchanged:: 3.12

The :attr:`~function.__type_params__` attribute is now copied by default.

.. decorator:: wraps(wrapped, assigned=WRAPPER_ASSIGNMENTS, updated=WRAPPER_UPDATES)

This is a convenience function for invoking :func:`update_wrapper` as a

function decorator when defining a wrapper function. It is equivalent to

``partial(update_wrapper, wrapped=wrapped, assigned=assigned, updated=updated)``.

For example::

Without the use of this decorator factory, the name of the example function

would have been ``'wrapper'``, and the docstring of the original :func:`!example`

would have been lost.

.. _partial-objects:

:class:`partial` Objects

:class:`partial` objects are callable objects created by :func:`partial`. They

have three read-only attributes:

.. attribute:: partial.func

A callable object or function. Calls to the :class:`partial` object will be

forwarded to :attr:`func` with new arguments and keywords.

.. attribute:: partial.args

The leftmost positional arguments that will be prepended to the positional

arguments provided to a :class:`partial` object call.

.. attribute:: partial.keywords

The keyword arguments that will be supplied when the :class:`partial` object is

called.

:class:`partial` objects are like :ref:`function objects <user-defined-funcs>` in that they are

callable, weak referenceable, and can have attributes. There are some important

differences. For instance, the :attr:`~definition.__name__` and :attr:`~definition.__doc__` attributes

are not created automatically.