PEP: 3114

Title: Renaming iterator.next() to iterator.__next__()

Author: Ka-Ping Yee <ping@zesty.ca>

Status: Final

Type: Standards Track

Created: 04-Mar-2007

Python-Version: 3.0

Post-History:

Abstract

The iterator protocol in Python 2.x consists of two methods:

``__iter__()`` called on an iterable object to yield an iterator, and

``next()`` called on an iterator object to yield the next item in the

sequence. Using a ``for`` loop to iterate over an iterable object

implicitly calls both of these methods. This PEP proposes that the

``next`` method be renamed to ``__next__``, consistent with all the

other protocols in Python in which a method is implicitly called as

part of a language-level protocol, and that a built-in function named

``next`` be introduced to invoke ``__next__`` method, consistent with

the manner in which other protocols are explicitly invoked.

Names With Double Underscores

In Python, double underscores before and after a name are used to

distinguish names that belong to the language itself. Attributes and

methods that are implicitly used or created by the interpreter employ

this naming convention; some examples are:

* ``__file__`` - an attribute automatically created by the interpreter

* ``__dict__`` - an attribute with special meaning to the interpreter

* ``__init__`` - a method implicitly called by the interpreter

Note that this convention applies to methods such as ``__init__`` that

are explicitly defined by the programmer, as well as attributes such as

``__file__`` that can only be accessed by naming them explicitly, so it

includes names that are used *or* created by the interpreter.

(Not all things that are called "protocols" are made of methods with

double-underscore names. For example, the ``__contains__`` method has

double underscores because the language construct ``x in y`` implicitly

calls ``__contains__``. But even though the ``read`` method is part of

the file protocol, it does not have double underscores because there is

no language construct that implicitly invokes ``x.read()``.)

The use of double underscores creates a separate namespace for names

that are part of the Python language definition, so that programmers

are free to create variables, attributes, and methods that start with

letters, without fear of silently colliding with names that have a

language-defined purpose. (Colliding with reserved keywords is still

a concern, but at least this will immediately yield a syntax error.)

The naming of the ``next`` method on iterators is an exception to

this convention. Code that nowhere contains an explicit call to a

``next`` method can nonetheless be silently affected by the presence

of such a method. Therefore, this PEP proposes that iterators should

have a ``__next__`` method instead of a ``next`` method (with no

change in semantics).

Double-Underscore Methods and Built-In Functions

The Python language defines several protocols that are implemented or

customized by defining methods with double-underscore names. In each

case, the protocol is provided by an internal method implemented as a

C function in the interpreter. For objects defined in Python, this

C function supports customization by implicitly invoking a Python method

with a double-underscore name (it often does a little bit of additional

work beyond just calling the Python method.)

Sometimes the protocol is invoked by a syntactic construct:

* ``x[y]`` --> internal ``tp_getitem`` --> ``x.__getitem__(y)``

* ``x + y`` --> internal ``nb_add`` --> ``x.__add__(y)``

* ``-x`` --> internal ``nb_negative`` --> ``x.__neg__()``

Sometimes there is no syntactic construct, but it is still useful to be

able to explicitly invoke the protocol. For such cases Python offers a

built-in function of the same name but without the double underscores.

* ``len(x)`` --> internal ``sq_length`` --> ``x.__len__()``

* ``hash(x)`` --> internal ``tp_hash`` --> ``x.__hash__()``

* ``iter(x)`` --> internal ``tp_iter`` --> ``x.__iter__()``

Following this pattern, the natural way to handle ``next`` is to add a

``next`` built-in function that behaves in exactly the same fashion.

* ``next(x)`` --> internal ``tp_iternext`` --> ``x.__next__()``

Further, it is proposed that the ``next`` built-in function accept a

sentinel value as an optional second argument, following the style of

the ``getattr`` and ``iter`` built-in functions. When called with two

arguments, ``next`` catches the StopIteration exception and returns

the sentinel value instead of propagating the exception. This creates

a nice duality between ``iter`` and ``next``:

iter(function, sentinel) <--> next(iterator, sentinel)

Previous Proposals

This proposal is not a new idea. The idea proposed here was supported

by the BDFL on python-dev [1]_ and is even mentioned in the original

iterator PEP, :pep:`234`::

Objections

There have been a few objections to the addition of more built-ins.

In particular, Martin von Loewis writes [2]_::

Transition Plan

Two additional transformations will be added to the 2to3 translation

tool [3]_:

* Method definitions named ``next`` will be renamed to ``__next__``.

* Explicit calls to the ``next`` method will be replaced with calls

to the built-in ``next`` function. For example, ``x.next()`` will

become ``next(x)``.

Collin Winter looked into the possibility of automatically deciding

whether to perform the second transformation depending on the presence

of a module-level binding to ``next`` [4]_ and found that it would be

"ugly and slow". Instead, the translation tool will emit warnings

upon detecting such a binding. Collin has proposed warnings for the

following conditions [5]_:

* Module-level assignments to ``next``.

* Module-level definitions of a function named ``next``.

* Module-level imports of the name ``next``.

* Assignments to ``__builtin__.next``.

Approval

This PEP was accepted by Guido on March 6, 2007 [6]_.

Implementation

A patch with the necessary changes (except the 2to3 tool) was written

by Georg Brandl and committed as revision 54910.

References

.. [1] Single- vs. Multi-pass iterability (Guido van Rossum)

.. [2] PEP: rename it.next() to it.__next__()... (Martin von Loewis)

.. [3] 2to3 refactoring tool

.. [4] PEP: rename it.next() to it.__next__()... (Collin Winter)

.. [5] PEP 3113 transition plan

.. [6] PEP: rename it.next() to it.__next__()... (Guido van Rossum)

Copyright

This document has been placed in the public domain.