Iterators

What a for loop really does

for asks the object for an iterator, then calls next() on it until it says it is finished:

names = ["Ada", "Grace"]
it = iter(names)

print(next(it))
print(next(it))

try:
    next(it)
except StopIteration:
    print("StopIteration, so the loop would end here")

Ada
Grace
StopIteration, so the loop would end here

Every for loop you have written has been doing exactly this underneath.

Iterable versus iterator

An iterable can produce an iterator. An iterator produces values one at a time and gets used up:

names = ["Ada", "Grace"]

print(list(names))
print(list(names))

it = iter(names)
print(list(it))
print(list(it))

['Ada', 'Grace']
['Ada', 'Grace']
['Ada', 'Grace']
[]

The list can be walked again and again. The iterator is exhausted after one pass, which is why the second list(it) is empty.

Making your own

Implement __iter__ to return something with __next__:

class Countdown:
    def __init__(self, start):
        self.start = start

    def __iter__(self):
        self.current = self.start
        return self

    def __next__(self):
        if self.current <= 0:
            raise StopIteration
        self.current -= 1
        return self.current + 1

for n in Countdown(3):
    print(n)

3
2
1

Raising StopIteration is how the iterator says it is done.

Generators do it for you

Almost always, write a generator instead. yield turns a function into an iterator, and Python handles the state and the StopIteration:

def countdown(start):
    while start > 0:
        yield start
        start -= 1

for n in countdown(3):
    print(n)

print(list(countdown(2)))

3
2
1
[2, 1]

Six lines became three, with no class and no bookkeeping.

Why it is worth it

A generator produces values on demand, so it never holds the whole sequence. This reads a notional huge file without loading it:

def first_n(source, n):
    for index, value in enumerate(source):
        if index >= n:
            return
        yield value

def naturals():
    n = 1
    while True:
        yield n
        n += 1

print(list(first_n(naturals(), 5)))

[1, 2, 3, 4, 5]

naturals() is infinite, and asking for five costs five.

Generator expressions

The comprehension form, with round brackets:

squares = (n * n for n in range(5))
print(type(squares).__name__)
print(sum(squares))
print(sum(squares))

generator
30
0

The second sum is zero because the generator was used up by the first. This is the same exhaustion rule, and it catches everyone once.

Making a class iterable the easy way

If your object wraps something already iterable, hand back its iterator:

class Playlist:
    def __init__(self, songs):
        self.songs = songs

    def __iter__(self):
        return iter(self.songs)

p = Playlist(["a", "b"])
print(list(p))
print(list(p))

['a', 'b']
['a', 'b']

Because a fresh iterator is made each time, this object can be looped over repeatedly — the behaviour people expect from a collection.