Chapters
Python114 chapters

CollectionsChapter 42 of 114

The collections Module

Counter, defaultdict, deque and namedtuple, and when each is the right tool.

Counter

Counts things. It is a dictionary subclass, so everything you know still works:

Python
from collections import Counter

words = ["a", "b", "a", "c", "a", "b"]
counts = Counter(words)

print(counts["a"])
print(counts["never seen"])
print(counts.most_common(2))
print(sorted(counts.items()))

Output

3
0
[('a', 3), ('b', 2)]
[('a', 3), ('b', 2), ('c', 1)]

A missing key gives 0 rather than raising, which is exactly right for a tally.

It counts any iterable, including a string, and Counters can be added:

Python
from collections import Counter

print(Counter("mississippi").most_common(2))
print(sorted((Counter("aab") + Counter("bc")).items()))

Output

[('i', 4), ('s', 4)]
[('a', 2), ('b', 2), ('c', 1)]

defaultdict

A dictionary that creates missing values for you. You give it a factory — a function called with no arguments:

Python
from collections import defaultdict

groups = defaultdict(list)
for subject, name in [("maths", "ada"), ("maths", "grace"), ("code", "kat")]:
    groups[subject].append(name)

print(dict(groups))

Output

{'maths': ['ada', 'grace'], 'code': ['kat']}

Compare that with setdefault, which does the same job without the special type. Use defaultdict when every value has the same shape and you are doing it in a loop.

Python
from collections import defaultdict

groups = defaultdict(list)
print(groups["physics"])
print(dict(groups))
print("chemistry" in groups)
print(dict(groups))

Output

[]
{'physics': []}
False
{'physics': []}

deque

A list is slow at the front: removing the first item shifts everything down. A deque is fast at both ends:

Python
from collections import deque

queue = deque(["a", "b"])
queue.append("c")
queue.appendleft("start")

print(list(queue))
print(queue.popleft())
print(queue.pop())
print(list(queue))

Output

['start', 'a', 'b', 'c']
start
c
['a', 'b']

Give it a maxlen and it becomes a sliding window that discards from the other end automatically:

Python
from collections import deque

last_three = deque(maxlen=3)
for n in range(6):
    last_three.append(n)

print(list(last_three))

Output

[3, 4, 5]

That is a neat way to keep the last N log lines without ever growing.

namedtuple

A tuple whose positions have names. Still a tuple in every way:

Python
from collections import namedtuple

Point = namedtuple("Point", ["x", "y"])
p = Point(4, 9)

print(p.x, p.y)
print(p[0])
print(tuple(p))
print(p._replace(x=0))

Output

4 9
4
(4, 9)
Point(x=0, y=9)

_replace returns a new one, because tuples cannot change. The leading underscore is to keep the name from clashing with a field you might call replace.

For anything with behaviour, a dataclass is the better tool. Reach for namedtuple when you want a lightweight record that is still a tuple.

Which to use

You wantUse
to tally thingsCounter
a dictionary of lists or countsdefaultdict
a queue, or a sliding windowdeque
a small record with named fieldsnamedtuple

Test yourself

2 questions

What does a Counter give you for a key it has never seen?

Show the answer

0 — Which is exactly right for a tally: you can increment without checking first.

Why is deque better than a list for a queue?

Show the answer

Removing from the front is fast, where a list has to shift everything — A deque is fast at both ends. Give it a maxlen and it becomes a sliding window.

Next chapter

If...Else

Run code only when a condition holds, and choose between branches.