CollectionsChapter 42 of 114
The collections Module
Counter, defaultdict, deque and namedtuple, and when each is the right tool.
Counter
Counts things. It is a dictionary subclass, so everything you know still works:
from collections import Counter
words = ["a", "b", "a", "c", "a", "b"]
counts = Counter(words)
print(counts["a"])
print(counts["never seen"])
print(counts.most_common(2))
print(sorted(counts.items()))Output
3
0
[('a', 3), ('b', 2)]
[('a', 3), ('b', 2), ('c', 1)]A missing key gives 0 rather than raising, which is exactly right for a tally.
It counts any iterable, including a string, and Counters can be added:
from collections import Counter
print(Counter("mississippi").most_common(2))
print(sorted((Counter("aab") + Counter("bc")).items()))Output
[('i', 4), ('s', 4)]
[('a', 2), ('b', 2), ('c', 1)]defaultdict
A dictionary that creates missing values for you. You give it a factory — a function called with no arguments:
from collections import defaultdict
groups = defaultdict(list)
for subject, name in [("maths", "ada"), ("maths", "grace"), ("code", "kat")]:
groups[subject].append(name)
print(dict(groups))Output
{'maths': ['ada', 'grace'], 'code': ['kat']}Compare that with setdefault, which does the same job without the special type. Use defaultdict when every value has the same shape and you are doing it in a loop.
from collections import defaultdict
groups = defaultdict(list)
print(groups["physics"])
print(dict(groups))
print("chemistry" in groups)
print(dict(groups))Output
[]
{'physics': []}
False
{'physics': []}deque
A list is slow at the front: removing the first item shifts everything down. A deque is fast at both ends:
from collections import deque
queue = deque(["a", "b"])
queue.append("c")
queue.appendleft("start")
print(list(queue))
print(queue.popleft())
print(queue.pop())
print(list(queue))Output
['start', 'a', 'b', 'c'] start c ['a', 'b']
Give it a maxlen and it becomes a sliding window that discards from the other end automatically:
from collections import deque
last_three = deque(maxlen=3)
for n in range(6):
last_three.append(n)
print(list(last_three))Output
[3, 4, 5]
That is a neat way to keep the last N log lines without ever growing.
namedtuple
A tuple whose positions have names. Still a tuple in every way:
from collections import namedtuple
Point = namedtuple("Point", ["x", "y"])
p = Point(4, 9)
print(p.x, p.y)
print(p[0])
print(tuple(p))
print(p._replace(x=0))Output
4 9 4 (4, 9) Point(x=0, y=9)
_replace returns a new one, because tuples cannot change. The leading underscore is to keep the name from clashing with a field you might call replace.
For anything with behaviour, a dataclass is the better tool. Reach for namedtuple when you want a lightweight record that is still a tuple.
Which to use
| You want | Use |
|---|---|
| to tally things | Counter |
| a dictionary of lists or counts | defaultdict |
| a queue, or a sliding window | deque |
| a small record with named fields | namedtuple |
Test yourself
2 questionsWhat does a Counter give you for a key it has never seen?
Show the answer
0 — Which is exactly right for a tally: you can increment without checking first.
Why is deque better than a list for a queue?
Show the answer
Removing from the front is fast, where a list has to shift everything — A deque is fast at both ends. Give it a maxlen and it becomes a sliding window.
If...Else
Run code only when a condition holds, and choose between branches.