MEPX
Get started
Chapters
Python114 chapters

Basics

  1. Introduction
  2. Get Started
  3. Syntax
  4. Comments
  5. Variables
  6. Variable Names
  7. Multiple Assignment
  8. Global Variables
  9. Data Types
  10. Numbers
  11. Casting
  12. Booleans
  13. Operators
  14. User Input

Strings

  1. Strings
  2. Slicing
  3. Modify Strings
  4. Concatenation
  5. f-Strings
  6. String Formatting
  7. Escape Characters
  8. String Methods
  9. Text and Unicode

Collections

  1. Lists
  2. List Access
  3. Change Items
  4. Add Items
  5. Remove Items
  6. Loop Lists
  7. List Comprehension
  8. Sort Lists
  9. Copy Lists
  10. Join Lists
  11. List Methods
  12. Tuples
  13. Sets
  14. Dictionaries
  15. Nested Dictionaries
  16. Dictionary Methods
  17. Set Methods
  18. Dict and Set Comprehensions
  19. The collections Module

Control flow

  1. If...Else
  2. Match
  3. While Loops
  4. For Loops
  5. Break and Continue

Functions

  1. Functions
  2. Arguments
  3. args and kwargs
  4. Default Values
  5. Return Values
  6. Scope
  7. Recursion
  8. Lambda
  9. Decorators
  10. Generators
  11. Closures

Classes and objects

  1. Classes and Objects
  2. The __init__ Method
  3. The __str__ Method
  4. Methods
  5. self
  6. Inheritance
  7. Polymorphism
  8. Iterators
  9. Dataclasses
  10. Enums
  11. Magic Methods

Modules and the standard library

  1. Modules
  2. PIP
  3. Dates
  4. Math
  5. JSON
  6. RegEx
  7. itertools
  8. os and sys
  9. Random Numbers

Files and errors

  1. File Handling
  2. Read Files
  3. Write Files
  4. Delete Files
  5. Try...Except
  6. Raising Exceptions
  7. CSV Files
  8. Context Managers
  9. Paths with pathlib

Beyond the basics

  1. Virtual Environments
  2. Type Hints
  3. Async Basics
  4. NumPy Intro
  5. Pandas Intro
  6. Matplotlib Intro
  7. Working with APIs
  8. Testing
  9. Logging
  10. Command-line Arguments
  11. Debugging
  12. SQLite
  13. Packaging

Machine learning

  1. What Machine Learning Is
  2. Your First Model
  3. Features and Labels
  4. Train and Test Split
  5. Classification
  6. Regression
  7. Evaluating a Model
  8. Overfitting and Underfitting
  9. Preprocessing
  10. Pipelines
  11. Clustering
  12. Neural Networks
  13. Saving and Using a Model
  14. Where It Goes Wrong

Modules and the standard libraryChapter 76 of 114

itertools

Building blocks for looping, that never build the whole sequence.

Back: RegExNext: os and sys

Everything here is lazy

itertools functions return iterators. Nothing is computed until you ask, and nothing is stored, so they work on inputs far larger than memory. Wrap in list() to see one.

chain: treat several sequences as one

Python
from itertools import chain

print(list(chain([1, 2], [3], [4, 5])))

groups = [[1, 2], [3], [4, 5]]
print(list(chain.from_iterable(groups)))

Output

[1, 2, 3, 4, 5]
[1, 2, 3, 4, 5]

chain.from_iterable is the flattener, and unlike [x for g in groups for x in g] it never builds the intermediate list.

islice: a slice for anything

You cannot write gen[:5] on a generator. islice is the slice that works:

Python
from itertools import islice, count

print(list(islice(count(1), 5)))
print(list(islice("abcdefgh", 2, 6)))
print(list(islice(range(10), 0, 10, 3)))

Output

[1, 2, 3, 4, 5]
['c', 'd', 'e', 'f']
[0, 3, 6, 9]

count(1) is endless. Taking five from it costs five.

The endless three

Python
from itertools import count, cycle, repeat, islice

print(list(islice(count(10, 5), 4)))
print(list(islice(cycle("ab"), 5)))
print(list(repeat("x", 3)))

Output

[10, 15, 20, 25]
['a', 'b', 'a', 'b', 'a']
['x', 'x', 'x']

Always pair count and cycle with something that stops, or the loop never ends.

accumulate: running totals

Python
from itertools import accumulate

print(list(accumulate([1, 2, 3, 4])))
print(list(accumulate([1, 2, 3, 4], max)))
print(list(accumulate([3, 1, 4], initial=0)))

Output

[1, 3, 6, 10]
[1, 2, 3, 4]
[0, 3, 4, 8]

groupby: runs of equal items

The one that surprises people: it groups consecutive items, so sort first if you want all of a kind together.

Python
from itertools import groupby

data = ["apple", "avocado", "banana", "blueberry", "apricot"]

for letter, group in groupby(data, key=lambda w: w[0]):
    print(letter, list(group))

Output

a ['apple', 'avocado']
b ['banana', 'blueberry']
a ['apricot']

a appeared twice, because "apricot" was not next to the others. Sorting by the same key fixes it:

Python
from itertools import groupby

data = ["apple", "avocado", "banana", "apricot"]
data.sort(key=lambda w: w[0])

for letter, group in groupby(data, key=lambda w: w[0]):
    print(letter, list(group))

Output

a ['apple', 'avocado', 'apricot']
b ['banana']

The sort is stable, so inside the a run the words keep the order they had in the original list rather than becoming alphabetical.

Combinations and permutations

Python
from itertools import combinations, permutations, product

print(list(combinations("abc", 2)))
print(list(permutations("abc", 2)))
print(list(product([1, 2], "ab")))

Output

[('a', 'b'), ('a', 'c'), ('b', 'c')]
[('a', 'b'), ('a', 'c'), ('b', 'a'), ('b', 'c'), ('c', 'a'), ('c', 'b')]
[(1, 'a'), (1, 'b'), (2, 'a'), (2, 'b')]

Combinations ignore order; permutations do not. product is the nested loop: product(a, b) is every pairing, and repeat= gives you a loop of loops.

Python
from itertools import product

print(list(product([0, 1], repeat=2)))

Output

[(0, 0), (0, 1), (1, 0), (1, 1)]

Filtering pairs

Python
from itertools import takewhile, dropwhile, compress

numbers = [1, 2, 3, 10, 1]

print(list(takewhile(lambda n: n < 5, numbers)))
print(list(dropwhile(lambda n: n < 5, numbers)))
print(list(compress("abcd", [1, 0, 1, 0])))

Output

[1, 2, 3]
[10, 1]
['a', 'c']

takewhile stops at the first failure, unlike filter, which checks everything. Note the trailing 1 survived dropwhile: once it starts yielding, it never tests again.

pairwise

Python
from itertools import pairwise

readings = [10, 12, 11, 15]
print(list(pairwise(readings)))
print([b - a for a, b in pairwise(readings)])

Output

[(10, 12), (12, 11), (11, 15)]
[2, -1, 4]

Differences between neighbours, without any index arithmetic.

Test yourself

2 questions

Why does groupby show the same key twice sometimes?

Show the answer

It groups consecutive items, so you must sort first — Sort by the same key you group by, and each key appears once.

How do you take the first five items of an endless generator?

Show the answer

islice(source, 5) — You cannot slice a generator, and list() on an endless one never returns.

Exercises for this chapter
Next chapter

os and sys

Talk to the operating system: environment, paths, and the running interpreter.

Back: RegExNext: os and sys
Back: RegExNext: os and sys