Testing

Why bother

You already test. You run the thing and look at the output. A test just writes that check down so it runs again automatically, on every change, for ever.

The moment it pays for itself is the first time you refactor.

unittest, which is built in

Subclass TestCase, write methods starting with test, and assert:

Python

import unittest
import io

def add(a, b):
    return a + b

class TestAdd(unittest.TestCase):
    def test_adds_two_numbers(self):
        self.assertEqual(add(2, 3), 5)

    def test_handles_negatives(self):
        self.assertEqual(add(-1, 1), 0)

suite = unittest.TestLoader().loadTestsFromTestCase(TestAdd)
result = unittest.TextTestRunner(stream=io.StringIO()).run(suite)
print("ran", result.testsRun, "tests")
print("failures", len(result.failures), "errors", len(result.errors))

Output

ran 2 tests
failures 0 errors 0

Normally you run it from the command line, and unittest finds the tests itself. These examples load the class explicitly so they work on this page, which runs your code outside the __main__ module that discovery looks in:

bash

python -m unittest
python -m unittest test_math.py -v

A failing test tells you what it wanted

Python

import unittest
import io

def add(a, b):
    return a * b          # wrong on purpose

class TestAdd(unittest.TestCase):
    def test_adds(self):
        self.assertEqual(add(2, 3), 5)

suite = unittest.TestLoader().loadTestsFromTestCase(TestAdd)
result = unittest.TextTestRunner(stream=io.StringIO()).run(suite)
print("failures:", len(result.failures))
print("6 != 5" in result.failures[0][1])

Output

failures: 1
True

The message names both values. That is the whole point of assertEqual over a bare assert.

The assertions worth knowing

Python

import unittest
import io

class Examples(unittest.TestCase):
    def test_them(self):
        self.assertEqual(2 + 2, 4)
        self.assertTrue([1])
        self.assertIn("a", "cat")
        self.assertIsNone(None)
        self.assertAlmostEqual(0.1 + 0.2, 0.3)
        with self.assertRaises(ValueError):
            int("abc")

suite = unittest.TestLoader().loadTestsFromTestCase(Examples)
result = unittest.TextTestRunner(stream=io.StringIO()).run(suite)
print("failures", len(result.failures), "errors", len(result.errors))

Output

failures 0 errors 0

assertAlmostEqual is the float comparison, and assertRaises is how you test that something fails properly — which is as important as testing that it works.

setUp runs before each test

Python

import unittest
import io

class TestBasket(unittest.TestCase):
    def setUp(self):
        self.items = ["apple"]

    def test_starts_with_one(self):
        self.assertEqual(len(self.items), 1)

    def test_can_add(self):
        self.items.append("pear")
        self.assertEqual(len(self.items), 2)

suite = unittest.TestLoader().loadTestsFromTestCase(TestBasket)
result = unittest.TextTestRunner(stream=io.StringIO()).run(suite)
print("ran", result.testsRun, "failures", len(result.failures))

Output

ran 2 failures 0

Both tests passed, and the second's append did not leak into the first. setUp builds a fresh one each time; tearDown cleans up afterwards.

pytest

Not in the standard library, and what most projects use. A test is a plain function with a plain assert:

bash

python -m pip install pytest

Python

# test_math.py
def add(a, b):
    return a + b

def test_adds_two_numbers():
    assert add(2, 3) == 5

def test_handles_negatives():
    assert add(-1, 1) == 0

bash

python -m pytest
python -m pytest -k negatives -v

pytest rewrites the assert so a failure still shows both values, which gets you unittest's reporting with none of the ceremony. It also runs unittest tests, so adopting it is not a rewrite.

What to test

The behaviour, not the implementation. A test that breaks when you rename a

private helper is a liability.

The edges: empty input, zero, one item, the largest allowed value.
The failures: does bad input raise the right exception?
Every bug you fix. Write the test that would have caught it, then fix it.

Python

import unittest
import io

def parse_age(text):
    value = int(text)
    if value < 0:
        raise ValueError("age cannot be negative")
    return value

class TestParseAge(unittest.TestCase):
    def test_reads_a_number(self):
        self.assertEqual(parse_age("30"), 30)

    def test_allows_zero(self):
        self.assertEqual(parse_age("0"), 0)

    def test_rejects_negative(self):
        with self.assertRaises(ValueError):
            parse_age("-1")

    def test_rejects_text(self):
        with self.assertRaises(ValueError):
            parse_age("thirty")

suite = unittest.TestLoader().loadTestsFromTestCase(TestParseAge)
result = unittest.TextTestRunner(stream=io.StringIO()).run(suite)
print("ran", result.testsRun, "failures", len(result.failures))

Output

ran 4 failures 0

Note test_allows_zero. Zero is falsy, and a sloppy implementation using if not value would reject it — so that test is the one that earns its keep.

Why bother

unittest, which is built in

A failing test tells you what it wanted

The assertions worth knowing

setUp runs before each test

pytest

What to test

Test yourself

Why use assertEqual rather than a bare assert in unittest?

What should you do first when you find a bug?

Logging