Chapter 8 of 10All chapters
Chapter 8 of 10
Designing for failure
Everything breaks eventually.
Assume it
Machines die, networks drop and dependencies slow down. Timeouts, retries with backoff and circuit breakers turn a dependency failure into degraded service rather than collapse.
- Every remote call needs a timeout, or one slow service takes yours down.
- Retrying without backoff turns a blip into a stampede.
Removing single points
Walk the diagram and ask what happens if each box vanishes. Anything with no answer is a single point of failure, and the fix is a second one plus a way to fail over.
- Redundancy costs money and is the only thing that buys availability.