Chapter 6 of 10All chapters
Chapter 6 of 10
Noise and significance
Is that a real change.
Everything fluctuates
Metrics move day to day without any cause worth investigating. Reacting to every wobble produces explanations for randomness and a great deal of wasted work.
- Look at the historical range of variation before calling a change unusual.
- Small denominators produce large percentage swings by arithmetic.
Control charts
Plotting a metric with its normal range makes ordinary variation visible and unusual movement obvious. It is a simple defence against overreacting.