MEPX
Chapter 10 of 10All chapters
  1. 1Framing the problem
  2. 2Data
  3. 3Training and testing
  4. 4Underfitting and overfitting
  5. 5Measuring performance
  6. 6Common algorithms
  7. 7How training works
  8. 8The pipeline
  9. 9After deployment
  10. 10Common pitfalls

Chapter 10 of 10

Common pitfalls

How projects fail.

The usual list

Leakage, a test set used repeatedly, a metric that does not match the business cost, training data unlike production data, and a baseline nobody measured.

  • Always compare against the dumbest reasonable baseline.
  • A model that cannot be explained will not be trusted or adopted.

Ethics in practice

Models trained on historical decisions reproduce historical patterns, including unfair ones. Checking performance separately across groups is a basic step, not an advanced one.