MEPX
Chapter 7 of 10All chapters
  1. 1Framing the problem
  2. 2Data
  3. 3Training and testing
  4. 4Underfitting and overfitting
  5. 5Measuring performance
  6. 6Common algorithms
  7. 7How training works
  8. 8The pipeline
  9. 9After deployment
  10. 10Common pitfalls

Chapter 7 of 10

How training works

Loss and gradient descent.

The loop

Make a prediction, measure the error with a loss function, adjust the parameters slightly in the direction that reduces it, and repeat. That is gradient descent.

  • The learning rate controls step size: too large overshoots, too small crawls.
  • Training stops when held-out performance stops improving.

Hyperparameters

Settings chosen before training, such as tree depth or learning rate. They are tuned on validation data, and tuning on the test set quietly invalidates it.