MEPX
Chapter 9 of 10All chapters
  1. 1Framing the problem
  2. 2Data
  3. 3Training and testing
  4. 4Underfitting and overfitting
  5. 5Measuring performance
  6. 6Common algorithms
  7. 7How training works
  8. 8The pipeline
  9. 9After deployment
  10. 10Common pitfalls

Chapter 9 of 10

After deployment

Where models quietly rot.

Drift

The world changes and the data changes with it, so performance decays. Monitoring inputs and outcomes is what catches it before users do.

  • Compare live input distributions against training data.
  • Retraining on a schedule is normal, not an admission of failure.

Feedback loops

A deployed model changes the behaviour it later learns from. A recommendation system trained on its own recommendations narrows over time unless deliberately counteracted.