MEPX
Chapter 8 of 10All chapters
  1. 1The artificial neuron
  2. 2Layers and depth
  3. 3How it learns
  4. 4Activation functions
  5. 5Architectures
  6. 6Keeping it general
  7. 7Training in practice
  8. 8Transfer learning
  9. 9What they cannot do
  10. 10Learning it yourself

Chapter 8 of 10

Transfer learning

Not starting from scratch.

Reusing a model

Take a network trained on a large general dataset and adapt it with a small specific one. The early layers already encode generally useful patterns.

  • This is why useful models can be built without a million examples.
  • Fine-tuning adjusts some layers; feature extraction freezes them all.

Foundation models

Very large models trained once on broad data and adapted to many tasks are now the normal starting point. It changed the economics of applied machine learning.