MEPX
Chapter 3 of 12All chapters
  1. 1What AI means
  2. 2How machine learning works
  3. 3Training data
  4. 4Kinds of learning
  5. 5Neural networks
  6. 6Large language models
  7. 7Limits and hallucination
  8. 8Bias and fairness
  9. 9Using AI well
  10. 10Privacy and safety
  11. 11Beyond text
  12. 12AI and work

Chapter 3 of 12

Training data

The model is what it read.

Quantity and quality

The examples a model learns from decide what it can do. Gaps in the data become gaps in the model, and errors in the data become errors it repeats confidently.

  • A model trained mostly on English does worse in other languages.
  • Labelled data is expensive, which is why so much effort goes into learning from unlabelled text.

The cutoff

Training happens up to a point in time. After that the model knows nothing new unless the information is supplied in the conversation or fetched by a tool.