MEPX
Chapter 6 of 12All chapters
  1. 1What AI means
  2. 2How machine learning works
  3. 3Training data
  4. 4Kinds of learning
  5. 5Neural networks
  6. 6Large language models
  7. 7Limits and hallucination
  8. 8Bias and fairness
  9. 9Using AI well
  10. 10Privacy and safety
  11. 11Beyond text
  12. 12AI and work

Chapter 6 of 12

Large language models

Predicting the next piece of text.

What they actually do

A language model estimates which piece of text is likely to come next, then does it again with its own output included. Everything else, including apparent reasoning, comes out of that loop.

  • Text is split into tokens, roughly word fragments, not whole words.
  • The context window is how much text the model can consider at once.

Why fluency misleads

Predicting plausible text is exactly the skill that makes wrong answers sound right. Confidence in the output carries no information about accuracy, which is the single most important thing to internalise.