MEPX
Chapter 11 of 12All chapters
  1. 1What AI means
  2. 2How machine learning works
  3. 3Training data
  4. 4Kinds of learning
  5. 5Neural networks
  6. 6Large language models
  7. 7Limits and hallucination
  8. 8Bias and fairness
  9. 9Using AI well
  10. 10Privacy and safety
  11. 11Beyond text
  12. 12AI and work

Chapter 11 of 12

Beyond text

Images, audio and agents.

Other kinds of model

The same training ideas generate images, transcribe speech, synthesise voices and drive robots. Multimodal models handle several of these at once, taking an image and text together.

  • Convincing fake images, video and voices are now cheap, which changes what evidence means.
  • Provenance markers and content credentials are an attempt to answer that.

Agents

An agent is a model given tools and allowed to act in a loop: search, run code, call an interface. Capability grows and so does the cost of a mistake, which is why permission and review matter.