MEPX
Chapter 8 of 10All chapters
  1. 1What a prompt is doing
  2. 2Giving context
  3. 3Examples
  4. 4Multi-step problems
  5. 5Controlling format
  6. 6Roles and personas
  7. 7Iterating
  8. 8Evaluating prompts
  9. 9Prompt injection
  10. 10Folklore and limits

Chapter 8 of 10

Evaluating prompts

Beyond it looked better.

Test sets

Collect ten to twenty representative inputs with notes on what a good answer contains. Run any prompt change against all of them before adopting it.

  • Single examples mislead because output varies between runs.
  • Record failures; they are the most informative cases.

Measuring

Score against criteria you wrote in advance. For production use, automated checks plus periodic human review is the practical arrangement.