Reading AI news

A short filter.

Questions to ask

What exactly did the system do, on what data, evaluated how, compared with what baseline? Demonstrations are chosen; failures are rarely published.

Benchmarks can be contaminated by training data that includes them.
A press release is not a paper, and a paper is not a replication.

Language to notice

Learns, understands, decides and wants are convenient shorthands that smuggle in assumptions. Rewriting a claim without them often shows what is actually being asserted.