AI in Plain WordsUnit 2: How machines learn
› AI-13 · Lesson 13 of 50
Memorizing isn't learning
overfitting / test set
A model can score well on its practice examples just by memorizing them, including their accidents and noise. Then it does worse on anything new. This is called overfitting. Picture a student who memorizes the practice test, then fails the real one. So builders hold back some examples the model never sees in training, called the test set, and score it on those. Even that can be too hopeful if the real world differs from the test examples.
Think
Why doesn't a teacher put the exact homework questions on the test?
Try it
For grown-ups
If the test repeats the homework, a student can pass on memory alone. New questions show whether they learned the idea. Builders hold back a test set for the same reason.