The math under the botStop 3 of 6
› Ages 8–16 · about 20 minutes
Walk downhill
Multivariate calculus and continuous optimization
Learning is a short step downhill on a score you can measure.
Tonight
A model keeps a score for how wrong it is. Picture that score as height. Training looks at the slope under your feet, takes a small step downhill, and looks again. Too long a step and you skip the valley. Too short a step and you are still on the hill at bedtime.
Say it in one sentenceLearning is walking downhill on how wrong you are.
Already in the library
- GameLoss MountainWalk the robot downhill in the fog. That is the training step, on paper.Open →
- GameWeight PondTune the frog's weights, and see why one slope cannot separate every pond.Open →
- VideoHow Does AI Actually Learn?A tiny perceptron, with every on-screen number coming from the same small model.Open →
Textbook names
For ages 13–16 and the grown-up. One sentence each. The picture above is the lesson.
- Loss
- The number that says how wrong the model is right now.
- Gradient
- The direction of steepest uphill. Training steps the other way.
- Learning rate
- How long each step is.
- Saddle
- A flat spot that is not the bottom. A little momentum keeps the walk moving through it.
- Chain rule
- How a change at the last step traces back to the first. That tracing is backpropagation.
The rest of this field
Hessians and Newton steps look at the curve, not only the slope. Lagrange multipliers and the Karush–Kuhn–Tucker conditions are for hills that have rules about where you may stand. AdamW and its cousins are bookkeeping that resizes the step as you walk.