Lesson 33 of 48
The gradient
The arrow that points straight uphill — and therefore, straight downhill.
Collect the partials into a vector and it acquires a geometric meaning none of them had alone: the direction of steepest increase. Negate it and you have the whole idea of training.
Do this
Compute a gradient by hand and verify every component numerically. Then sample a few hundred random unit directions and confirm none of them climbs faster.
The question that unlocks the next lesson
Why does gradient descent step in the direction of the negative gradient?
- ABecause the gradient points toward the minimum
- BBecause the gradient points in the direction of steepest increase, so its negative is steepest decrease
- CBecause the negative gradient is always shorter
- DBecause the gradient is undefined at minima