The math under the botStop 6 of 6
› Ages 8–16 · about 20 minutes
Tiny numbers run out
Numerical analysis and computational precision
The chip has a budget of digits, and a budget of trips to memory.
Tonight
Draw a calculator with only a few boxes for each number. A number can be too big for the boxes and spill, or so small that it rounds down to nothing. Training is millions of those boxes, filled and emptied together. A lot of the real work is keeping numbers inside the boxes, and not walking back to memory for the same number twice. The building around the chips — power, cooling, the network between them — is part of the same budget.
Say it in one sentenceThe whiteboard uses perfect numbers. The chip has a budget of digits and a budget of memory trips.
Already in the library
- GameServer FarmKeep the building powered, cool, and caught up. The arithmetic is not the only limit.Open →
- ExhibitA Training RunThe Hardware Atlas stop for what a training job actually occupies.Open →
- Field guideInference costWhy answering one prompt has a price, even after the model is trained.Open →
Textbook names
For ages 13–16 and the grown-up. One sentence each. The picture above is the lesson.
- Rounding error
- The leftover when a number does not fit the boxes.
- Overflow and underflow
- Too big to store, or so small it becomes zero.
- Condition
- A problem where a tiny change in the input makes a huge change in the answer. Those need more careful digits.
- Compute versus memory
- Sometimes the chip is waiting on the numbers to arrive, not on the arithmetic.
- Parallelism
- Splitting the table of numbers across many chips, then agreeing on the result.
The rest of this field
FlashAttention and low-precision formats such as BF16 and FP8 are engineering so the downhill walk still works inside those boxes. Data parallel, tensor parallel, and pipeline parallel are names for who holds which part of the table.