Skip to content
Hi, Bot

› Math for AI

Eight ideas do most of the work inside AI. Read only the first line of each for a three-minute refresher, or open a card to try it and see the real math. For the bigger picture, with things to try at the table, see the math under the bot.

Print the one-page cheat sheet (PDF)

1. Vectors

A vector is a list of numbers that points somewhere. AI turns words, pictures and sounds into lists like this so it can work with them.

Try it and see the real math

Make the arrow twice as long. What happens to both numbers?

x = 3
y = 4

(3, 4) · length 5

The real math

v = (x, y), length |v| = √(x² + y²)

The arrow (3, 4) has length √(9 + 16) = 5. Double it to (6, 8) and the length doubles to 10.

See it in the Embedding Engine →

2. Dot product and similarity

Multiply two lists number by number and add it all up. Big means they point the same way, zero means they have nothing in common, and below zero means they point opposite ways.

Try it and see the real math

Turn one arrow until the score hits zero. Which way is it pointing?

Turn the amber arrow

dot = 6.36 · cosine = 0.71

The real math

a · b = a₁b₁ + a₂b₂, cosine = a · b ÷ (|a| |b|)

Same direction scores 1. A quarter turn (90°) scores 0. Pointing opposite ways (180°) scores −1.

See it in the Embedding Engine →

3. Matrices

A matrix is a grid of numbers that moves things: it can stretch, turn or flip every point at once. AI models are mostly big grids like this, mixing numbers together.

Try it and see the real math

Tap Turn twice. Is that the same as turning the robot upside down?

matrix so far = [[1, 0], [0, 1]]

The real math

M v = (m₁₁x + m₁₂y, m₂₁x + m₂₂y)

Turn = [[0, −1], [1, 0]]. Turn twice: [[0, −1], [1, 0]] × [[0, −1], [1, 0]] = [[−1, 0], [0, −1]], a half turn.

See it in the Attention Engine →

4. Functions and slope

A function is a rule: put a number in, get a number out. The slope says how much the answer changes each time you nudge the input. A model's weights are slopes it gets to pick.

Try it and see the real math

Change b. Does the line get steeper, or just move?

y = 1·x + 0 · one step right = 1 up

The real math

y = w·x + b, slope = w

With w = 1 and b = 0, one step right always means one step up. Change b and the whole line slides, but the slope stays 1.

See it in the Weight Pond →

5. Gradients and learning rate

To learn, a model checks which way is downhill on its mistakes and takes a step that way. The learning rate is how big each step is: too small is slow, too big overshoots.

Try it and see the real math

Try each step size. Which one jumps right past the bottom?

Step size (learning rate)

step 0: w = 0 · 3 from the bottom · mistake 9

The real math

w ← w − rate × slope, slope of (w − 3)² is 2(w − 3)

Start at w = 0. Rate 0.1 moves to 0.6. Rate 0.5 lands on 3 in one step. Rate 1.1 jumps to 6.6, farther from 3 than where it started.

See it in the Model Lab →

6. Probability

Probability says how often something happens if you try many times. AI does not know the answer for sure. It picks what is most likely, and it learns those odds by counting.

Try it and see the real math

Roll 10 times, then 1000. When do the bars even out?

Roll the die
  1. 130%
  2. 220%
  3. 30%
  4. 430%
  5. 50%
  6. 620%

Dashed line: a fair share, 16.7%.

The real math

P(face) = 1 ÷ 6 ≈ 16.7%

Ten rolls are lumpy: one face may never show up. After 1000 rolls every face lands within 3 points of 16.7%.

See it in the Next Word →

7. Softmax

Softmax turns any scores into shares that add up to 100%. The highest score gets the biggest share. Those shares are probabilities, not a promise that the top pick is right.

Try it and see the real math

Raise every score by the same amount. Do the shares change?

A
67%
B
24%
C
9%

Total: 100%

The real math

share(i) = eˢⁱ ÷ (eˢ¹ + eˢ² + eˢ³)

Scores 2, 1, 0 become about 67%, 24% and 9%. Add 1 to every score and the shares stay exactly the same.

See it in the Attention Engine →

8. Loss

Loss is one number for how wrong a model is. Training is just making that number smaller. Squaring each miss means one big miss hurts more than a few small ones.

Try it and see the real math

Get every guess right. What is the loss now?

loss = 16 + 0 + 0 = 16

The real math

loss = (guess₁ − answer₁)² + (guess₂ − answer₂)² + …

One guess off by 4 costs 16. Two guesses off by 2 cost only 4 + 4 = 8.

See it in the Model Lab →

Nothing you do here leaves this device. Every number on this page is checked by a test. More AI ideas →