Arrows for ideas

Tonight

On a big sheet, put kitten and puppy close together and asphalt far away. You just drew a map of meaning. A computer stores each word as a list of numbers — an arrow. Words whose arrows point the same way get treated as similar. Finding a note that matches a question is finding the nearest arrows, then answering from those notes.

Say it in one sentenceSimilar ideas point the same way. Search is finding the nearest arrows.

Already in the library

• PrintableThe Word GalaxyPlace word cards by feel. Distance becomes meaning, including a step you can reuse on a new word.Open →
• Field guideEmbeddingThe list of numbers a word or a paragraph becomes.Open →
• Field guideRAGFetch the nearest notes first, then ask the model to answer from them.Open →

Textbook names

For ages 13–16 and the grown-up. One sentence each. The picture above is the lesson.

Vector

A list of numbers with a direction. That is the arrow.

Cosine similarity

A score for how much two arrows point the same way.

Matrix

A table of arrows. A layer of a model uses one to move many ideas at once.

Projection

The shadow of an arrow onto a direction you care about. Attention is built from shadows like this.

Low-rank update

A small correction written on top of a frozen model, instead of repainting every number.

The rest of this field

Eigenvalues, singular value decomposition, and principal component analysis find the few directions that matter inside a huge table of arrows. Storing those numbers with fewer digits is quantization, which meets the last stop on this path.