› Embedding Engine

A model reads numbers, not words

An embedding is a list of numbers for a word or a picture. Two things count as alike when their lists point the same way, and cosine similarity measures that: 1 means the same direction, 0 means nothing in common. Search engines, video suggestions and chatbots that look things up all find the closest lists first.

Real embeddings have hundreds of numbers that nobody named, learned from billions of sentences and pictures. Ours has seven we named and wrote by hand, so you can read every one. Each picture's numbers come from its parts by one rule: a box body or an antenna adds metal, a round body or ears add fur, and the feet set wheels or legs. Image search models learn to put a picture near its words from millions of captioned pictures.

A model first cuts text into tokens in the Tokenizer Engine, then gives each token a list of numbers like these. The pictures are the Bots and Critters from the Dataset Engine.

Nothing you make leaves this device. Every number here was written by hand for Hi, Bot. More engines →