Skip to content
Hi, Bot

AI in Plain WordsUnit 4: Language, pictures and action

› AI-32 · Lesson 32 of 50

Many senses

multimodal

BlueskyX

Since pictures, sound and text can all be turned into numbers, one model can be trained on all of them together. Models like this are called multimodal. They can describe a photo, read a chart, listen to speech and answer out loud. Combining senses helps: a photo of a broken bike chain plus "what's wrong?" gives more to go on than either alone. Each sense also brings its own ways to be wrong.

Think

What is a question you would want to ask about a photo?