AI in Plain WordsUnit 4: Language, pictures and action
› AI-32 · Lesson 32 of 50
Many senses
multimodal
Since pictures, sound and text can all be turned into numbers, one model can be trained on all of them together. Models like this are called multimodal. They can describe a photo, read a chart, listen to speech and answer out loud. Combining senses helps: a photo of a broken bike chain plus "what's wrong?" gives more to go on than either alone. Each sense also brings its own ways to be wrong.
Think
What is a question you would want to ask about a photo?