Multimodal

Describes an AI model that can take in or produce more than one kind of data, such as text, images, audio and video.

Also known as multimodal model, multimodal AI, natively multimodal

A multimodal model handles more than one “mode” of information. It might read a photo and answer a question about it in text, listen to speech and reply aloud, or turn a written prompt into a video. Each mode is converted into tokens the model can process together.

“Natively multimodal” means the model was trained on several kinds of data from the start, rather than having separate tools bolted together. You meet the term in model launches, voice assistants and world models, which often mix video, audio and text.