Foundation Models

Gato

Gato is a generalist agent from DeepMind (2022): a single 1.2-billion-parameter transformer trained on around 600 tasks spanning Atari games, image captioning, dialogue, and real robot block stacking, with all modalities serialized into a common token sequence. Gato demonstrated that one set of weights could perform vision, language, and control tasks, though per-task performance trailed specialists. It is widely cited as a precursor to robot foundation models.

Why it matters for physical AI

Gato provided early evidence that control can be folded into the same sequence-modeling recipe as language and vision, foreshadowing the token-based vision-language-action models now central to physical AI.