Foundation Models

Vision Transformer (ViT)

The Vision Transformer (ViT) is a neural architecture, introduced by Dosovitskiy et al. in 2020, that applies a standard transformer to images by splitting them into fixed-size patches, linearly embedding each patch, and processing the resulting token sequence with self-attention. Given large-scale pretraining, ViTs match or surpass convolutional networks and now serve as the default vision backbone for CLIP, DINOv2, SAM, and most vision-language models.

Why it matters for physical AI

Because ViTs share the transformer substrate with language models, they make unified multimodal architectures natural, which is why nearly every modern robot foundation model tokenizes camera input through a ViT-family encoder.