Foundation Models

CLIP

CLIP (Contrastive Language-Image Pre-training) is a vision-language model from OpenAI (Radford et al., 2021) that learns aligned image and text embeddings by contrastively training on 400 million web image-text pairs. Its dual encoders enable zero-shot classification, open-vocabulary retrieval, and semantic similarity scoring without task-specific fine-tuning. In robotics, CLIP embeddings ground language in perception for systems like CLIPort and open-vocabulary object detectors, and serve as reward or goal encoders.

Why it matters for physical AI

CLIP-style aligned embeddings are the connective tissue between web-scale semantics and robot perception, enabling open-vocabulary grasping, language-specified goals, and semantic scene understanding without exhaustive robot-specific labels.