Foundation Models

Cross-Attention

Cross-attention is the transformer mechanism in which queries from one sequence attend over keys and values from another, fusing information across modalities or streams, in contrast to self-attention within a single sequence. Introduced with the transformer's encoder-decoder attention (Vaswani et al., 2017), it is how vision-language-action models condition action generation on visual tokens and language instructions, and how diffusion policies inject observation embeddings into denoising networks.

Why it matters for physical AI

Cross-attention is the primary architectural fusion point in modern robot policies, binding language goals, multi-view images, and proprioception into unified action prediction; its design shapes grounding quality and inference cost.