Foundation Models

Encoder-Decoder Architecture

Encoder-Decoder Architecture is a neural network pattern in which an encoder compresses the input into a latent representation and a decoder expands that representation into the output. It spans sequence-to-sequence models with attention, the original transformer of Vaswani et al. (2017), U-Net's skip-connected variant for dense prediction, and autoencoders. The design separates understanding the input from generating the output, letting the two sides use different modalities or resolutions.

Why it matters for physical AI

Robot policies are frequently encoder-decoder systems, vision and language encoders feeding action decoders, and diffusion policies use U-Net or transformer decoders, so the pattern's trade-offs shape policy architecture design.