Robot Learning

Layer Normalization

Layer normalization is a neural network technique, introduced by Ba et al. in 2016, that normalizes the activations of a layer across the feature dimension for each individual sample, then applies a learned scale and shift. Unlike batch normalization it is independent of batch size and behaves identically at training and inference time, making it well suited to transformers and recurrent networks, where it is now standard.

Why it matters for physical AI

Stable normalization is one reason transformer-based robot policies train reliably; it also suits the small or streaming batch regimes common in on-robot learning where batch statistics are unreliable.