Robot Learning

Cosine Annealing

Cosine annealing is a learning rate schedule that decays the learning rate from its initial value to a minimum following a half-cosine curve over training, introduced by Loshchilov and Hutter (2017) as part of SGDR, which also proposed warm restarts. The smooth decay avoids abrupt transitions of step schedules, and combined with linear warmup it is the default schedule for training transformers, including vision-language-action models.

Why it matters for physical AI

Training recipes for large robot policies are sensitive to schedule choices, and cosine decay with warmup is the de facto standard that practitioners tune when pretraining or fine-tuning foundation models on robot data.