Math & Kinematics

KL Divergence

KL divergence (Kullback-Leibler divergence) is an asymmetric measure of how one probability distribution differs from a reference distribution, zero only when they match. In robot learning it constrains policy updates in trust-region and proximal methods like TRPO and PPO, regularizes latent spaces in variational autoencoders, keeps fine-tuned policies close to pretrained ones, and defines the distillation loss between teacher and student models.

Why it matters for physical AI

Bounding distribution shift with KL terms is how RL fine-tuning avoids destroying pretrained competence, a stabilizing mechanism now central to adapting robot foundation models safely.