Robot Learning

Behavioral Regularization

Behavioral regularization is a family of techniques, most prominent in offline reinforcement learning, that constrain a learned policy to stay close to the behavior policy that generated the training data, penalizing divergence measured by KL divergence, action-space distance, or implicit weighting. Algorithms such as TD3+BC, AWAC, and advantage-weighted regression instantiate it, preventing the policy from exploiting value-function errors on out-of-distribution actions.

Why it matters for physical AI

Improving beyond demonstrations without real-world exploration requires offline reinforcement learning, and behavioral regularization is the mechanism that keeps such improvement from hallucinating actions the data cannot support.