Robot Learning

Residual Reinforcement Learning

Residual reinforcement learning trains a policy to output corrective actions that are added to a base controller, such as a hand-designed feedback law or model-based planner, rather than learning control from scratch. Introduced concurrently by Johannink et al. and Silver et al. around 2018, the decomposition lets the base controller handle nominal behavior and safety while the learned residual absorbs unmodeled effects, excelling in contact-rich tasks like insertion.

Why it matters for physical AI

Starting from a competent controller slashes exploration cost and risk on physical hardware, making residual learning one of the most deployment-friendly ways to add learning to existing industrial systems.