Robot Learning

Deep Deterministic Policy Gradient (DDPG)

Deep Deterministic Policy Gradient (DDPG) is an off-policy actor-critic algorithm for continuous action spaces, introduced by Lillicrap et al. in 2015. It combines the deterministic policy gradient theorem with DQN-style innovations, using a replay buffer and target networks to stabilize learning of a Q-function critic and a deterministic actor. Successors such as TD3 and SAC address its overestimation bias and brittleness to hyperparameters.

Why it matters for physical AI

Continuous torque and velocity commands are the native action space of robots, and DDPG was among the first deep RL methods to handle them directly. Its descendants still power simulation-trained locomotion and manipulation controllers.