Robot Learning

TD3 (Twin Delayed DDPG)

TD3 (Twin Delayed Deep Deterministic Policy Gradient) is an off-policy actor-critic reinforcement learning algorithm for continuous control, introduced by Fujimoto et al. in 2018 as a successor to DDPG. It curbs the overestimation bias that destabilizes DDPG through three mechanisms: twin critics with a minimum-value target, delayed policy updates, and target policy smoothing via clipped noise. TD3 remains a standard baseline alongside SAC and PPO on continuous-control benchmarks.

Why it matters for physical AI

Sample-efficient off-policy algorithms like TD3 make reinforcement learning tractable for robot control, where every real-world trial is expensive, and its stabilization tricks inform the training of value functions used in modern robot learning pipelines.