Robot Learning

Actor-Critic Methods

Actor-critic methods are a family of reinforcement learning algorithms that maintain two learned components: an actor, the policy that selects actions, and a critic, a value function that evaluates them. The critic's estimates reduce the variance of policy gradient updates relative to pure Monte Carlo returns. Modern instances such as PPO, SAC, and TD3 dominate continuous-control robotics, powering most simulation-trained locomotion and dexterous manipulation results.

Why it matters for physical AI

Nearly every large-scale simulation training pipeline for legged robots and dexterous hands rests on an actor-critic algorithm, making their stability and sample efficiency a direct constraint on what behaviors can be trained.