Robot Learning

Soft Actor-Critic (SAC)

Soft Actor-Critic (SAC) is an off-policy, maximum-entropy deep reinforcement learning algorithm (Haarnoja et al., 2018) that trains a stochastic policy to maximize expected return plus policy entropy, using twin Q-networks and a replay buffer. The entropy term encourages exploration and improves robustness, while off-policy reuse of past data yields strong sample efficiency. SAC became a default choice for continuous-control robotics, including learning quadruped locomotion and manipulation skills directly on hardware.

Why it matters for physical AI

Sample-efficient, stable off-policy learning is what makes reinforcement learning on physical robots feasible at all, and SAC remains the backbone of most real-world RL pipelines.