Robot Learning

Model-Free Reinforcement Learning

Model-free reinforcement learning is the family of RL methods that learn policies or value functions directly from interaction without constructing an explicit dynamics model. Major branches include policy-gradient and actor-critic methods such as PPO and SAC, and value-based methods such as Q-learning. Model-free algorithms are simpler and asymptotically strong but sample-hungry, which is why robotics applies them mainly inside massively parallel simulators.

Why it matters for physical AI

PPO running in GPU-parallel simulation is the engine behind nearly all deployed learned locomotion; the paradigm's sample cost is what pushed the field toward simulation-first training and sim-to-real transfer.