Robot Learning

Implicit Q-Learning (IQL)

Implicit Q-Learning is an offline reinforcement learning algorithm (Kostrikov, Nair, and Levine, 2021) that avoids querying the value of out-of-distribution actions by fitting a state-value function with expectile regression, then extracting a policy via advantage-weighted regression. Its stability and simplicity made it a standard baseline for offline RL benchmarks like D4RL and a common choice for learning from static robot datasets.

Why it matters for physical AI

Robots accumulate large logs of mixed-quality experience that cannot be safely re-explored; offline algorithms like IQL turn those static datasets into improved policies without on-robot trial and error.