Robot Learning

Q-Function (Action-Value Function)

A Q-function (action-value function) is the expected cumulative discounted reward obtained by taking a given action in a given state and following a policy thereafter. It satisfies the Bellman equation, and the optimal Q-function directly induces an optimal policy through greedy action selection. Q-functions are the central object of value-based reinforcement learning, estimated with neural networks in deep RL methods such as DQN and SAC.

Why it matters for physical AI

Value estimates let robots rank candidate actions without exhaustive physical trials, and learned Q-functions underpin offline RL, grasp quality scoring, and policy improvement on real hardware.