Robot Learning

Preference Learning

Preference learning is the inference of objectives from comparative judgments, typically a human indicating which of two trajectories better accomplishes a task, rather than from hand-specified rewards or demonstrations. Christiano et al. (2017) showed that deep reinforcement learning from such pairwise preferences can train complex behaviors from surprisingly few queries, establishing the reward-model recipe later scaled into RLHF for language models.

Why it matters for physical AI

Many robot objectives, such as gentle handling or socially acceptable motion, are easier to recognize than to formalize, and preference queries give non-experts a practical channel for specifying them.