Foundation Models

Reinforcement Learning from Human Feedback (RLHF)

Reinforcement learning from human feedback (RLHF) is a training paradigm that fits a reward model to human preference comparisons between behavior samples, then optimizes a policy against that learned reward with reinforcement learning. Building on the preference-learning work of Christiano et al. (2017), RLHF became the standard alignment stage for large language models and is being adapted to align robot behavior with hard-to-specify human expectations.

Why it matters for physical AI

Reward specification is a chronic failure point in robot RL, and preference-based reward models offer a scalable way to encode judgments like gentleness or appropriateness that resist manual formalization.