Robot Learning

N-Step Return

An n-step return is a reinforcement learning target that sums n discounted rewards before bootstrapping from a value estimate at the n-th subsequent state. It interpolates between one-step temporal-difference learning (low variance, high bias) and full Monte Carlo returns (high variance, no bootstrap bias). Algorithms such as A3C and Rainbow use multi-step targets to propagate reward information faster through the value function.

Why it matters for physical AI

Reward signals in robot tasks are often sparse and delayed, so multi-step targets speed up credit assignment and stabilize value learning in RL-based controllers and offline RL pipelines.