Robot Learning

Decision Transformer

Decision Transformer is an offline reinforcement learning approach, introduced by Chen et al. in 2021, that casts control as conditional sequence modeling. A causal transformer is trained on trajectories of returns-to-go, states, and actions, and at test time is conditioned on a desired return to autoregressively generate actions. This reframing replaces value bootstrapping and policy gradients with supervised sequence prediction.

Why it matters for physical AI

Treating control as sequence modeling lets robotics borrow the architectures, tooling, and scaling behavior of large language models. The paradigm influenced action-as-token designs now common in vision-language-action models.