Foundation Models

Next-Token Prediction

Next-token prediction is the training objective in which a model learns to predict the next discrete token in a sequence given all previous tokens, the recipe behind GPT-style large language models. Applied to robotics, actions, states, and sensor readings are discretized into tokens so a single autoregressive transformer can model trajectories, as in RT-1, RT-2, and Gato.

Why it matters for physical AI

Casting control as sequence modeling lets robotics borrow architectures, scaling laws, and pretraining infrastructure from language modeling, a foundation of current vision-language-action models.