Data & Benchmarks

Success Rate

Success rate is the fraction of evaluation trials in which a robot completes a task according to a predefined success criterion, and it is the primary headline metric in manipulation and robot learning research. Meaningful reporting requires specifying the number of trials, initial condition distribution, object sets, and success detection method, since small trial counts carry wide confidence intervals. Cross-lab comparability remains poor because physical setups differ in unmeasured ways.

Why it matters for physical AI

Claims about robot foundation models rise or fall on evaluation rigor; understanding what a reported success rate does and does not control for is essential to interpreting progress.