VLA robustness benchmarkIntermediate
LIBERO-RIG: A Hardware-Setup Robustness Benchmark for Vision-Language-Action Models
Leading robot AI models that score 93 to 98% on a standard benchmark drop to about 51% success when the robot's own setup changes markedly, such as a moved camera or a slower controller. This preprint measures that weakness across nine models in 881,500 simulated episodes.