vision-language-action modelsIntermediate
Do VLAs Really Follow Instructions? Diagnosing How-Level Instruction Following in Vision-Language-Action Models
A KAIST preprint finds that today's robot AI models ignore instructions about how to do a task, such as which part of an object to grasp, and that such hints actually lower success. Re-labelling training data with step-by-step descriptions made the robots respond to corrections in 84% of cases, but did not reliably make them succeed more.