Evaluating robot world models against real executionsIntermediate
WorldOracle: Benchmarking Action-Conditioned World Models against Real-World Executions
AgiBot and NVIDIA researchers built a preprint benchmark that feeds robot world models the exact actions a real robot performed and checks whether the predicted video shows what really happened, including failed grasps and spills. Every one of seven models tested drifted on long runs, and the best rollouts of the robot arm often still got the physical outcome wrong, a warning for companies that plan to evaluate or train robots inside such models.