Papers

PrefPI: Preference-Guided Steering into Out-of-Distribution Behaviors

This preprint introduces PrefPI, an iterative framework that steers pretrained generative robot policies toward user-preferred behaviors using only relative preference labels on the robot's own rollouts. On a real-robot cube pick-and-place task, it lifted object transport height from 10.7 cm to 19.8 cm after 150 labeled trajectories and cut completion time from 19.8 s to 10.8 s. It matters because it shows preference feedback can push policies outside their original behavior distribution without new demonstrations or reward functions.