Multi-fidelity RLExpert
Multi-Fidelity Policy Gradients Stabilize Data-Scarce Reinforcement Learning
This preprint introduces MFPG-PPO, an algorithm that uses abundant low-fidelity simulator data as a control variate to reduce variance in scarce high-fidelity policy-gradient updates. On the hardest simulated locomotion task it matches PPO trained with 16 times more high-fidelity data, and on a physical Franka arm it reaches 91.7% success with only 4 real episodes per update, versus 55.0% for PPO on high-fidelity data alone. If validated, the approach could make on-policy robot learning much more sample-efficient in real-world deployments.