Battery-Aware Reinforcement Learning for Aggressive Quadrotor Flight

Does giving a learned quadrotor controller additional thrust-command range and an explicit filtered battery-voltage input improve aggressive tracking and racing, and how much benefit comes from voltage feedback versus the command range alone?

Battery discharge and load-induced voltage sag reduce available thrust during flight, so maneuvers that are feasible early in a flight can become hard or infeasible later. Existing flight controllers use conservative command limits and voltage compensation to tolerate this variation, but conservative limits leave usable performance unused, and learned controllers may not react early enough through motion feedback alone.

Prior approaches include firmware voltage compensation such as Betaflight sag compensation and ArduPilot voltage scaling, battery-constrained NMPC, and learned racing policies that simulate battery effects but do not expose voltage directly to the actor. Other adaptive methods use L1 disturbance estimates or recurrent world models from motion history. These methods either retain conservative authority, rely on indirect motion feedback, or command motors directly rather than isolating explicit voltage feedback with an existing rate controller.

The authors train a Gaussian feedforward PPO policy in Isaac Lab/Isaac Sim with a coupled load-transient battery model, voltage-dependent rotor-speed dynamics, and deployed firmware compensation and motor saturation. The policy commands collective thrust and rates through the existing onboard controller. Compared policies are Stock (default thrust-command limit, no voltage input), High (higher limit, no voltage), and High-V (higher limit plus a 10 s filtered voltage input). Training includes domain randomization and a battery-privileged critic. The hardware platform is a 38 g Crazyflie Brushless with onboard voltage logging.

On real hardware across 50 circle-tracking flights, all methods track 3.36 m/s circles at about 4 cm RMSE. At 3.84 m/s, RMSE is 14.26 cm for Stock, 8.53 cm for High, and 7.23 cm for High-V: High-V improves 49.3% over Stock and 15.3% over High. At a 4.08 m/s stress speed, High-V remains best while High falls behind Stock. In 27 evaluated races, all flights complete. Mean 20-lap time is 106.22 s for Stock, 99.76 s for High, and 95.24 s for High-V, a 10.3% improvement over Stock and 4.5% over High. Mean long-flight peak speed reaches 6.95 m/s for High-V versus 6.08 m/s for Stock. In simulation, supplying the policy with voltage from the opposite battery condition worsens hard-circle tracking and increases racing lap time by 1.84% at 3.70 V with about zero effect at 4.20 V. Battery-model validation reduces voltage RMSE from 50.2 mV to 32.1 mV and high-load 95th-percentile absolute error from 161.9 mV to 83.6 mV on held-out flights.

The authors note future work should combine voltage feedback with motion-history adaptation and extend the model to current and power prediction. Simulation under-predicts tracking error at the 4.08 m/s stress speed, and the motion-capture area constrains the racing course to short acceleration segments. The study uses a single small Crazyflie Brushless platform, limited trials (50 circle flights, 27 races, with only three 20-lap trials per method), controlled indoor motion capture, and one battery model identified/validated on one vehicle. Code release is not reported in the paper.