A Unified Framework for Empowerment and Predictive Control
This preprint presents a controller that turns empowerment, an information-theoretic measure of future controllability, into an objective for sampling-based model predictive control. It uses the same action distribution for probing and executed control and needs only first-order derivatives. On simulated pendulum swing-up tasks with sparse rewards, combining this intrinsic objective with a task cost improved success rates up to 100%, while task cost alone failed.
Can a single action distribution unify probing and control so that empowerment becomes a trajectory-level objective for sampling-based model predictive control, and does adding that intrinsic objective to sparse task costs improve closed-loop performance?
Sampling-based MPC depends on informative cost functions, but sparse task rewards provide almost no learning signal. Existing empowerment controllers estimate channel capacity with a virtual probing policy separate from the executed control policy and require second-order dynamics derivatives for the control update, making them hard to integrate with standard MPC and unstable in some systems.
Earlier empowerment-based controllers use a zero-mean Gaussian probing distribution for channel capacity via water-filling, then derive actions by gradient ascent on the empowerment estimate, requiring second-order derivatives. Sampling-based MPC variants such as random shooting, predictive sampling, CEM, and MPPI usually optimize dense hand-shaped costs; with sparse costs they fail to find swing-up trajectories.
The method allows the probing distribution mean to be nonzero and optimizes that mean control sequence with zero-order MPC, while water-filling still optimizes the covariance. Empowerment is evaluated from a first-order linearization of T-step dynamics. Both empowerment and a task cost are evaluated on the same sampled trajectories in a combined objective J = beta*E + C, so probing and executed control are the same sequence.
Across three simulated pendulum swing-up tasks initialized downward, task cost alone failed for random shooting, predictive sampling, CEM, and MPPI. Combining empowerment with a sparse upright-cone cost improved success over empowerment alone: CEM single pendulum 100% vs 59%, predictive sampling cart pole 100% vs 81%, and predictive sampling double pendulum 86% vs 0%. Other combined results include random shooting single pendulum 95%, MPPI single pendulum 93%, CEM cart pole 99%, and CEM double pendulum 46%. Evaluation used 10 random seeds. In a hopper fall-recovery test, the method recovered from a fallen configuration and stabilized in a crouched posture, whereas classical empowerment failed and exhibited large oscillations.
The authors note strong sensitivity to hyperparameters, especially planning horizon, with non-monotonic effects; large numbers of parallel rollouts are required for stable optimization, particularly in complex environments; and the linear-Gaussian empowerment estimate may not scale. The evaluation is simulation-only and limited to classical control and a single hopper, without real-robot or high-dimensional manipulation/locomotion validation. The work is a preprint without peer review.
Companies building sampling-based MPC controllers for legged robots, manipulators, or autonomous systems could use this as a task-agnostic shaping signal when dense rewards are hard to engineer. Before deployment, horizon tuning would need to be automated, sample efficiency improved, and results reproduced on real hardware. If those issues are addressed, practical use in contact-rich locomotion or sparse-reward manipulation is plausible within 1 to 3 years; in its current form it is more a research asset than a production controller.
The full text is not republished here because the paper's license does not allow it. Read the original on arXiv.