Learning Reflexive Behavior for Contact-Rich Manipulation

Can a proprioceptive reflex policy trained on simple simulated interaction primitives act as a reusable low-level execution layer that preserves task success and limits contact force when high-level commands or environmental geometry are imperfect?

High-level motion commands often encode intent without matching local instance geometry, such as a surface or hole position. Conventional position, Cartesian impedance, and hybrid force–position execution layers impose fixed tracking objectives, stiffnesses, or force-control directions, so command error can turn into excessive contact force, jamming, or instability.

Prior execution layers used IK plus joint PD control, Cartesian impedance/admittance with hand-tuned gains, or hybrid motion–force control with a specified force axis and setpoint. Learned interaction controllers such as FORGE condition a policy on a force threshold but are not a frozen reflex beneath an independently supplied task command. These approaches trade tracking against compliance mainly through gain tuning or require task-specific force-direction knowledge, which breaks down when the nominal geometry is wrong.

A PPO-trained proprioceptive policy observes 0.5 s of joint positions, joint-target errors, commanded and measured end-effector poses, and a binary compliance mode, then outputs scaled residual joint-position targets at 50 Hz. Training uses three virtual spring–damper interaction primitives (spring, plane, rail), an asymmetric actor–critic setup with privileged force information for the critic, and a reward with a force-projected compliant position reference plus an excess-force penalty. The frozen policy runs above compliant joint-level controllers on two robot types, beneath scripted trajectories or an LSTM insertion policy.

On real dual-arm box lifting, the reflex completed all five commanded penetration depths from 7.5 to 17.5 cm; IK+PD diverged at 17.5 cm. Reflex average joint position tracking error stayed below 12°, and simulated palm contact force stayed at or below the 2 N training threshold, while IK+PD force rose with commanded depth. In real peg insertion with 0.02 mm clearance, reflex success was 58% at 0 mm, 56% at 2 mm, 36% at 5 mm, and 42% at 9 mm offset, versus 22%, 18%, and 0% at 5 mm for IK+PD; estimated mean force was 6.3–7.6 N for reflex versus 17.4–18.9 N for IK+PD where both methods ran. With 0.1 mm clearance, reflex achieved 100% at 0 and 2 mm offsets and 84% at 9 mm, versus 96% and 52% for IK+PD at 0 and 9 mm; zero-offset insertion time was 1.71 s versus 6.81 s. In simulation, reflex insertion success was 99.0% at 1 mm, 93.1% at 0.1 mm, and 74.8% at 0.02 mm clearance, compared with IK+PD at 97.9%, 72.8%, and 54.6%, and Impedance at 90.0%, 39.7%, and 10.4%. Removing any one interaction primitive reduced success at unseen clearances. In simulated surface following, reflex contact-force p95 was 7.5 N on a flat surface and 7.4 N on rough surfaces, versus 7.4 N and 8.7 N for Hybrid; rough-surface tangential RMSE was 31.3 mm versus 32.6 mm.

The authors state that evaluation is limited to a narrow set of contact-rich tasks, that the primitives model only translational constraints and not rotational coupling such as opening a door with a rigid grasp, and that the reflex depends on an explicit compliance-mode input. Additional evident limitations include simulation-only surface-following evaluation, 50 real insertion trials per condition, a joint-position-error proxy rather than direct force measurement on the humanoid box-lifting robot, and no reported code or checkpoint release.