No Eyes, No Problem: What Does Touch Contribute to Blind Baoding Ball Manipulation?

Can a reinforcement-learning policy trained only in simulation rotate two balls in a real robot hand without vision, and what does binary touch add over joint sensing alone?

In-hand manipulation means constant making and breaking of contact, which is hard to simulate and hard to see because the hand hides the object. Most systems supply the object's position from cameras, markers or privileged simulator data, which are unavailable or unreliable in many real tasks.

Earlier Baoding-ball work on hardware reached about half a rotation (Nagabandi et al., Yang et al.) or about three rotations in 10 seconds at roughly 0.30 rotations per second (Yuan et al.), mostly with external object-state information, and did not report continuous runs.

The team built a simulated hand matched to the real sensor positions and joint coupling, calibrated each force-sensing resistor to an on-off threshold, and added two randomisations during training: random limits on how fast commands may change (slew-rate randomisation) and STAT, which randomly makes some touch sensors stick. The policy sees only joint encoders and 16 binary contact signals and was deployed zero-shot at 60 Hz.

Over 10 hardware trials the full method averaged 34.75 rotations (range 2 to 74), 94.8 seconds before a drop and 0.36 rotations per second, with exchange efficiency of 0.90 of the theoretical maximum. The same policy with touch zeroed managed 5.00 rotations and 19.0 seconds (MEKAWIRE: 6.95x and 5.0x lower); a policy trained without touch reached 4.40, and open-loop replay 1.70. Without the two randomisations, policies scored 0.35 and 0.50 rotations despite looking best in simulation. In simulation, removing touch raised the drop rate from about 2% to 32%. A separate run, outside the statistics, completed 112.5 rotations in 310 seconds (0.36 per second, MEKAWIRE).

One task, one hand and one pair of 55 g balls; ten trials with a standard deviation of 26 rotations show wide spread. The 112.5-rotation record is a single run outside the reported set. The command slew limit was set by hand per policy (0.65 and 0.53), the touch sensors used in simulation and on hardware were activated in different regions, and the comparison with earlier work relies on numbers the authors infer from other papers and videos.