Watch, Infer, Coordinate: Inferring Robot Partner Constraints for Zero-Shot Coordination
Preprint. The authors introduce Watch, Infer, Coordinate, a benchmark and inference method for a helper robot to watch a constrained partner collaborate with a demonstrator, infer the partner's low-level physical constraints, then coordinate zero-shot on a new joint-carrying task. In simulation across three manipulation settings, the approach reaches 85–93% success on obstacle tasks, close to an oracle that knows the true constraints, and far above baselines. This matters for multi-robot fleets where one unit may degrade or fault and partners must adapt without explicit communication.