ActiveWAM: Evidence-Aware Active Vision for World-Action Models

Can a unified world-action model learn to jointly choose which task-relevant visual evidence to retain in a finite observation window and which new camera view to acquire, while still producing end-effector actions?

Active-camera manipulation creates a retain-acquire tradeoff: changing the view can reveal new task evidence but can also push previously useful cues out of the policy's finite visual history, and keeping the view still may forgo information needed later. Existing systems often treat observation control separately from manipulation or rely on scripted heuristics, so they cannot reason about the combined consequences of arm and camera motion.

Prior work includes fixed or scripted camera policies, active-vision benchmarks such as AV-ALOHA, TAVIS, and AVR, decoupled gaze selection via look-at inverse kinematics, and test-time viewpoint search as in See2Act. World-action models such as Fast-WAM and EasyWAM-Unified unify future-video prediction with actions but do not solve evidence retention, while Stem-OB uses diffusion inversion on single observations and does not constrain which historical cues survive a view change.

ActiveWAM uses training-time task-guided history inversion built on a frozen Wan video prior. It selects task-bearing text spans from the language instruction, reads cross-attention weights, and minimizes feature distance between transformed and source histories, while a mutual-nearest-neighbor correspondence loss preserves visible temporal changes. A view-aware history adapter tags visual tokens with camera identity, measured pose, exposure time, and validity, then summarizes them into a fixed context. The same diffusion head jointly denoises future-video latents and arm/head action chunks, treating zero head motion as a valid stay action. Paired raw and transformed histories share identical future and action targets during flow-matching training, and action-only deployment uses raw observations without inversion or candidate ranking. RoboTwin-AV adds executable pan/tilt control and synchronized RGB demonstrations across 50 bimanual tasks; the physical platform is a fixed-base dual-arm robot with a RealSense D455 and a 2-DOF pan/tilt head.

On TAVIS, ActiveWAM improves by up to 17.0 points over the strongest baseline; pose-OOD gains are +13.0/+10.6 points on Head tasks and +12.0/+11.0 points on Hands tasks for GR1T2/Reachy2. On RoboTwin-AV, it scores 80.3% clean, 60.7% appearance, 65.0% pose, and 53.3% compound-shift success; the strongest baselines are 76.3/46.3/56.0/34.0% for clean/appearance/pose/compound. Ablations show compound success drops from 53.3% to 41.7% without inversion, 34.0% without history, and 23.7% with inversion and head control both off. Learned head control reaches 62.7% pose success with matched grounding, 86.3% object-goal co-visibility, 1.9 rad travel, and 1.6 s reacquisition time, versus 57.3%, 73.7%, 4.1 rad, and 2.3 s for target centering. On LIBERO-Plus, ActiveWAM reaches 76.9% versus 68.2% for raw history. In physical kitchen trials, ActiveWAM completes 66.7% of first subtasks and 50.0% of full tasks, compared with 48.3% and 23.3% for Fast-WAM; action inference is about 165 ms on an RTX 4090, about 15 Hz effective with 25 Hz interpolated commands.

The authors state the system uses a fixed-focal-length head, a short history window, and prescribed stage switching. Physical failures include targets outside the reachable pan/tilt field of view in 16.7% of failed trials, premature termination in 33.3%, manipulation errors in 40.0%, and hardware issues in 10.0%. Evaluation is confined to one dual-arm platform, a single RGB pan/tilt camera with two DOF, simulation, and 60 physical trials per method. No code release is reported, and training uses a frozen 5B video prior, which may carry substantial compute cost.