새로 나온 로봇, 피지컬 AI 논문과, 각 논문이 업계에 갖는 의미.
Vision-Language Models (VLMs) and world models have demonstrated strong capabilities in semantic understanding and future scene evolution for autonomous driving. However, effectively integrating both capabilities for trajectory generation remains challenging.
Evaluating robot policies before deployment typically requires producing new trajectories through real-world execution, simulators, or learned world models. These rollout-based approaches impose infrastructure requirements; simulator and world-model evaluators additionally face prediction errors that can compound over the evaluation horizon.
Action-conditioned world models can reduce costly real-robot interaction in reinforcement learning for Vision-Language-Action models (VLA-RL), but hallucinations can mislead policy optimization. Extending them to richer VLA interfaces and long-horizon tasks raises challenges in spatiotemporal and cross-modal consistency, exploration-reliability balance in RL, and compounding rollout errors.
Occupancy world models provide a structured, metric representation of future driving scenes, making them a promising basis for end-to-end autonomous driving. However, existing occupancy-aware planners typically couple occupancy prediction and action generation through either hierarchical coupling or parallel multi-task learning.
A robot's world model must remember objects that leave the camera's view. FloWM, a learned world model, keeps them in a spatial grid of features that it moves with five built-in velocities.
World-model-based visual navigation enables action planning by predicting future states under candidate trajectories. However, long-horizon rollout remains challenging: models are often trained with ground-truth visual histories but deployed in closed-loop prediction, and evaluating many candidates with step-by-step rollout is computationally expensive.
World models (WMs) and world-action models (WAMs) focus on future world prediction and executable action generation, respectively. Although these models learn shared physical dynamics and interaction semantics from embodied trajectories, they are typically trained in isolation, which restricts knowledge reuse between prediction and control.
World models have emerged as a promising paradigm for embodied AI, yet existing systems often remain fragmented in the modalities they model and the functions they support. In particular, visual dynamics, geometric structure, and executable action are commonly coupled through separate interfaces or downstream modules rather than being jointly represented and evolved within a shared dynamics model.
We introduce WorldOracle, a benchmark specifically designed to evaluate action-conditioned world models (ACWMs) against paired real-world executions. Given the same initial observation and low-level action sequence, WorldOracle asks whether a predicted rollout reproduces what happens in the real world, including physical consequences and task-state transitions.
Learned world models for autonomous driving operate in unconstrained latent spaces, yet the physical systems they represent are governed by strict non-holonomic constraints: ground vehicles cannot realize instantaneous pure lateral motion. We study long-horizon latent rollout modeling for autonomous driving and ask whether rollout quality can be improved by assigning a motion-aware geometry to latent transitions, rather than by redesigning the predictor architecture.
World Action Models (WAMs), often built upon pretrained video-generation world models, have emerged as a promising framework for general-purpose and open-ended robot control. However, their real-world deployment often suffers from discontinuous and jerky execution.
Joint-embedding predictive architectures (JEPAs) learn world models directly in latent space, but unconstrained autoregressive predictors may contract the latent distribution over long rollouts. We introduce Symplectic Hamiltonian JEPA (LeHami), which replaces the standard predictor with an action-conditioned Hamiltonian flow over a structured latent state (q, p) integrated using an exactly symplectic leapfrog step.
Scaling robot manipulation policies requires large amounts of diverse, action-labeled data, yet collecting such data in the real world remains expensive and difficult to scale. Existing methods either depend on human teleoperation, rely on simulation with limited visual realism, or use human videos that lack embodiment-specific annotations.
Generalist robot policies are trained with imitation learning and are bounded by the quality and coverage of their demonstrations. Reinforcement learning can improve and extend them, either by fine-tuning the policy weights or by training a small module to steer the input noise of a frozen generative policy.
Action-conditioned world models offer a promising foundation for generalist robotic intelligence by predicting the consequences of actions. However, existing robotic datasets provide limited coverage of the diverse interaction dynamics required for generalizable world modeling.
AVL-JEPA는 행동 조건부 물리 변화를 잃는 JEPA 세계 모델의 '인과 동역학 정보 붕괴'를 막기 위해 행동 앵커와 시각 불변 경로를 결합한 방법이다. TwoRoom, PushT, OGBench Cube, Reacher 시뮬레이션 평가에서 가우시안 노이즈와 채도 변화 아래 성공률을 크게 높였고, TwoRoom의 물리 결과 정렬 Spearman 상관을 0.35에서 0.53으로 올렸다. 이 논문은 프리프린트이며 동료 심사 여부는 확인되지 않았다.
이 논문은 프리프린트로, 잠재 세계 모델 계획에서 후보 행동을 평가하는 잠재 상태가 성공 기준 물리량을 충분히 정밀하게 담지 못하는 문제를 해결하고자 훈련 중에만 성공 기준 물리량을 회귀하는 보조 손실을 제안했다. LeWM에서 PushT는 3.5%포인트, cube는 3.4%포인트, stack은 5.05%포인트의 절대 성공률 상승을 보였고, 동결된 DINOv2 인코더를 쓰는 시스템에서도 stack 기준 최대 7.10%포인트 개선이 재현됐다. 훈련 뒤 선형 헤드를 버려 시험 시간 모델과 입력, 비용이 같아 실용적이다.
PL-MPC는 정책 제약형 TD-M(PC)2 백본에서 비평가 지도, MPPI 종단 가치 평가, 플래너-정책 전이를 동시에 바꾼 방법이다. HumanoidBench에서는 balance-hard의 TAR(Total Average Return)가 98±18에서 387±255로, hurdle은 199±13에서 466±200으로 올랐고, KUKA IIWA14 실물 실험에서는 학습 물체 크기와 미학습 크기 모두에서 TD-M(PC)2보다 높은 성공률을 보였다. 아직 동료 심사를 거치지 않은 프리프린트다.
Latent world models learn action-conditioned dynamics in representation space and often score candidate actions by Euclidean distance to a goal representation. Joint training typically regularizes the representation to prevent collapse, but the resulting representation geometry also determines how terminal errors are weighted during planning.
논문은 arXiv 로봇 피드에서 가져옵니다. 요약이 아직 작성되지 않은 논문은 초록의 첫 부분을 보여줍니다.