ロボティクスとフィジカルAIの最新論文と、それぞれが産業にもたらす意味。
Vision-Language Models (VLMs) and world models have demonstrated strong capabilities in semantic understanding and future scene evolution for autonomous driving. However, effectively integrating both capabilities for trajectory generation remains challenging.
Evaluating robot policies before deployment typically requires producing new trajectories through real-world execution, simulators, or learned world models. These rollout-based approaches impose infrastructure requirements; simulator and world-model evaluators additionally face prediction errors that can compound over the evaluation horizon.
Action-conditioned world models can reduce costly real-robot interaction in reinforcement learning for Vision-Language-Action models (VLA-RL), but hallucinations can mislead policy optimization. Extending them to richer VLA interfaces and long-horizon tasks raises challenges in spatiotemporal and cross-modal consistency, exploration-reliability balance in RL, and compounding rollout errors.
Occupancy world models provide a structured, metric representation of future driving scenes, making them a promising basis for end-to-end autonomous driving. However, existing occupancy-aware planners typically couple occupancy prediction and action generation through either hierarchical coupling or parallel multi-task learning.
A robot's world model must remember objects that leave the camera's view. FloWM, a learned world model, keeps them in a spatial grid of features that it moves with five built-in velocities.
World-model-based visual navigation enables action planning by predicting future states under candidate trajectories. However, long-horizon rollout remains challenging: models are often trained with ground-truth visual histories but deployed in closed-loop prediction, and evaluating many candidates with step-by-step rollout is computationally expensive.
World models (WMs) and world-action models (WAMs) focus on future world prediction and executable action generation, respectively. Although these models learn shared physical dynamics and interaction semantics from embodied trajectories, they are typically trained in isolation, which restricts knowledge reuse between prediction and control.
World models have emerged as a promising paradigm for embodied AI, yet existing systems often remain fragmented in the modalities they model and the functions they support. In particular, visual dynamics, geometric structure, and executable action are commonly coupled through separate interfaces or downstream modules rather than being jointly represented and evolved within a shared dynamics model.
We introduce WorldOracle, a benchmark specifically designed to evaluate action-conditioned world models (ACWMs) against paired real-world executions. Given the same initial observation and low-level action sequence, WorldOracle asks whether a predicted rollout reproduces what happens in the real world, including physical consequences and task-state transitions.
Learned world models for autonomous driving operate in unconstrained latent spaces, yet the physical systems they represent are governed by strict non-holonomic constraints: ground vehicles cannot realize instantaneous pure lateral motion. We study long-horizon latent rollout modeling for autonomous driving and ask whether rollout quality can be improved by assigning a motion-aware geometry to latent transitions, rather than by redesigning the predictor architecture.
World Action Models (WAMs), often built upon pretrained video-generation world models, have emerged as a promising framework for general-purpose and open-ended robot control. However, their real-world deployment often suffers from discontinuous and jerky execution.
Joint-embedding predictive architectures (JEPAs) learn world models directly in latent space, but unconstrained autoregressive predictors may contract the latent distribution over long rollouts. We introduce Symplectic Hamiltonian JEPA (LeHami), which replaces the standard predictor with an action-conditioned Hamiltonian flow over a structured latent state (q, p) integrated using an exactly symplectic leapfrog step.
Scaling robot manipulation policies requires large amounts of diverse, action-labeled data, yet collecting such data in the real world remains expensive and difficult to scale. Existing methods either depend on human teleoperation, rely on simulation with limited visual realism, or use human videos that lack embodiment-specific annotations.
Generalist robot policies are trained with imitation learning and are bounded by the quality and coverage of their demonstrations. Reinforcement learning can improve and extend them, either by fine-tuning the policy weights or by training a small module to steer the input noise of a frozen generative policy.
Action-conditioned world models offer a promising foundation for generalist robotic intelligence by predicting the consequences of actions. However, existing robotic datasets provide limited coverage of the diverse interaction dynamics required for generalizable world modeling.
本論文はプレプリントである。DeltaWorldは、ロボット操作映像の将来予測において、次の潜在状態全体ではなく動作が引き起こす潜在変化を予測するDelta-LTMと、ロボットと物体の相互作用領域を強調するInteraction-aware Latent Alignmentを導入した世界シミュレータである。独自収集のクロスロボットデータセットでIWS比FVDを46.6%、LPIPSを31.1%削減し、長距離の動作条件付き映像予測の一貫性を改善した。
AVL-JEPAは、JEPA世界モデルが視覚情報を保持しながら行動の物理的帰結情報を失う「因果ダイナミクス情報崩壊」を防ぐ手法である。TwoRoom、PushT、OGBench Cube、Reacherの4タスクで評価し、ガウスノイズ下のPushTで成功率が15.3%から81.5%へ、TwoRoomで65.7%から88.7%へ向上した。プレプリントであり、査読済みではない。
本プレプリントは、潜在世界モデルのプランニングにおいて、タスクの成功判定に使われる物理量（成功基準量）を補助損失の教師信号として直接学習させる手法を提案した。LeWMを用いたPushTとcubeで成功率がそれぞれ3.5ポイント、3.4ポイント（絶対差）向上し、統計的に有意だった。テスト時は線形ヘッドを破棄するためモデルや計算コストは変わらず、成功基準に含まれる全量を教師にすることで有意な劣化は見られなかった。
ロボット動画生成モデルが、テキストで指定された質量や粘性など外見から判断できない物理特性を無視する問題を測るベンチマークPhysicsLENSを開発した。本論文はプレプリントであり、査読済み採録は明記されていない。同一フレームとタスクで物性記述だけを変えたマッチドシナリオを用意し、4モデル439本の動画を人で評価した結果、物理的に妥当に見える47本中34本が指定特性に従っていなかった。自動評価器は妥当性AUC 0.72、タスク完了AUC 0.81を達成したが、隠れた特性の判定は難しいと報告した。
プレプリントとして公開された本研究では、TD-M(PC)2を基盤に、批評家の学習目標を多段階化するMTD、計画時の終端価値に不確実性ペナルティを加えるATE、実現報酬で重み付けした模倣を行うRADを組み込んだPL-MPCを提案した。HumanoidBenchのbalance-hardでTotal Average Returnが98±18から387±255へ、hurdleで199±13から466±200へ向上し、実機のKUKA IIWA14によるレンチとナットの位置合わせでもTD-M(PC)2を上回る成功率を報告した。
論文は arXiv のロボティクス分野から取得しています。当社の要約が未作成の場合は、要旨の冒頭を表示します。