Neue Studien zu Robotik und Physical AI, jeweils mit ihrer Bedeutung für die Branche.
Integrating world models into Vision-Language-Action models (VLAs) has emerged as a promising direction, enabling policies to anticipate future dynamics beyond reactive observation-to-action mapping. However, existing approaches vary simultaneously in their world representations, integration paradigms, and policy backbones, making it difficult to determine what truly matters for effective world model integration.
Foundation models are driving a paradigm shift in robotic intelligence, moving beyond low-level action prediction to goal-directed orchestration of data collection, model training, task planning, and real-world execution. We characterize this emerging paradigm as an agentic robots, in which agents reason over human instructions, interact with robotic environments, rectify execution errors autonomously, and dynamically adjust operational strategies.
Vision-Language-Action (VLA) models are gaining attention in autonomous driving, inheriting commonsense priors from large language models (LLMs) and widening the navigation interface from a fixed command set to free-form natural-language instructions. However, this open-ended interface exposes a new vulnerability surface: an inattentive driver may issue instructions that are unsafe given the current scene, and a malicious user may issue dangerous ones on purpose.
Vision-Language Models (VLMs) and world models have demonstrated strong capabilities in semantic understanding and future scene evolution for autonomous driving. However, effectively integrating both capabilities for trajectory generation remains challenging.
Evaluating robot policies before deployment typically requires producing new trajectories through real-world execution, simulators, or learned world models. These rollout-based approaches impose infrastructure requirements; simulator and world-model evaluators additionally face prediction errors that can compound over the evaluation horizon.
Action-conditioned world models can reduce costly real-robot interaction in reinforcement learning for Vision-Language-Action models (VLA-RL), but hallucinations can mislead policy optimization. Extending them to richer VLA interfaces and long-horizon tasks raises challenges in spatiotemporal and cross-modal consistency, exploration-reliability balance in RL, and compounding rollout errors.
Occupancy world models provide a structured, metric representation of future driving scenes, making them a promising basis for end-to-end autonomous driving. However, existing occupancy-aware planners typically couple occupancy prediction and action generation through either hierarchical coupling or parallel multi-task learning.
A robot's world model must remember objects that leave the camera's view. FloWM, a learned world model, keeps them in a spatial grid of features that it moves with five built-in velocities.
World-model-based visual navigation enables action planning by predicting future states under candidate trajectories. However, long-horizon rollout remains challenging: models are often trained with ground-truth visual histories but deployed in closed-loop prediction, and evaluating many candidates with step-by-step rollout is computationally expensive.
World models (WMs) and world-action models (WAMs) focus on future world prediction and executable action generation, respectively. Although these models learn shared physical dynamics and interaction semantics from embodied trajectories, they are typically trained in isolation, which restricts knowledge reuse between prediction and control.
World models have emerged as a promising paradigm for embodied AI, yet existing systems often remain fragmented in the modalities they model and the functions they support. In particular, visual dynamics, geometric structure, and executable action are commonly coupled through separate interfaces or downstream modules rather than being jointly represented and evolved within a shared dynamics model.
Forscher von NVIDIA und dem KAIST beschreiben in einem Preprint, wie ein von einem Basismodell gesteuerter Roboter neue Montageaufgaben nacheinander lernt, ohne alte zu vergessen, indem er ein kleines Verstärkungslern-Modul von Aufgabe zu Aufgabe mitnimmt. Am realen Arm sank die Roboterzeit bis zur Beherrschung jeder neuen Fügeaufgabe um 31,0 bis 68,5 %, frühere Aufgaben gelangen weiterhin in durchschnittlich 19,4 von 20 Versuchen.
Forscher von AgiBot und NVIDIA stellen in einem Preprint einen Benchmark vor, der Welt-Modellen exakt die Aktionen eines realen Roboters eingibt und prüft, ob das vorhergesagte Video zeigt, was tatsächlich geschah, inklusive verfehlter Griffe und verschütteten Wassers. Alle sieben getesteten Modelle verloren über lange Zeiträume an Genauigkeit, und selbst das Modell mit der besten Armführung sagte das physische Ergebnis oft falsch voraus.
Learned world models for autonomous driving operate in unconstrained latent spaces, yet the physical systems they represent are governed by strict non-holonomic constraints: ground vehicles cannot realize instantaneous pure lateral motion. We study long-horizon latent rollout modeling for autonomous driving and ask whether rollout quality can be improved by assigning a motion-aware geometry to latent transitions, rather than by redesigning the predictor architecture.
World Action Models (WAMs), often built upon pretrained video-generation world models, have emerged as a promising framework for general-purpose and open-ended robot control. However, their real-world deployment often suffers from discontinuous and jerky execution.
Joint-embedding predictive architectures (JEPAs) learn world models directly in latent space, but unconstrained autoregressive predictors may contract the latent distribution over long rollouts. We introduce Symplectic Hamiltonian JEPA (LeHami), which replaces the standard predictor with an action-conditioned Hamiltonian flow over a structured latent state (q, p) integrated using an exactly symplectic leapfrog step.
Scaling robot manipulation policies requires large amounts of diverse, action-labeled data, yet collecting such data in the real world remains expensive and difficult to scale. Existing methods either depend on human teleoperation, rely on simulation with limited visual realism, or use human videos that lack embodiment-specific annotations.
Generalist robot policies are trained with imitation learning and are bounded by the quality and coverage of their demonstrations. Reinforcement learning can improve and extend them, either by fine-tuning the policy weights or by training a small module to steer the input noise of a frozen generative policy.
Action-conditioned world models offer a promising foundation for generalist robotic intelligence by predicting the consequences of actions. However, existing robotic datasets provide limited coverage of the diverse interaction dynamics required for generalizable world modeling.
Ein Preprint des chinesischen Humanoid-Herstellers Galbot und der Chinesischen Akademie der Wissenschaften prüft, welche fertigen Bildencoder videobasierte Robotersteuerungen zuverlässiger machen, und kommt zu dem Schluss, dass ein einzelner starker Universal-Encoder den Großteil des Gewinns liefert. Das daraus abgeleitete Rezept erzielt im Humanoid-Benchmark RoboCasa GR1 einen Bestwert von 84,7 Prozent und hebt die Erfolgsquote bei drei Aufgaben auf einem realen Galbot-Roboter um jeweils 10 Punkte.
Die Studien stammen aus den Robotik-Feeds von arXiv. Solange unsere Zusammenfassung fehlt, erscheinen die ersten Zeilen des Abstracts.