MEKAWIRE
Research

Papers

New robotics and physical-AI papers, with what each one means for the industry.

1 papers
TopicAllManipulation · 81Navigation · 76General robotics · 67Simulation · 35Robot learning · 34VLA · 33Perception · 32Safety · 21Locomotion · 20Humanoid · 13Aerial · 4World Action Models · 4World-action model · 3humanoids · 3Aerial manipulation · 2Surgical · 2Tactile · 2humanoid loco-manipulation planning · 2Manipulation · 2vision-language-action models · 2world models · 23D Assembly · 13DGS dataset · 15G Simulation · 1A 100-hour wearable dataset of people walking and working with their hands, recorded as video, whole-body motion, finger motion and touch for humanoid training · 1A 31-scene sound-triggered mobile manipulation benchmark with 14.36M binaural RIRs; an occupancy-map navigation policy reaches 75.28% in unseen homes vs 71.47% for MAGNet · 1A 3D world action model pre-trained on 1.15 million human video episodes reaches 69.0% on ten simulated dexterous-hand tasks, 11.7 points above NVIDIA's GR00T N1.6 · 1A 4-billion-parameter perception model that points at and boxes objects precisely, used as the visual backbone for robot manipulation and driving policies · 1A 4B driving VLA reaches 91.5 PDMS on NAVSIM v1 with self-distillation that trains in 2.4 hours per 10,000 samples against 33.4 hours for reinforcement learning · 1A 50 Hz force-control add-on lifts a pi0 robot model's average score on four contact-rich tasks from 15% to 80% without retraining the model on force data · 1A Franka cobot estimates a worker's shoulder and elbow flexion limits from at most 25 mug handovers, with mean error 0.045 to 0.087 on a 0 to 1 scale over 1,600 simulated cases, using a particle filter over impairment coefficients · 1A Hamiltonian, exactly symplectic predictor keeps a JEPA world model's latent volume constant and cuts early velocity error 2.8x on a visual pendulum with at most one-eighth of the training compute, though the best baseline still plans better · 1A Kwangwoon University team plans constrained two-arm and humanoid whole-body motions on a GPU by exploring the constraint surface's tangent space, cutting mean planning time on a ROBOTIS AI Worker task from 291.96 ms to 25.94 ms · 1A LoRA-tuned OpenVLA-OFT on a FAIRINO FR3 completes 39 of 42 real 3D-printing workcell transfers (92.9%) with 1.74 mm mean trajectory error · 1A Stanford VLA policy docks a simulated lunar rover 90% of the time with 51% fewer model calls and beats a modular stack in 5 of 6 dust and glare tests. · 1A UTS cobot learns latte art from one barista pour, cutting transition errors 3.4x and moving the heart from a 280 ml to a 400 ml cup without a new demonstration · 1A benchmark and 3.1k-demonstration dataset that tests whether humanoid robot policies can pick the right tool and then use it, standing still and while walking, on simulated and real Unitree G1 robots · 1A benchmark that scales manipulation tasks from 1 to 10 subgoals in fixed scenes to isolate long-horizon failure, comparing subgoal orchestration with tool-calling systems. · 1A coding agent reads three distilled demonstrations, writes and refines a robot policy program, and succeeds on 96.3% of 3,000 sealed LIBERO-PRO episodes against 89.3% for ASPIRE on the same model · 1A coding agent that rewrites task skills lifted real-robot success from 4/15 to 15/15 without retraining the pi0.5 policy underneath, and needed 75% fewer tokens when it started from simulation skills. · 1A controlled comparison of discrete-token and continuous action heads for single-task policies and PaliGemma-based VLA models, and why discrete heads fail or succeed · 1A dual memory added to the pi0.5 VLA lifts real-robot success on four memory-dependent tasks from 37.5% to 62.5% and RoboMME benchmark success from 44.51% to 49.22% · 1A fast-slow supervising harness with failure attribution lifts a frozen pi0.5 policy from 17.5% to 75.2% on new LIBERO-Pro states with a 4B planner, and reaches 31 of 40 real UR7e trials · 1A fixed-size latent memory for vision-language-action policies, trained with question answering and key-frame feature reconstruction, for long-horizon manipulation. · 1A frontier language model controlling a Franka arm learns from its own failed attempts by tracing late failures back to earlier decisions, with no demonstrations and no weight updates · 1A frozen world model rehearses earlier tasks while a noise-steering module teaches a frozen generalist policy new ones, lifting eight sequential CALVIN tasks to 74.8% from 63.7% where LoRA fell to 11.7% · 1A general chat-style vision-language model drives a robot arm through mid-level commands and live feedback, with no robot training data · 1A learnable anisotropic Gaussian regulariser for JEPA world models raises latent planning success over LeWorldModel in all four visual control environments without changing the predictor or planner · 1A lightweight verifier that predicts the consequences of a VLA's planned actions and decides how many of them to execute before looking again · 1A meta-learned rater picks which simulation scenes to generate for sim-and-real co-training, lifting real full-task success to 58/80 from 23/80 and matching expert manual alignment · 1A modular, open-source actuation platform drives tendon-driven and concentric tube continuum robots with 0.29 mm and 4.89 degree accumulated error and mounts on a da Vinci Research Kit arm for a 2.53 mm surgical forceps · 1A physics-based benchmark from KAIST that checks whether vision-language-action robot models crush or deform the objects they pick up, not only whether they finish the task · 1A quadruped with no arm pushes two stacked boxes to a precise pose by crouching as it shoves; 90.1% success in simulation and zero-shot transfer to a Unitree Go1 · 1A reinforcement-learning reaching controller trained only in Isaac Lab that guides a UR10e arm to the stem of hydroponic lettuce without touching the growing channels · 1A residual correction layer that keeps the size, energy and posture of humanoid gestures such as waving, bowing and crouching when an existing whole-body tracker executes them · 1A risk-aware tactile adapter lifts real-robot success on USB, cap and plug tasks to 73.3% against 40.0% for pi0.5, and averages 71.8% on the eight-task UniVTAC simulation benchmark with 133 million parameters · 1A robot world model trained on human videos and synthetic failures cut video error (FVD) by 52.6% against the best baseline and predicted six real policies' success rates with r = 0.983. · 1A runtime harness around a frozen pi0.5 VLA cuts safety violations from 52.6% to 6.1% on LIBERO-Safety and lifts failure recovery from 13% to 80% · 1A safety layer that decides, at every control step, whether a flipping humanoid should finish the move, abort and land on its feet, or fall in a way that protects its head and hands · 1A single RL policy that pushes, tips and grasps reaches target poses for hard-to-grasp objects in 76.39% of simulated trials, 20.52 points above the best baseline, after pretraining its geometry encoder to predict contact-induced object motion · 1A single fixed stereo camera that learns where to look lets a bimanual robot match wrist-camera policies on clear views (69% vs 64%) and more than double them when the wrist view is blocked (48% vs 22%) · 1A skill-based agent on pi0.5 lifts real Astribot S1 success from 10% to 57.5% by retraining on about 20 self-curated runs per task per round. · 1A task-space model predictive contouring controller with a constrained joint-velocity layer keeps a UR5e tool 20 to 22% closer on average to a path whose frame is knocked mid-motion than PI control or time-indexed MPC, while peak error stays unchanged · 1A training add-on for VLA models such as OpenVLA-OFT and pi0.5 that keeps, rather than discards, background and lighting cues and lifts success under unseen conditions · 1A vision-language task predictor reading a self-revising finite-state memory of the user's routine, with pi0.5 as executor, completes 41 of 60 proactive desk-assistance tasks on a Franka arm against 23 for pi0.5 alone · 1A vision-language-action policy that predicts the visual effect of its next action, trained end to end on robot demonstrations and action-free human videos. · 1A world model that rolls out skill experts and logs the first failed subtask decides where real demonstrations go, lifting real success to 75.0% from 13.3% on Franka and 83.8% from 40.0% on AgileX with 150 extra demos · 1A zero-shot planner that asks a vision-language model where to hold an object for a task reaches 94.5% on TaskGrasp against 73.7% for the best prior method and 93 of 100 real grasps on a Franka Research 3 · 1AMR Operations · 1Active perception for VLA policies: training data aligned to camera-move endpoints · 1Active vision manipulation · 1Adapting a frozen flow-matching VLA by learning only the noise it starts from · 1Adapting a frozen world-model driving planner to an unseen low-level controller by correcting its imagined future with a controller code learned from planned and executed trajectory pairs. · 1Adaptive MPC · 1Adaptive action horizons · 1Adaptive intervention · 1Adding a small latent world model to a pi0.5 policy so that predicted consequences of a planned action condition the action generator, without video generation or large-scale video pretraining · 1Adding a tactile expert to the pi0.5 vision-language-action model so a robot learns contact-heavy tasks such as wiping, inserting and picking up chips from 50 demonstrations, without a separate tactile pretraining stage · 1Aerial navigation · 1Agricultural Robotics · 1Ajou University and Samsung Electronics researchers train VLA policies with 3D motion targets reconstructed from ordinary video, raising real UR5e hang-cup success from 46.7% to 86.7% with no extra sensor or inference cost · 1An ACT policy that outputs only end-effector force and torque, trained on bilateral force-feedback teleoperation data, averages 76.8% on five contact-rich assembly tasks against 40% for a position-based baseline · 1An LLM agent evolves staged reward programs that a frozen object-aware humanoid controller executes without retraining, lifting success on eight new loco-manipulation task families to 86.5% from 34.6% · 1An energy-based control layer lifts Galaxea's G0.5 VLA from 63.3% to 91.7% on real bimanual tasks and flags failures with 0.92 AUROC, so a robot asks for help before it fails · 1An example-driven autoregressive diffusion model with a style encoder and an RL tracker lets a Unitree G1 humanoid walk in unseen styles under joystick control, running about four hours without falls on an onboard Jetson Thor · 1An object-centric VLA that tracks each object's identity and state and learns to predict how objects will move under the robot's actions · 1An open-source ROS 2 controller cuts control-mode switching on a Franka arm from about 210 ms to 0.01 ms and lets reflexes act before factory safety stops. · 1An oracle-fed Kalman tracker matches the FloWM world model's out-of-view object memory with a 14 KB state against 5.6 MB and keeps 93% of blocks on diagonal motion where FloWM finds 11% · 1Articulation Estimation · 1Asking pi0.5 for an intent such as clearing a table instead of naming the action costs 15.2 points of task success, and people decompose multi-step requests less as tasks grow · 1Atomic-WAM puts a reasoner in front of Nvidia's Cosmos3-Nano world action model and lifts RoboLab success under vague instructions from 20.6% to 31.6%. · 1Audit of seven robot-learning benchmarks finds 22 bugs and 4 design flaws that make sped-up VLA models look better than the original · 1Autonomous Driving · 1Autonomous Exploration · 1BAAI's Human2Dex trains two different robot hands from one set of human wrist-camera videos, reaching 90.8% success on the Linker O6 and 80.8% on the 20-DoF Wuji hand across six real tasks, up from 26.3% with raw human data. · 1Battery-aware RL · 1Belief-space planning · 1Benchmark for adding wrist force sensing to VLA manipulation policies · 1Benchmark for few-shot skill transfer with dexterous robot hands and tactile sensing · 1Bimanual handover planning with heterogeneous industrial grippers · 1Bimanual manipulation: training priors for joint-space VLA policies · 1Black-box correction of deployed robot policies with world-model planning · 1Blending operator corrections with the policy's own actions during interactive imitation learning lifted real-robot success to 50% from 15% and to 70% from 40% against HG-DAgger, with 57% smoother hand-overs · 1Blind in-hand manipulation: sim-to-real Baoding ball rotation with binary touch · 1Byzantine localization · 1Canal inspection · 1CapPro lets a pi0.5 robot policy attempt a new manipulation task after watching one human video, without new robot demonstrations or fine-tuning · 1Carnegie Mellon simulates robot grinding and sanding in Isaac Sim; synthetic images lift real unsanded-area segmentation IoU from 77.15% to 84.47% · 1Chinese Academy of Sciences researchers let a 4B vision-language model choose among seven verified surgical skills instead of steering the needle, completing 44 of 50 closed-loop brain-electrode implantation trials on ex-vivo tissue · 1Choosing among 16 sampled human reactions after retargeting them to a Unitree G1 shows the best human motion and the best robot motion differ in 78.98% of cases, and raises retargeting feasibility by 1.61% at 9.2 Hz. · 1Coding agents that control robots and keep what they learn as reusable skills and lessons · 1Compositional Manipulation · 1Consolidating fine-tuned VLA skills with a task-conditioned hypernetwork · 1Construction navigation · 1Contact-Rich Manipulation · 1Continual reinforcement learning on top of robot foundation models · 1Continuum sensing · 1Controllable multimodal flow-matching policies · 1Controlled comparison of how to add world models to VLA robot policies · 1Cross-embodiment transfer of humanoid whole-body controllers · 1Cross-hand dexterous learning: a matched seven-hand, six-task simulation benchmark comparing action representations for one shared policy · 1Curriculum RL · 1DexBioLab, a 14-task simulated biology-lab benchmark with flames, vapour and bubbles, finds that pi0.5, GR00T-N1.7, Diffusion Policy and Fast-WAM all average under 4% success on its seven tool-use, precision and language-grounding tasks. · 1Diagnosing why VLA robots fail at long household tasks that require moving around, and fixing planning, base positioning and grasping with agentic tools around an unchanged VLA · 1Distributed Manipulation · 1Distribution Shift · 1Driving Simulation · 1ECHO-G generates full-body gestures for a talking Unitree G1 humanoid directly from speech audio and its transcript, halving the gesture-distribution gap of the best baseline (FGD 2.278 vs 4.725) at 5.96 ms per frame on a Jetson AGX Orin. · 1Efficient VIO · 1Efficient VLA inference: replacing the pi0.5 action expert with a compact Mamba model · 1Ego4WAM scored 20.29 on the RoboDojo benchmark with about 0.1K hours of robot data and 15K hours of first-person human video, level with Xiaomi-1, which used 7.2K robot hours, and found video without action labels more than doubles a world-action model's score. · 1Egocentric video · 1Embodied Agents · 1Embodied policy · 1Empowerment MPC · 1End-to-end drone navigation from stereo RGB images only, trained in simulation and flown on real hardware without fine-tuning · 1Endoscopic navigation · 1Endovascular robotics · 1Evaluating memory in VLA robot policies: interventional protocol and ChainBench · 1Evaluating robot policies without running them: fitted Q-evaluation on offline trajectories with a DINOv3 encoder and a vision-language success detector, against video world-model evaluators · 1Evaluating robot world models against real executions · 1Event-camera VO · 1Exoskeletons · 1Failure recovery · 1Faster on-device inference for flow-matching vision-language-action models · 1Federated 3D Perception · 1Fine-tuning pretrained robot policies such as NVIDIA's GR00T N1.7 on a few demonstrations without losing robustness to new camera views, backgrounds and objects · 1Fisheye detection · 1Fitting a robot's measured response to velocity commands, instead of simulating its actuators, lets a navigation policy trained in about five minutes reach the goal collision-free in 20 of 20 real Unitree Go2 runs against 4 of 20 for the default motion model · 1Floorplan SLAM · 1Force-aware VLA · 1Force-aware robot foundation model pretrained on 2,000 hours of human tactile-glove data · 1ForceWAM reaches 85% to 95% real-robot success on four force-sensitive tasks on a Flexiv arm, against 20% to 65% for the best of seven baselines, by predicting future force as well as video. · 1Foundation-model coordination of multiple humanoids · 1GPU-batched task-and-motion planning that optimises among thousands of solutions produces learnable demonstrations: 75 real Franka demos per task at 66 an hour, 67% average real policy success, 82% versus 34% for MimicGen in simulation · 1Geometry-aware manipulation · 1Georgia Tech's A-SLS controller plans with the parameter learning it is guaranteed to achieve, keeping a 61-state quadruped with unknown joint damping stable in 38 of 40 runs at a 16.1 ms median solve time · 1Georgia Tech's spectral skills cut a Unitree G1 humanoid's global tracking error 62% against the SONIC controller and let one frozen controller chain and combine motions · 1Giving a VLA policy memory by feeding it compact latent codes of what changed between consecutive camera frames · 1Goal-conditioned deformable object manipulation with 3D point flows · 1Grasping · 1HKUST and BYD train a VLA on simulated approaches plus real contact clips, cutting real data collection 34% and reaching 27.5% success at unseen object placements where full-task baselines scored 0% · 1Hanyang University and LG Electronics recover 4-bit VLA robot policies to 99.3% of full precision with 42.9% less model memory and a 1.73x speedup on Jetson Thor · 1Humanoid control, generative modeling of policy weights · 1Humanoid whole-body control · 1Humanoid whole-body control with a behavior world model · 1Humanoid whole-body training data from robot-specific video generation · 1Imitation Learning · 1JEPA World Models · 1Kyutai's 2.4B-parameter REKI, pretrained only on public data, scored 90.4% on LIBERO-Plus, 5.6 points ahead of the next public-data VLA, and predicts an action chunk in 21.8 ms on one H100. · 1LEAP solved all 10 contact-rich motion-planning benchmarks on robots from a 2-joint hopper to the 32-joint Talos humanoid, where Crocoddyl solved 7 and Aligator 8, and ran a planned hopscotch on a real Unitree Go2. · 1Lab automation · 1Laboratory Manipulation · 1Language-to-motion for humanoids: event-grounded diffusion that generates Unitree G1 motion and edits it turn by turn · 1Latent World Models · 1Learning from matched successful and failed visual futures lifts FACT to 81.2% from 64.5% in simulation and real dexterous-hand success to 71.7% from 51.0% on positions with an initial success · 1Letting a humanoid revise its cart-pushing plan mid-motion as it learns the cart's mass and wheel constraints · 1Letting a robot's vision-language-action model improve during deployment by reading its own recent attempts and rewards, without retraining any weights · 1Letting humans and coding agents build and tune physics simulations of robot tasks from a phone video · 1LiDAR Security · 1Lifelong RL · 1Long-horizon manipulation with a language-planned world-action model · 1Low-cost passive compliant leader arm for consistent imitation-learning data · 1MAPF · 1MAV guidance · 1Making touch part of the robot's action: a shared code for motion and contact lifts real-world success on peeling, flipping, cutting and wiping by 21 to 36.5 points · 1Manipulation LfD · 1Marine simulation · 1Miniature Locomotion · 1Mining Perception · 1Mobile manipulation from human video · 1Monocular SSC · 1Motion Planning · 1Motion quality of VLA-driven arms: replaying identical commands and network delays through different low-level controllers to measure jerk against tracking error · 1Multi-Agent RL · 1Multi-Robot Exploration · 1Multi-Robot Manipulation · 1Multi-UAV Exploration · 1Multi-UAV State Estimation · 1Multi-agent RL · 1Multi-agent motion planning · 1Multi-arm coordination with one VLA policy per arm · 1Multi-fidelity RL · 1Multi-robot manipulation · 1Multi-skill Manipulation · 1Multi-view perception · 1Multimodal simulation · 1NTU adds a small learned message channel to frozen VLA arms: 76.0% success on a hidden-information handover against 1.3% without messages, and no gain once the arms share a camera view · 1NUS's ReCo let a Unitree Go2 with an arm track moving end-effector paths in 75 of 100 runs on rough terrain against 42 for the best baseline, cutting position error 28.7% by training the walking policy to respond predictably. · 1NVIDIA's Robota-Embed finds the right past robot experience 47.33% of the time at clip level against 25.2% for Gemini Embedding 2, and feeding that experience to policies lifts DROID-Sim success from 47.0% to 56.3%. · 1National Tsing Hua University researchers show fine-tuned pi0.5 and GR00T N1.7 policies replay memorised tasks when a request needs a new action, and fix it with paired counterfactual training that lifts real UR5e unseen-position success from 3/40 to 35/40 · 1NavHarness guided a Unitree Go2 along eight real indoor and outdoor routes with 83.3% success against 50.0% for the best trained navigation model, by having a general multimodal model set and check its own short-range goals. · 1Navigation & Mapping · 1Object Pose Estimation · 1Object Search · 1Object-centric 3D world models for model-predictive manipulation · 1One reinforcement-learning policy that walks a 55 kg industrial quadruped over gaps, platforms, grating stairs and beams straight from raw LiDAR points, with no terrain map · 1One reinforcement-learning policy that walks a Unitree Go2 with a 6-DoF arm and places the gripper to within 3.22 cm under simulated lunar gravity, with safety limits enforced during training · 1One world model retrained from NVIDIA Cosmos 3 Edge that both predicts scenes and controls 20 robot families · 1Open-weights VLA family for dual-arm robots: embodiment midtraining before task finetuning, plus online correction from a few human episodes · 1Optimus-R adds a 7M-parameter skill memory to pi0.5 and reaches 88.6% on LIBERO with 30% of the data, versus 72.1% for full fine-tuning, while training in 86 instead of 560 GPU hours. · 1Panoramic 3D Detection · 1Panoramic segmentation · 1Pedestrian tracking · 1Pengcheng Laboratory releases the first actions of a VLA chunk after one denoising step and refines the rest during motion, cutting action wait 2.26x on a Dobot Nova2 at an unchanged 84% success · 1PersonNav · 1Personalising a pretrained vision-language-action model to one home or office from a 20 to 30 minute phone scan, using a Gaussian-splat digital twin to generate pickup demonstrations automatically · 1Planning · 1Playing robot demonstrations backwards and at different speeds let a policy trained on 50 demos reach 85.2% success on 16 RoboTwin tasks, level with a baseline trained on 100. · 1Policy Optimization · 1Post-training VLA policies on messy real data: a reachability graph scores which action chunks move a precise assembly task forward · 1Predicting contact failures before they happen with a vision, action and force world model · 1Pretraining a delivery robot's navigation model on 10,000 hours of first-person human walking video by predicting scene structure instead of pixels, then adapting it with 17 hours of robot driving · 1ProWorld, a robot world model trained on 6,486 hours of data, predicts real-robot policy success with a 0.95 Pearson correlation, against 0.33 and 0.67 for Ctrl-World and Cosmos 3 Nano · 1Proprioception-only sim-to-real precision assembly · 1Quadruped loco-manipulation: learned whole-body control · 1Quadruped locomotion · 1RawVLA, a 168 FPS neural camera pipeline placed in front of frozen VLA policies, keeps a real dual-arm pi0.5 robot at 66.5% success in low light where the default camera processing drops it to 0%. · 1Real-time VLA · 1Real-to-sim · 1Real-to-sim-to-real data generation for manipulation · 1Real-world reinforcement learning that splits a dexterous robot hand's finger skill from the arm's coordination policy, so tool-use tasks such as working a power drill or a pipette can be learned in about an hour · 1Reinforcement learning for industrial assembly · 1Representation alignment · 1Resampling demonstrations by their shared relative tempo lifts real bimanual success to 77.8% from 58.3% while cutting task time 2.02x, with no runtime speed controller · 1Researchers at JD's Joy Future Academy pre-train a VLA policy on latent actions learned from egocentric human video, raising average real-robot success on an AgiBot G1 from 63.3% to 85.0% as annotated video grows from 1,000 to 5,000 hours · 1Rewording one instruction cuts OpenVLA's LIBERO success by 38 points; test-time voting over paraphrases does not fix it, an hour of LoRA training on paraphrases does. · 1Robot VLA models store distance to an object on two nearly perpendicular internal rulers, coarse meters and fine centimeters, and the pattern recurs in pi0.5 and MolmoAct2 on real robot data · 1Robot policies trained only on simulator-checked world-model videos reach 55.0% real-world success, 86% of what 50 human demonstrations deliver, and triple it in unfamiliar scenes · 1Robot security: visual attacks that redirect vision-language-action policies · 1Robot spatial memory: monocular 3D scene graphs for mobile manipulation · 1Robot-Use Agents · 1Runtime Verification · 1Runtime recovery for a frozen VLA policy: a latent world model forecasts whether to keep going, move the camera or replan the arm · 1Runtime safety for VLA policies: a training-free barrier filter that guards the gripper, wrist and forearm against a moving hazard on edge hardware · 1Rutgers' PL-MPC lifts a real KUKA arm's zero-shot wrench-on-nut success to 74.2% from 61.3% and quadruples return on HumanoidBench balance-hard (98 to 387) by fixing three links in the planner-learner loop. · 1Safe Generative Planning · 1Safe RL-MPC · 1SafeDriveVLA cut collisions under unsafe spoken instructions from 60 to 38 and raised driving score to 85.9 from 72.8 against SimLingo in CARLA, by imagining what would happen if it obeyed before acting. · 1Safety monitoring · 1Safety-critical control · 1Scissor-shaped handheld teaching tool that lets people feel grip forces while recording robot demonstrations, used to fine-tune pi0.5 on dual-arm Franka and Flexiv robots · 1Selective repair of cached action chunks for long-horizon manipulation, using a joint future-action belief and a per-action impact model. · 1Shrinking a 3.3-billion-parameter robot foundation model into a 451-million-parameter student that runs in two inference steps · 1Simulation geometry · 1Skill Discovery · 1Skill2Real lifts a real UR5e arm's mean task completion from 27.50% to 78.75% across four tasks using skills learned only in simulation by foundation-model agents, with no real-world training. · 1Social Navigation · 1Social navigation · 1Social-WM cuts human collisions in simulated crowd navigation to 21.67% from 39.09% for the NavThinker world-model baseline (44.6% fewer) while raising success to 63.77%, by learning which commanded moves a robot can actually carry out near people. · 1Soft Pneumatic Actuators · 1Soft manipulation · 1Sonar SLAM · 1Speeding up world-model-based visual navigation by screening candidate trajectories with a distilled long-stride latent model before refining with an autoregressive teacher. · 1Splitting each two-handed robot action into separate left and right tokens that share a timestamp, so a pi0.5-based policy coordinates two dexterous arm-hand systems better · 1Starting a robot policy's flow-matching generation from its own recent actions and frames instead of random noise, so vision-language-action and world-action models train in fewer epochs and act in one sampling step · 1Steering frozen diffusion and flow-matching robot policies toward good contact by predicting and scoring the tactile consequences of candidate actions during generation. · 1Steering pretrained diffusion and VLA robot policies beyond their initial behaviour using only relative preferences and classifier-free guidance. · 1StreamRig tracks a walking humanoid's motion from four cameras after simulation-only training, cutting rotation drift 63% against the best rival at 26.2 ms per five-camera frame · 1Streaming 3D reconstruction · 1Submovement decomposition · 1Surgical robotics · 1Switching off one attack-linked internal feature only when a probe spots an adversarial patch lifts pi0.5 task success under attack from 26.0% to 32.4% without retraining. · 1Synthetic data generation for robot policies · 1T2-z lets a frozen robot world model learn hidden friction, mass and rules from one or two trial pushes, lifting real-robot action success from 26.5% to 70.8%. · 1TDV turns one human video into object trajectories that train a robot policy, reaching 91.8% average success on 12 RLBench tasks and 71 of 78 real UR5 trials after augmenting the single demo to 50 trajectories. · 1Tactile Exploration · 1Tactile prediction from human egocentric video, transferred to robot hands · 1Teaching a Unitree G1 humanoid martial-arts jumps and spins from video of national-level athletes, with fall recovery learned inside the same policy · 1Teaching a real humanoid to return tennis balls using public walking data and ordinary tennis videos instead of new motion capture · 1Teaching a soft-skinned humanoid face to move its eyes, neck and eyelids like a person: 27.9% lower closed-loop error than the best learned rival and about 80 times lower than scripted eye-head rules · 1Temporal manipulation · 1Terrain traversability · 1Test-time steering in learned nonlinear coordinates of expert hidden states lifts frozen imitation-learning policies under observation noise by 3.7 to 12.0 points on four simulated manipulation tasks · 1Test-time verification for vision-language-action policies · 1Testing whether failure alarms for robot AI models actually save time once false alarms and reset costs are counted · 1Text-to-3D Policy · 1Text-to-motion for humanoids: a body-part-aware latent diffusion model that writes Unitree G1 joint references a frozen tracker can execute · 1Tool Co-Design · 1Toyota's North American R&D lab cuts a driving world model's 10-second position error by 44% (84.2 m to 47.2 m) by penalising sideways motion a car cannot make, without changing the model. · 1Training a VLA for a drone with a robot arm entirely on synthetic trajectories, then chaining local skills across sites with a scene graph to complete multi-step missions · 1Training a vision-language-action model to predict the physical effect of its next actions instead of the raw motor commands, then decoding actions from that effect · 1Training care robots that help people stand up and transfer, using simulated humans who move back · 1Training household mobile manipulators with targeted human corrections instead of more demonstrations · 1Trajectory Correction · 1Transferring task structure from human videos to robot world-action models · 1Traversability · 1Treating slow-module invocation in dual-system VLA models as a calibrated cost-benefit decision explains when fixed cadences are optimal, and closed-loop LIBERO tests show language subgoal rewrites cutting OpenVLA-7B success from 48 to 6 of 100 episodes · 1Turning a frontier multimodal agent's robot executions into reusable text-plus-code skills, with tactile feedback, to raise first-attempt success and cut runtime. · 1Turning a safety shield's would-be corrections into training data: a pi0.5 policy gains 8.5 points of success and causes 35.6% less contact cost, with no shield at deployment · 1Turning first-person human demonstrations into training data a humanoid can use directly, so a vision-language-action model trained without any robot demonstrations can carry objects about 10 m on a Unitree G1 · 1Turning ordinary human videos into skill-labelled training data for a dexterous robot hand · 1Turning phone-recorded human demonstrations into per-robot post-training data by retargeting object motion · 1Turning recorded human hand-object demonstrations into physically valid robot-hand trajectories with one shared reinforcement-learning policy instead of one optimisation per clip · 1Turning third-person human videos of table-tennis serves, pickups and ladder climbing into Unitree G1 skills by preserving where and how fast contacts happen · 1Two Unitree G1 humanoids playing table-tennis doubles with learned collision-risk filtering · 1UAV Navigation · 1UAV Swarms · 1UAV traffic monitoring · 1UBTech's Thinker team trains one world model that generates video, metric depth and robot actions together, ranking first on EWMBench, WorldOlympiad and WorldArena and reaching 92.6% average success on the 50 RoboTwin 2.0 tasks · 1USC's WACL lets a robot arm press, push and lift with its forearm while the gripper holds a cup, succeeding in 78% of constrained trials against 33% for a strong baseline, from 10 bare-hand human videos per task. · 1Underwater Robotics · 1Underwater active mapping · 1Underwater perception · 1Unified world-action model · 1Using a learned multi-view world model as the training environment for reinforcement learning on a pi0.5 policy, so the robot improves with little extra real-world interaction · 1Using a video generator to turn four real clips of a person carrying a box into 256 synthetic interaction videos, then into simulation training data for a humanoid that picks up, carries and drops unseen objects · 1VISTA, a touch-and-vision robot policy, reaches 80.9% success on eight contact-rich simulated tasks from 50 demonstrations each and 22 of 30 real trials against 4 of 30 for the strongest baseline · 1VLA Conditioning · 1VLA execution speed · 1VLA inference efficiency · 1VLA models trained on whole task sequences hit 40.0% on seen combinations but 7.7% on new ones; chaining atomic skills with start-pose-robust training lifted real-robot progress on unseen combinations to about 0.9 of the task. · 1VLA robustness · 1VLA robustness benchmark · 1VLM preference feedback for robot reinforcement learning · 1Video world models · 1Vision-Language Navigation · 1Vision-based competitive task allocation with a five-frame proposal window and an improved artificial potential field for collision-free dual-arm picking. · 1Vision-language-action models without wrist cameras (learned foveation) · 1Visual Generalization · 1Visual Grounding · 1Visual Reward Models · 1Visual SLAM for mobile robots · 1Visuomotor Fusion · 1Waseda's PACT Hand, designed together with its teleoperation retargeter, copies human finger postures to 3.9 mm against 12.8 mm for the LEAP Hand and grasps 86% of everyday objects against 64%. · 1Weighting rare actions by inverse kernel density lets a behaviour-cloned driving policy trained on eight CARLA routes score 98 instead of 34 in an unseen town, with zero red-light infractions · 1Whether labelling messy human demonstrations by quality improves robot foundation models · 1Whole-body mobile manipulation and teleoperation · 1Why humanoid reinforcement-learning pipelines report success that is not there: failures hidden in curriculum gates, evaluation scripts and reference motions as well as in the reward, with measured examples from one simulated humanoid · 1World Models · 1World models · 1World-action model that retrains on its own successful, mediocre and failed real-robot rollouts and improves round after round · 1World-model view imagination for occlusion-robust manipulation · 1compliant humanoid whole-body control · 1construction robotics · 1data engine converting egocentric human video into replayable bimanual dexterous robot demonstrations · 1dexterous in-hand manipulation with learned world models · 1efficient inference for vision-language-action robot policies · 1fleet memory for warehouse robots · 1humanoid dexterous manipulation · 1humanoid learning from human video · 1humanoid teleoperation · 1humanoid teleoperation with heavy unknown payloads · 1humanoid whole-body control · 1humanoid whole-body manipulation · 1humanoid-human interaction · 1large-scale tactile demonstration dataset and robot-free data collection · 1mobile robot navigation · 1pi0.5 and VLANeXt both succeed about 69% of the time under a camera shift, but pi0.5's successful runs become 22.5% longer and 24.1% jerkier while VLANeXt's barely change · 1post-training vision-language-action policies for industrial disassembly · 1real-world reinforcement learning for robot policies · 1robot foundation model training · 1robot learning from human video · 1robustness benchmark for vision-language-action models · 1tactile manipulation · 1tactile pretraining for vision-language-action models · 1teleoperation and data collection for legged manipulation · 1terrain-aware behavioral foundation model for humanoid whole-body control · 1training-signal allocation for vision-language-action robot policies · 1visuo-tactile world models · 1VLA · 1world models for dexterous manipulation · 1
LevelAnyBeginnerIntermediateExpert
arXiv
visuo-tactile world modelsExpert

ACTS: Learning Visuo-Tactile World Models from Human Physical Interaction

Yuxiang Ma, Weihan Xu, Pengfei Ye, Anna Mei, Yuxin Ray Song, Tingle Li, Haonan Chen, Edward H. AdelsonMassachusetts Institute of Technology

A world model trained only on 4.9 hours of people pushing and pulling objects with handheld GelSight touch sensors predicts the next half-second of a scene 0.7 to 2.0 dB PSNR better than two visuo-tactile models trained from scratch on the same data, researchers from MIT, Harvard, UW, Berkeley and Stanford report in a preprint. Run on a robot without fine-tuning, its predicted touch imprints still appear and vanish with real contact, while its camera predictions transfer unevenly.

World ModelsTeleoperation
NVIDIA
Read the full paper (PDF)

Topic pages

Manipulation · 83Navigation · 76General robotics · 67Simulation · 35Robot learning · 34VLA · 34Perception · 32Safety · 21Locomotion · 20Humanoid · 13Aerial · 4World Action Models · 4World-action model · 3humanoids · 3world models · 4

Papers come from arXiv robotics feeds. Where our summary is not written yet, you see the opening lines of the abstract.