Detect and Suppress: A Mechanistic Defense against Adversarial Patches in VLA Models
Preprint. The authors build an inference-time defense for vision-language-action (VLA) policies against adversarial image patches by training a sparse autoencoder, identifying an internal feature that tracks patch presence, and suppressing that feature only when a linear probe detects an attack. On LIBERO-10 under intermittent UADA attacks, conditional suppression raises success rate, e.g. from 26.0% to 32.4% for π0.5 at n=10, while avoiding the large benign-condition drop caused by continuous suppression. This points to lightweight, fine-tuning-free robustness layers for deployed robot policies.
Can VLA robot policies be made robust to adversarial visual patches by directly intervening on attack-related internal features at inference time, without fine-tuning?
Adversarial patches applied to camera observations can cause VLA robot controllers to fail. Existing defenses either preprocess inputs or fine-tune model parameters, but fine-tuning adds substantial training cost, and static defenses may not handle intermittent attacks or may degrade normal behavior when applied continuously.
Previous VLA defenses modify inputs via image preprocessing or update model parameters via adversarial fine-tuning. These approaches require additional policy training or may not target the internal representations that drive failures, and they do not provide a conditional, inference-only intervention.
The authors train a TopK sparse autoencoder on clean VLA internal representations, then identify an SAE feature whose activation correlates with patch presence by toggling an adversarial patch during rollouts. A logistic regression probe is trained on earlier-layer token-mean-pooled representations to detect attacks with low false positives. At inference, when the probe detects an attack, the selected feature's contribution is subtracted from prefix token representations at the target layer, using an intervention strength λ; VLA parameters remain frozen.
Detection: probe recall 81.76% (n=3) and 84.91% (n=10), precision 92.71% and 96.45%, false positive rate 0.39% and 0.71%, below the 2% target. Main evaluation on LIBERO-10 with 500 rollouts per setting: π0.5 baseline vs conditional defense success rate: n=3 46.8% vs 49.2% (+2.4 points), n=10 26.0% vs 32.4% (+6.4 points); SmolVLA n=3 18.6% vs 19.6% (+1.0), n=10 6.2% vs 7.6% (+1.4). Continuous suppression without detection is much worse for π0.5 (9.4% and 16.2%) and slightly worse for SmolVLA. Under benign conditions, conditional defense keeps π0.5 at 54.6% vs baseline 54.4%, but SmolVLA drops from 41.4% to 36.6%; continuous suppression drops to 35.2% and 19.0%. Intervention strength sweep shows attack success peaks around λ=0.2–0.3, and benign performance falls with stronger continuous suppression.
Authors state gains on SmolVLA are modest and benign performance still degrades for that model; future work should evaluate unseen attacks and physical robots. Evident limitations: evaluation is simulation-only on LIBERO-10, uses a single UADA universal patch per victim model, only two VLA models, probe threshold/feature selected from related data, and the absolute success rates under attack remain low. Code release is not reported in the paper.
VLA model providers and companies deploying autonomous manipulation robots could use this as an inference-time safety layer if they need defense against visual adversarial patches without expensive retraining. Near-term relevance is limited because validation is simulation-only and gains are small on SmolVLA; within 1–3 years, if real-robot validation, detection robustness across unseen attacks, and stronger multi-model gains are achieved, it could be integrated into robot foundation model stacks.
The full text is not republished here because the paper's license does not allow it. Read the original on arXiv.