TACTIC：大模型动态规划路边LiDAR攻击，CARLA仿真碰撞率达100%

Abstract

Physical LiDAR attacks are often evaluated using fixed primitives and manually selected parameters, despite their strong dependence on surrounding traffic. We present TACTIC, a scene-aware framework that uses a multimodal large language model (MLLM) to coordinate state-adaptive roadside LiDAR attacks. Under a gray-box threat model, TACTIC relies only on an attacker-operated roadside perception stack, without accessing the victim LiDAR’s native point clouds or internal processing. Local perception provides metric vehicle states, while the MLLM combines these measurements with roadside imagery to infer relational traffic context and construct a semantic scene graph. Based on this representation, TACTIC selects and configures two complementary primitives: push-away, which shifts the perceived range of a lead vehicle, and phantom-obstacle braking, which triggers emergency braking through obstacle injection. Measured traffic states and empirically calibrated constraints ground the generated tactics in physically feasible operating regions. To accommodate MLLM latency, TACTIC overlaps reasoning and execution asynchronously while high-rate local perception detects scene changes and triggers replanning. Across 280 randomized CARLA trials, the full policy achieves a 100% collision rate, compared with 35% for a fixed rule, 60% for random selection, and 75% for a restricted LLM using mode selection with default parameters. Joint physical-and-image input achieves 100% success, versus 65% with physical measurements alone and 75% with imagery alone, while asynchronous \Delta refresh reduces scene-mutation response from 7.4 s to 2.0 s. These results show that scene-dependent tactical planning can expose context-sensitive LiDAR failure modes that fixed attack policies may miss.

I Introduction

LiDAR is widely used in autonomous driving but remains vulnerable to physical attacks that suppress genuine returns, inject phantom objects, and induce unsafe vehicle behavior [1, 2, 3]. Studying these attacks in controlled environments is essential for identifying safety-critical failure modes, evaluating countermeasures, and improving protection for passengers and other road users.

Existing physical LiDAR attacks mainly demonstrate individual attack primitives. A target may be removed through spoofing [4], while a nonexistent obstacle may be introduced through point injection [2]. However, these attacks are typically evaluated using manually defined rules and fixed parameters in simplified traffic settings, such as straight-road driving [5], predefined turning scenarios [6], or pre-assumed vehicle maneuvers [7]. Such evaluations establish whether an individual primitive can work, but provide limited insight into how attack effectiveness changes with surrounding traffic scenes.

We focus on vehicle-to-vehicle interactions because they directly expose these dynamic dependencies and are central to autonomous-driving safety. Unlike collisions with walls or road dividers, which are dominated by static geometry and may additionally depend on map availability, localization, or lateral control, interactions among vehicles depend on continuously evolving traffic relationships. A nearby vehicle may be the target’s leader or follower in the same lane, or an unrelated participant in an adjacent lane; the same perturbation can therefore produce very different outcomes. Physical attack execution is consequently a scene-dependent planning problem that requires reasoning over traffic topology, spacing, relative motion, acceleration, and time-to-collision (TTC), rather than applying a fixed primitive or parameter set.

We address this gap with TACTIC, a temporal and context-aware framework for state-adaptive, LLM-orchestrated roadside LiDAR attacks, illustrated in Fig. 1. Under a gray-box threat model, the roadside attacker independently observes traffic without accessing the victim LiDAR’s native point clouds, filtering parameters, or internal processing. TACTIC combines object-level local perception with roadside imagery and uses a multimodal large language model (MLLM) for relational scene reasoning and context-dependent tactical planning.

Specifically, local perception detects and tracks individual vehicles and provides measurable physical states such as position, velocity, spacing, and TTC, while the MLLM jointly considers these measurements and visual road context to infer same-lane, adjacent-lane, and leader–follower relationships. These relations form a semantic scene graph for tactical decision making. Based on this graph, TACTIC selects and configures two complementary collision primitives: a push-away attack, which shifts the preceding vehicle’s perceived range, and a phantom-obstacle braking attack, which triggers emergency braking through obstacle injection. Because the primitives exploit different vehicle relationships and feasibility regions, their effectiveness depends on the current scene.

Introducing an LLM into the loop creates two additional challenges: generated tactics may violate physical feasibility, and LLM inference may lag behind rapidly changing traffic. TACTIC addresses the former using measured vehicle states, calibrated operating bounds, and deterministic verification, leaving the MLLM to focus on relational scene understanding and tactical selection. To address latency, reasoning and execution overlap asynchronously, while high-rate local perception detects scene changes and triggers replanning only when needed.

We evaluate TACTIC in CARLA using randomized multi-vehicle scenarios, a realistic driver model, and a two-stage automatic emergency braking controller. Across 280 primary CARLA trials spanning feasibility, policy, perception, timing, and benign-control evaluations, the full LLM policy achieves a 100% collision rate, compared with 35% for a fixed rule and 60% for random selection. A restricted LLM that selects the attack mode but uses default parameters reaches 75%, showing that joint mode and parameter adaptation provides additional benefit. Joint physical and visual input sustains 100% success, whereas physical-only and image-only inputs drop to 65% and 75%, respectively. Physical calibration reveals a sharp push-away feasibility transition near 10.5--11 m, while asynchronous replanning reduces scene-change response latency from 7.4 s to 2.0 s. The project website1 is accessible, and the source code2 is released.

The main contributions are as follows:

• We formulate physical LiDAR attack execution as a scene-dependent tactical planning problem, focusing on dynamic vehicle-to-vehicle interactions whose feasibility depends on evolving relational traffic context.
• We develop TACTIC, which combines object-level local perception and roadside imagery with multimodal LLM reasoning to construct semantic scene graphs and select complementary physical attack primitives and parameters.
• We ground LLM tactical decisions with measured vehicle states and calibrated physical constraints and introduce an asynchronous perception–LLM architecture for high-rate scene-change detection and replanning. Experiments demonstrate higher collision success than fixed and random policies and faster adaptation to changing scenes.

II Related Work

II-A Physical LiDAR Attacks on Autonomous Vehicles

Physical LiDAR attacks largely follow two directions. Removal attacks suppress genuine target returns through adversarial objects or physical spoofing [1, 8, 4, 9], whereas injection attacks introduce nonexistent obstacles to manipulate downstream perception and control [2, 10]. Recent work improves operational realism by attacking moving targets at longer range [5] and extending physical manipulation to localization, odometry, and sensor fusion [6, 7]. A recent systematization further characterizes how sensor perturbations propagate through autonomous-driving pipelines [3].

Despite these advances, attack execution remains largely scenario-specific: individual primitives are commonly evaluated with predefined conditions and parameters tailored to particular road configurations or maneuvers [5, 6, 7]. How to select and configure physical attacks as multi-vehicle traffic evolves has received substantially less attention. TACTIC addresses this gap by formulating physical attack execution as a scene-dependent tactical planning problem.

II-B Context-Aware Attack Planning

Context-dependent coordination of physical sensor attacks remains comparatively unexplored, with existing approaches typically executing individual attacks independently or using handcrafted rules that map predefined conditions to fixed actions. Attack-hardness analysis [11] shows that physical attack feasibility depends strongly on situation-specific conditions, but identifies these conditions primarily through offline search. LLM-based adversarial scenario generation [12] instead varies simulated driving scenarios rather than coordinating online physical attacks. Zhang et al. [3] further identify joint perception–decision attacks and scene-aware hybrid attack chains as open directions.

TACTIC moves this reasoning online: measured traffic states and calibrated physical constraints define the feasible action space, while relational scene understanding guides context-dependent selection and parameterization of complementary physical attack primitives.

II-C LLM-Based Scene and Tactical Reasoning

LLMs and multimodal LLMs (MLLMs) are increasingly used for high-level reasoning in autonomous driving. GPT-Driver reformulates motion planning as language modeling over structured driving states [13]; LMDrive integrates multimodal sensor observations and language for closed-loop driving [14]; and DriveLM introduces graph-structured visual reasoning across perception, prediction, and planning [15]. These studies demonstrate the potential of language models to reason over heterogeneous observations and relationships among traffic participants.

TACTIC extends this capability to physical-attack planning: local perception provides metric states, the MLLM infers relational context and selects tactics, while physical feasibility and execution remain deterministic. To reduce latency, TACTIC overlaps reasoning with execution following TypeFly’s generate-while-execute principle [16].

III Scene-Dependent Attack Formulation

We focus on vehicle-to-vehicle interactions and stress tests because they require reasoning over dynamic traffic relationships, whereas collisions with walls or road dividers are dominated by static geometry and may additionally depend on map availability, localization, and lateral control. These factors can confound the study of physical LiDAR attacks and obscure the role of surrounding traffic. Our work provides a direct setting for studying how dynamic relationships, including leader–follower structure, spacing, and relative motion, affect attack feasibility.

This objective makes physical attack execution inherently scene dependent. As shown in Fig. 1, a nearby vehicle may be the target’s leader or follower in the same lane, or an unrelated participant in an adjacent lane. The same physical primitive can therefore succeed or fail depending on the current traffic topology and vehicle dynamics. We formulate this scene-dependent attack space below; Sec. IV then introduces TACTIC to reason over the scene and select among feasible attack policies.

III-A Threat Model and Attack Objective

We consider a roadside attacker equipped with LiDAR and an RGB camera, following a deployment setting similar to the Moving Vehicle Spoofing System in [5]. The roadside perception stack detects and tracks surrounding vehicles at 20 Hz. Vehicles follow an intelligent driver model [17, 18] together with a two-stage AEB controller that applies partial and full braking at perceived TTC thresholds of 3.0 s and 1.6 s, respectively [19].

We adopt a gray-box threat model in which the attacker interacts with the victim only through the physical sensing channel. The attacker independently observes the traffic scene and may know relevant sensor and vehicle-control configurations, but has no access to the victim’s native point clouds, internal filtering states, or vehicle messages. A target vehicle T is designated, and the objective is to induce a collision between T and a neighboring traffic participant.

Let \mathcal{S}_{t} denote the roadside observation at time t, comprising vehicle positions, velocities, accelerations, inter-vehicle spacing, TTC, and the corresponding roadside image. Because these object-level measurements do not by themselves encode how traffic participants relate to one another, Sec. IV further interprets \mathcal{S}_{t} into a relational traffic context for tactical decision making. At each decision round, the policy outputs

\mathcal{P}_{t}=(m,\theta_{m}),

where m\in\{\texttt{push-away},\texttt{phantom-braking},\texttt{hold}\} denotes the selected physical attack primitive and \theta_{m} contains its mode-specific continuous parameters. Specifically, \theta_{m}=(\Delta d,\rho,o) for push-away and \theta_{m}=(w,\tau) for phantom-braking, as defined in Sec. III-B. A round corresponds to one decision–execution–replanning cycle of the policy in Sec. IV, and each trial is limited to at most R=3 such rounds.

III-B Complementary Physical Attack Primitives

Pointcloud push-away

Consider a target T following a lead vehicle A1, with another vehicle A2 behind T in the same lane. The attacker shifts the perceived range of A1, causing T to perceive a larger leading gap while the true T–A1 distance contracts. This primitive, denoted push-away, is parameterized by the induced range displacement \Delta d, separation ramp \rho, and ignition onset o.

Phantom-obstacle braking

The second primitive injects a virtual obstacle at distance w ahead of T. When its perceived TTC falls below the full-braking threshold, T performs emergency braking and the trailing vehicle A2 may become the collision agent. This primitive, denoted phantom-braking, is parameterized by phantom wall placement distance w and emission duration \tau.

Hold

The third primitive hold means keeping the previous primitive \mathcal{P}_{t-1}. This primitive avoids frequent policy changes during attacking.

The three primitives are complementary because they exploit different traffic relationships. push-away depends primarily on the T–A1 leader–follower geometry, whereas braking exploits the A2–T relationship and the target’s deceleration. This complementarity motivates scene-aware tactical selection rather than a fixed attack rule.

III-C Physical and Temporal Feasibility

The available action space is further bounded by physics-informed, empirically calibrated constraints. These constraints determine whether a candidate primitive can operate under the current scene, leaving the scene-aware policy in Sec. IV to choose among feasible alternatives.

Push-away feasibility

The induced displacement must be large enough to produce the required closing behavior while remaining inside roadside sensing coverage: \Delta d_{\min}=11~\mathrm{m},\,\Delta d_{\max}(o)=50-o~\mathrm{m}. The displacement must also develop before the target reaches the lead vehicle while remaining below the measured consistency threshold: \frac{\Delta d}{t_{\mathrm{arr}}}\leq\rho\leq 3.0~\mathrm{m/s}, where t_{\mathrm{arr}} denotes the remaining arrival time.

Braking feasibility

For the braking primitive, the injected obstacle must drive perceived TTC below the 1.6 s full-braking threshold. We therefore place the virtual wall according to w(v)=1.5v,\,w\in[5,15]~\mathrm{m}, where v is the measured target speed. The emission duration is calibrated to cover both the target’s braking response and the follower’s subsequent approach.

Temporal feasibility

Feasibility depends jointly on traffic geometry and execution time. Waiting can enlarge or shrink relevant inter-vehicle gaps, consume the remaining push-away displacement envelope, or change whether a trailing vehicle can reach the target after braking. Attack timing is therefore determined from the continuously measured spacing, TTC, and vehicle motion rather than by a fixed schedule. Sec. IV describes how TACTIC combines these feasibility conditions with relational scene understanding for state-adaptive tactical planning.

IV TACTIC: Scene-Aware LLM Tactical Planning

TACTIC operationalizes the scene-dependent attack space defined in Sec. III. Its design separates three responsibilities: local perception measures the traffic state, a multimodal LLM performs relational scene understanding and tactical selection, and deterministic modules enforce the physical constraints in Sec. III-C and execute the selected primitive. Because perception operates much faster than LLM inference, reasoning and execution proceed asynchronously, with a high-rate local monitor triggering re-planning when the cached scene becomes invalid.

IV-A Multimodal Semantic Scene Reasoning

The perception front end operates at the LiDAR’s native 20 Hz rate, clustering detections and associating them into persistent tracks. It provides object-level measurements including longitudinal and lateral position, velocity, acceleration, spacing, and TTC. These measurements accurately describe individual vehicle states but do not explicitly encode their semantic relationships.

To recover this relational context, each MLLM query combines the structured vehicle states with the corresponding roadside camera frame. The image provides complementary road semantics, such as lane structure and driving direction, while the structured measurements retain precise metric information. From these inputs, the MLLM constructs a semantic scene graph

G_{t}=(V_{t},E_{t}),

where nodes V_{t} represent tracked traffic participants and edges E_{t} encode relations such as same-lane leader–follower and adjacent- or opposite-lane interactions. Edge attributes retain metric quantities supplied by the perception stack, including spacing, relative motion, and TTC. This division keeps physical-state estimation outside the MLLM while assigning it the higher-level task of interpreting how traffic participants relate to one another [20, 21].

MLLM scene-graph generator

Figure 2 illustrates the multimodal generation process. The roadside image I_{t} is patchified and encoded by a SigLIP-2 vision transformer, followed by a lightweight merger that compresses each 2\times 2 block of patch embeddings into a visual token sequence,

Z=\mathrm{Merger}\!\left(\mathrm{ViT}(I_{t})\right)=\{z_{1},\ldots,z_{M}\}.

The tracked vehicle states, output schema, and cached topology are serialized into text tokens X=\{x_{1},\ldots,x_{n}\}. Visual and text tokens are jointly processed by the multimodal decoder using interleaved MRoPE positions, while mid-level visual features are injected into early decoder layers through DeepStack. This allows relational fields to condition jointly on visual road context and precise object-state measurements.

The resulting graph is autoregressively decoded into a constrained JSON representation. Letting y_{1},\ldots,y_{K} denote its serialized output tokens,

p(G_{t})=\prod_{k=1}^{K}p\!\left(y_{k}\mid y_{<k},Z,X\right).

Semantic fields, such as lane association and leader–follower relations, are inferred from the multimodal context, whereas metric fields retain measurements supplied by the local perception stack. This separation prevents the MLLM from replacing low-level state estimation while allowing it to resolve traffic relationships needed for tactical planning.

Scene-graph generation runs asynchronously following the generate-while-execute principle of TypeFly [16]. The initial query performs full multimodal scene interpretation. During steady state, subsequent queries reuse the cached graph and recent object states. If the vehicle set and relational topology remain unchanged, the \Delta path refreshes only dynamic edge attributes such as spacing and TTC; otherwise, a vehicle-set consistency check triggers full multimodal regeneration. The updated graph atomically replaces the cached topology once generation completes.

IV-B Physics-Grounded Tactical Policy

Given the semantic graph G_{t}, TACTIC determines which physically feasible primitive best matches the current traffic configuration and how its continuous parameters should be configured. The key design principle is to separate consequence, feasibility, and tactical selection rather than asking the LLM to infer all three from scratch.

H-group measure

Before each tactical decision, both primitives are scored using an offline-calibrated h-group measure. For the rear-end primitive, normalized gap, acceleration, and TTC terms use weights 0.15/0.70/0.15, with attack duration introduced at a second combination level with weight 0.10. For braking, acceleration and TTC use weights 0.85/0.15, again with a 0.10 duration term. Gap is omitted from the braking score because braking severity is driven by the target’s deceleration rather than a monotonic function of inter-vehicle distance. Grid-search calibration yields mean attack-to-benign separation margins of 96.5\times and 13.6\times for the rear-end and braking measures, respectively. The score is defined from the victim’s perspective: larger values indicate more severe consequences and do not encode attacker cost or feasibility.

Feasibility grounding

A high h-group score does not imply that the corresponding primitive is physically executable. TACTIC therefore supplies the LLM with the measured traffic state together with the mode-specific feasibility bounds derived in Sec. III-C. For push-away, these include the minimum and coverage-limited displacement,11\leq\Delta d\leq 50-o, and the admissible ramp band, \Delta d/t_{\mathrm{arr}}\leq\rho\leq 3.0~\mathrm{m/s}.

For braking, the wall placement follows the measured target speed, w(v)=1.5v within [5,15] m, while the minimum emission duration depends on the current follower gap. These values are recomputed from the current scene and provided numerically at every tactical decision.

The LLM therefore reasons over three complementary inputs: relational scene structure, potential consequence, and remaining feasibility margin. A lower-h-group primitive may be selected when the traffic geometry makes it substantially more feasible. Before execution, deterministic code projects the generated parameters back into the admissible set \Phi(\mathcal{S}_{t}), ensuring that physical bounds are system constraints rather than prompt suggestions. In short, measured states and calibrated constraints determine what can be executed, while the LLM determines which tactic best matches the scene.

Tactical policy

At round t, the LLM produces \mathcal{P}_{t}=(m,\theta_{m}), where m and \theta_{m} are defined in Sec. III-A. The full policy jointly selects the primitive and its parameters and additionally returns an estimated success likelihood and a short rationale used for logging and analysis. Execution may continue for at most R=3 rounds, allowing the tactic to be revised when the previous round fails or the traffic relationship changes.

Attack timing is handled separately from tactical selection. The high-rate controller continuously evaluates spacing, TTC, and vehicle motion against the feasible spatio-temporal region defined in Sec. III-C. It determines whether the cached policy remains executable, should be launched, or must be suspended and reconsidered. The LLM therefore selects and configures the tactic, but does not replace high-rate temporal monitoring.

Decision sources

To isolate the contribution of LLM tactical reasoning, we compare four decision sources in Sec. V. llm_policy jointly selects the primitive and its mode-specific continuous parameters, returning the primitive choice m together with its continuous parameters ((\Delta d,\rho,o) for push-away, (w,\tau) for the phantom wall), with a success estimate and rationale; llm selects only the primitive in a single shot without the policy layer, with the continuous parameters fixed at system defaults; rule always selects push-away and draws its parameters at random once per trial, exempt from the analytic floors, and reads nothing; and random commits to one uniformly sampled tactic and parameterization for the trial. The random group is not re-sampled after each round, avoiding multiple independent chances that would confound tactical quality.

IV-C Asynchronous Scene Monitoring and Replanning

LLM inference takes seconds, whereas the traffic state evolves at the perception rate. TACTIC therefore decouples high-rate scene monitoring from lower-rate semantic and tactical reasoning. While a cached policy is valid, execution continues concurrently with generation of the next scene interpretation. A lightweight local alerter compares the 20 Hz tracker state against the cached graph on every frame.

A scene mutation is declared when a vehicle appears or disappears, an inter-vehicle gap changes by more than 8 m or 40%, or the inferred road context changes. Because ordinary car-following gaps evolve gradually, these thresholds target structural scene changes rather than frame-level noise. A minimum 0.5 s replanning interval prevents repeated alerts from saturating the LLM scheduler.

When a mutation invalidates the current tactical assumptions, the alerter suspends execution and triggers a scene update. If the vehicle set and relational structure remain valid, the lightweight \Delta path updates the graph; otherwise, a full multimodal regeneration is requested. Tactical planning then resumes from the updated graph and feasibility state. The alerter only detects and localizes scene changes—semantic interpretation and policy regeneration remain the responsibility of the LLM.

This architecture allows perception to react at sensor rate without requiring the LLM to operate at 20 Hz. The resulting latency and scene-change response are evaluated in Sec. V.

V Experimental Evaluation

We evaluate TACTIC along three dimensions: (i) whether the physical constraints in Sec. III-C correctly characterize feasible operating regions; (ii) whether LLM-based tactical planning improves attack effectiveness and efficiency over fixed and random policies; and (iii) how multimodal scene reasoning and asynchronous replanning affect performance under changing traffic conditions. We first establish the experimental setting and physical operating bounds, then evaluate tactical decision quality, and finally isolate the contribution of individual system components.

V-A Experimental Setup

We implement TACTIC in CARLA 0.9.14 [22] using Town07 with synchronous simulation at 20 Hz, as illustrated in Figs. 3 and 4. Each trial contains the target T, a lead vehicle A1, a follower A2, and two additional traffic participants A3/A4. To isolate dynamic vehicle interactions from occlusion, we target the lane closest to the roadside perception stack. The attacker uses pole-mounted LiDAR and an RGB camera in a setting similar to MVS [5], with an effective observation range of approximately 50 m. The longitudinal vehicles cruise at 6.0 m/s nominally (closed-loop \approx 5.5 m/s), maintaining approximately stationary gaps without attack. Qwen [23] serves as the primary MLLM backend.

Monte Carlo protocol

The initial T–A1 gap is sampled uniformly from 18–28 m, spanning the measured push-away feasibility transition. Each configuration contains 20 trials with distinct seeds and at most three attack rounds. Success requires a physical collision, defined as a center-to-center distance below 5.0 m with residual impact speed of at least 1.5 m/s. We additionally run 20 attack-free control trials for 30 s and report Wilson 95% confidence intervals and Fisher’s exact test. Relevant secondary metrics are t_{90}, the time to reach 90% of commanded displacement; d{<}7, trials with minimum T–A1 distance below 7 m; R{<}1, displacement-tracking RMSE below 1 m; emission Gini G; and dose efficiency \eta. Together, these metrics capture not only collision success but also physical execution quality and attack efficiency.

V-B Feasibility Characterization and Attack Effectiveness

We first validate the physical constraints used to ground tactical decisions. These experiments establish the operating regions within which the LLM is allowed to reason, after which we evaluate whether scene-aware policy selection improves attack effectiveness.

Push-away feasibility

A locked-displacement sweep reveals a sharp transition: \Delta d=5 and 10 m succeed in 0/20 and 5/20 trials, whereas 15 and 20 m both succeed in 20/20 (Table I). A finer sweep gives 1/20 at 8 m, 0/20 at 9 m, 5/20 at 10 m, 12/20 at 10.5 m, and 20/20 at 11–12 m, motivating the sufficient floor \Delta d_{\min}=11 m. The upper bound follows sensing coverage, \Delta d_{\max}=50-o; beyond the sufficient floor, larger displacement mainly increases exposure rather than improving success.

The ramp must also complete before contact while remaining below the tracking-consistency limit. Instantaneous steps are detected in 10/10 trials and ramps above 3.0 m/s in 3/3, whereas no tested ramp at or below 3.0 m/s is flagged (0/10). Together with \rho\geq\Delta d/t_{\mathrm{arr}}, this defines the feasible band in Fig. 5. The full policy operates at \Delta d\approx 17.4 m and \rho\approx 2.97 m/s, placing its typical operating point safely inside this region.

Phantom-braking feasibility

We next characterize the complementary braking primitive. A locked sweep over emission duration at a fixed 13–15 m A2–T gap shows no clear duration threshold: success remains high from 2–10 s. Thus, within this tested range, the binding factor is the follower gap rather than emission duration. The virtual wall follows w(v)=1.5v, keeping perceived TTC below the 1.6 s full-braking threshold. These results confirm that the two primitives are limited by different aspects of scene geometry and therefore occupy distinct state-dependent feasibility regions.

Dose response

Figure 6 summarizes the resulting dose–response behavior of both primitives. Push-away success rises sharply around 10.5–11 m and saturates thereafter, validating the 11 m operating floor. In contrast, phantom-braking remains at or above 80% success (16/20–20/20) across 2–10 s emission durations at a fixed 13–15 m gap, further indicating that follower geometry rather than duration is the dominant constraint. The contrasting response profiles motivate selecting the primitive according to the current traffic state rather than applying one fixed policy.

Decision-source comparison

Having established the feasible action space, we next evaluate how different decision mechanisms use it. We compare four decision sources under the hybrid implementation, where both push-away and phantom-braking are available: the full LLM policy, restricted LLM, fixed rule, and committed random policy defined in Sec. IV-B. The full policy achieves 20/20 successes, significantly exceeding the fixed rule’s 7/20 and random policy’s 12/20 (Fisher p=1.3\times 10^{-5} and p=0.0033; Table II). The fixed rule always chooses push-away, leaving success dependent on sampled geometry; all 13 failures are AEB-arrested near-misses. Random selection can choose either primitive but cannot jointly adapt its parameters to the observed scene.

The restricted LLM achieves 15/20, significantly below the full policy (p=0.047), indicating that joint mode and parameter selection improves over mode selection alone. The full policy also has the lowest duty cycle (0.39) and emission dose (3.9), showing that adaptive parameterization improves both effectiveness and signal efficiency. Overall, these results support the central design choice of coupling scene-level primitive selection with continuous parameter adaptation.

Case studies

The quantitative results above are further illustrated by two representative executions. Figure 3 shows a phantom-braking episode. At T_{2}=8.0 s, the A2–T geometry becomes feasible; the cached policy is released, an 8 m phantom wall triggers emergency braking, and A2 rear-ends T at T_{3}=10.4 s. Figure 4 shows replanning under changing traffic. A push-away policy becomes feasible at T_{2}=3.9 s, but lead-vehicle acceleration invalidates the cached conditions and suspends execution. When feasibility is restored at T_{4}=13.3 s, the scene graph and policy are updated and execution resumes, eventually causing a T–A1 collision. Together, these examples illustrate how relational scene reasoning, primitive selection, and high-rate feasibility monitoring interact during execution.

Benign control

As a sanity check, no collision occurs in 20 attack-free trials; the minimum separation is 17.9 m and TTC never enters the hazard region. The observed collisions therefore arise from attack execution rather than the nominal traffic dynamics.

V-C Ablation and Runtime Analysis

We finally isolate the contributions of multimodal perception, execution timing, and asynchronous scene updates. These experiments evaluate the components that keep the tactical policy responsive as traffic conditions evolve.

Perception input

Removing either input channel significantly reduces success (Table ): physical-only input reaches 13/20 (p=0.0083) and image-only input 15/20 (p=0.047), versus 20/20 with joint input. The two ablations are not significantly different from each other (p=0.73), indicating that structured measurements and visual context provide complementary information. The joint representation therefore improves performance by combining precise metric state with semantic road and traffic context.

Attack timing

We next test whether deliberately waiting for a nominally vulnerable state improves execution. Immediate execution achieves 20/20 successes, versus 9/20 when waiting for a predefined vulnerable window (gap <12 m and closing speed >0.5 m/s within 4 s), with Fisher p=1.5\times 10^{-4} (Table ). Waiting also increases mean trial time from 34.5 to 43.7 s. Thus, delaying execution can consume the spatio-temporal feasibility envelope before the desired condition is reached; we therefore execute once the current state satisfies the primitive constraints.

Asynchronous replanning

Finally, we evaluate whether asynchronous processing reduces the latency introduced by MLLM reasoning. The reasoning channel makes approximately 2.5 LLM calls per trial. During steady traffic, all ten \Delta-path probes correctly preserve the existing topology while updating metric edge attributes. Under injected scene mutations, the local alerter suspends execution within 0.05 s. Synchronous regeneration requires 7.4 s before execution resumes, asynchronous regeneration reduces this to 3.7 s, and asynchronous \Delta refresh further reduces it to 2.0 s when topology remains unchanged (Table ). Decoupling high-rate monitoring from lower-rate MLLM reasoning therefore substantially reduces scene-update latency while preserving rapid local response.

Backend generality

To test whether these results depend on a particular MLLM, we repeat both LLM decision groups using Kimi [24]. Qwen [23] achieves 20/20 and 15/20 successes for the full and restricted policies, respectively, while Kimi achieves 20/20 and 16/20. The differences are not significant (Fisher p=1.0), suggesting that the observed advantage is not specific to one MLLM backend.

VI Conclusion

This paper presents TACTIC, a scene-dependent framework for physical LiDAR attack planning in dynamic traffic. TACTIC combines multimodal LLM-based relational reasoning with calibrated physical constraints and asynchronous replanning to select feasible attack tactics from evolving scene context. Across 280 CARLA trials in five ablation experiments, the full policy achieves a 100% collision rate across both primitives, significantly above the fixed-rule (35%) and random (60%) baselines. Joint physical-and-image perception sustains 100%, whereas single-channel ablations drop to 65% and 75%. Asynchronous replanning reduces scene-change response latency from 7.4 s to 2.0 s. These results demonstrate that scene-aware tactical planning can expose failure modes missed by static attack evaluation. Future work will extend TACTIC to hardware platforms, learned perception stacks, and richer sequence-level tactics.

References

1. [1] Y. Cao, C. Xiao, B. Cyr, Y. Zhou, W. Park, S. Rampazzi, Q. A. Chen, K. Fu, and Z. M. Mao (2019) Adversarial sensor attack on lidar-based perception in autonomous driving. In Proceedings of the 2019 ACM SIGSAC conference on computer and communications security, pp. 2267–2281.
2. [2] Z. Jin, X. Ji, Y. Cheng, B. Yang, C. Yan, and W. Xu (2023) Pla-lidar: physical laser attacks against lidar-based 3d object detection in autonomous vehicle. In 2023 IEEE Symposium on Security and Privacy (SP), pp. 1822–1839.
3. [3] Q. Zhang, S. Luo, Z. M. Mao, M. Pajic, and M. K. Reiter (2025) SoK: how sensor attacks disrupt autonomous vehicles: an end-to-end analysis, challenges, and missed threats. arXiv preprint arXiv:2509.11120.
4. [4] Y. Cao, S. H. Bhupathiraju, P. Naghavi, T. Sugawara, Z. M. Mao, and S. Rampazzi (2023) You can’t see me: physical removal attacks on \{lidar-based\} autonomous vehicles driving frameworks. In 32nd USENIX security symposium (USENIX Security 23), pp. 2993–3010.
5. [5] T. Sato, R. Suzuki, Y. Hayakawa, K. Ikeda, O. Sako, R. Nagata, R. Yoshida, Q. A. Chen, and K. Yoshioka (2025) On the realism of lidar spoofing attacks against autonomous driving vehicle at high speed and long distance. In Network and Distributed System Security Symposium (NDSS),
6. [6] J. Zhang, S. Cheng, L. Hu, J. Zhang, C. Shi, X. Han, T. Zhang, Y. Cheng, and W. Zhang (2025) The ghost navigator: revisiting the hidden vulnerability of localization in autonomous driving. In 34th USENIX Security Symposium (USENIX Security 25), pp. 3979–3998.
7. [7] Z. Song, X. Chen, Z. Zhang, K. Zhang, J. Lu, and W. Li (2025) Gradient-based adversarial attacks on deep lidar odometry. In 2025 IEEE International Conference on Robotics and Automation (ICRA), pp. 15188–15194.
8. [8] J. Tu, M. Ren, S. Manivasagam, M. Liang, B. Yang, R. Du, F. Cheng, and R. Urtasun (2020) Physically realizable adversarial examples for lidar object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 13716–13725.
9. [9] J. Sun, Y. Cao, Q. A. Chen, and Z. M. Mao (2020) Towards robust lidar-based perception in autonomous driving: general black-box adversarial sensor attack and countermeasures. In 29th USENIX Security Symposium, pp. 877–894.
10. [10] B. Nassi, Y. Mirsky, D. Nassi, R. Ben-Netanel, O. Drokin, and Y. Elovici (2020) Phantom of the adas: securing advanced driver-assistance systems from split-second phantom attacks. In Proceedings of the 2020 ACM SIGSAC Conference on Computer and Communications Security, pp. 293–308.
11. [11] H. Kim, R. Bandyopadhyay, M. O. Ozmen, Z. B. Celik, A. Bianchi, Y. Kim, and D. Xu (2024) A systematic study of physical sensor attack hardness. In 2024 IEEE Symposium on Security and Privacy (SP),
12. [12] Y. Mei, T. Nie, J. Sun, and Y. Tian (2025) LLM-attacker: enhancing closed-loop adversarial scenario generation for autonomous driving with large language models. arXiv preprint arXiv:2501.15850.
13. [13] J. Mao, Y. Qian, J. Ye, H. Zhao, and Y. Wang (2023) Gpt-driver: learning to drive with gpt. arXiv preprint arXiv:2310.01415.
14. [14] H. Shao, Y. Hu, L. Wang, G. Song, S. L. Waslander, Y. Liu, and H. Li (2024) Lmdrive: closed-loop end-to-end driving with large language models. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 15120–15130.
15. [15] C. Sima, K. Renz, K. Chitta, L. Chen, H. Zhang, C. Xie, J. Beißwenger, P. Luo, A. Geiger, and H. Li (2024) Drivelm: driving with graph visual question answering. In European conference on computer vision, pp. 256–274.
16. [16] G. Chen, X. Yu, N. Ling, and L. Zhong (2025) TypeFly: low-latency drone planning with large language models. IEEE Transactions on Mobile Computing 24 (9), pp. 9068–9079.
17. [17] A. Kesting, M. Treiber, and D. Helbing (2010) Enhanced intelligent driver model to access the impact of driving strategies on traffic capacity. Philosophical Transactions: Mathematical, Physical and Engineering Sciences, pp. 4585–4605.
18. [18] Y. Yao and S. Luo (2026) SEIDM: a safe and efficient intelligent driver model for autonomous driving behavior. arXiv preprint arXiv:2605.23915.
19. [19] European New Car Assessment Programme (Euro NCAP) (2024) Test protocol – aeb/lss vru systems. Note: v4.5.1, Feb. 2024
20. [20] J. Johnson, R. Krishna, M. Stark, L. Li, D. A. Shamma, M. Bernstein, and L. Fei-Fei (2015) Image retrieval using scene graphs. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3668–3678.
21. [21] D. Xu, Y. Zhu, C. B. Choy, and L. Fei-Fei (2017) Scene graph generation by iterative message passing. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5410–5419.
22. [22] A. Dosovitskiy, G. Ros, F. Codevilla, A. Lopez, and V. Koltun (2017) CARLA: an open urban driving simulator. In Proceedings of the 1st Annual Conference on Robot Learning (CoRL), pp. 1–16.
23. [23] Qwen Team (2025) Qwen3-vl technical report. Note: arXiv preprint arXiv:2511.21631
24. [24] K. Team, Y. Bai, Y. Bao, Y. Charles, C. Chen, G. Chen, H. Chen, H. Chen, J. Chen, N. Chen, et al. (2025) Kimi k2: open agentic intelligence. arXiv preprint arXiv:2507.20534.