MEKAWIRE
ExpertarXiv

Localized Conformal Safety Monitoring with Vision-Language Models for Autonomous Driving

Luís Marques, Rong Fang, Disha Kamale, Dmitry Berenson
In 30 seconds

This preprint introduces SLLCP, a post-hoc calibration layer over frozen vision-language models that turns raw safe/unsafe guesses into probabilistic safety prediction sets for autonomous driving. On 15,321 unseen CARLA frames, SLLCP flags 89.6% of collision-causing trajectories with Qwen3-VL-8B and 88.4% with Cosmos-Reason2-8B, versus 4.6% and 39.1% for the raw binary VLMs. It matters because it offers a planner-agnostic way to reduce missed unsafe trajectories before execution, albeit with more false alarms.

Research question

Can local, label-conditional conformal calibration turn unreliable frozen VLM trajectory-safety predictions into a monitor with probabilistically bounded missed-unsafe rates?

Problem

Before a self-driving car executes a candidate trajectory, it must judge whether it will collide within a short horizon; this is hard because nearby pedestrians and vehicles react to the ego vehicle, occlusions can hide actors, and forecasters may miss or mispredict responses. Existing geometric or forecasting checks depend on accurate scene knowledge, while raw VLM predictions are overconfident and lack safety guarantees.

Previous approach

Standard pipelines forecast nearby actors and then collision-check the planned path, but their validity degrades when an actor is missed or reacts differently. Raw VLM binary or thresholded probability monitors lack calibration; split conformal prediction gives only marginal coverage, label-conditional CP handles class imbalance but not scene-dependent difficulty, and split localized CP provides only marginal coverage despite local thresholds.

New approach

SLLCP uses frozen open-weight VLMs and their final hidden states. A linear logistic probe is fit on VLM embeddings from a local calibration split to produce a scalar unsafe score instead of using token probabilities. A 2D UMAP embedding of those hidden states provides scene-local distances. A Laplace kernel computes weighted per-label unsafe-score quantiles, and a disjoint correction split adds per-label residual quantile corrections. The monitor raises an alarm when unsafe is in the resulting prediction set. No VLM weights are updated.

Results

In CARLA on 15,321 held-out frames from unseen scenario IDs, with 2,489 unsafe frames and α=0.1, SLLCP achieved 89.6% unsafe coverage at 51.8% alarm FPR for Qwen, and 88.4% coverage at 44.5% FPR for Cosmos; ambiguity was 37.6% and 33.6%. Raw binary VLMs flagged only 4.6% of unsafe frames for Qwen and 39.1% for Cosmos. A 0.1/0.9 thresholded rule reached 20.1% coverage for Qwen and 78.1% for Cosmos, with Cosmos FPR at 37.3%. Ablations: removing locality gave 77.5%/80.5% coverage; removing projection gave 84.3%/82.4%; removing label-conditional correction gave 70.9%/66.7%; removing the logistic probe produced 99.2%/99.6% coverage but 95.6%/99.0% FPR. SLLCP improved over raw binary by +85.0 points for Qwen and +49.3 points for Cosmos, and reduced FPR versus the no-probe ablation by 43.8 and 54.5 points.

Limitations

Authors state that results are sensitive to the UMAP projection and kernel hyperparameters, video frames are correlated and break exchangeability, and scenario shift from calibration affects empirical coverage. Evident limitations include simulation-only CARLA evaluation, a front-facing camera setup, only two open-weight VLMs, high false-alarm rates above 44%, no real-world or closed-loop policy deployment, and no reported code release or production hardware study.

Industry impact

Autonomous driving developers, ADAS suppliers, and simulation or validation platforms could use this as an offline or online safety monitor over existing planners, especially for hard-to-model interactive risks. Near-term use in simulation testing, scenario mining, and offline trajectory triage is plausible within 1–3 years; production deployment is 3+ years away and would require reducing false alarms, validating on real-world correlated data, and meeting safety standards. Open-weight VLM access and feasible real-time embedding inference would also need to hold.

Full paper

Localized Conformal Safety Monitoring with Vision-Language Models for Autonomous Driving

Luís Marques, Rong Fang, Disha Kamale, Dmitry Berenson

Published under CC BY 4.0. Reproduced with attribution; original at arXiv:2610.02765 (PDF).

Abstract

Monitoring planned driving trajectories requires accurately estimating the collision likelihood with actors whose motion is itself impacted by the ego motion. Existing classical approaches are often limited by the quality of their forecasting model. Vision-language models (VLMs) have shown promise in reasoning about the consequences of high-level actions, yet their approximate predictions are unsuitable for safety-critical applications such as autonomous driving. Conformal prediction (CP) has emerged as a data-driven framework for quantifying the uncertainty of black-box model predictions. We propose Split Label-Localized Conformal Prediction (SLLCP), a post-hoc calibration layer over frozen VLMs that transforms their unreliable predictions into probabilistically calibrated safety prediction sets. We consider how the ability to estimate safety can depend on the observed driving scene and introduce a localized procedure that upweights relevant past experience when calculating uncertainty thresholds. We provide label-conditional finite-sample distribution-free coverage under exchangeability. Evaluated over 15k CARLA trajectories from unseen scenarios, SLLCP correctly flags 89.6\% of collision-causing trajectories with a Qwen backbone and 88.4\% with a Cosmos backbone, while the base VLMs only flagged 4.6\% and 39.1\% of the collision-causing trajectories, respectively. These results indicate that local, label-conditional calibration can reduce missed unsafe trajectories.

Index Terms:

Conformal prediction, safety monitoring, uncertainty quantification, vision-language models, autonomous driving

I Introduction and Related Work

Before self-driving cars execute planned trajectories, they must assess whether the intended motion can be safely executed, considering the surrounding scene. This is nontrivial: pedestrians and other vehicles move in response to the ego vehicle. For example, a trajectory passing a pedestrian near the curb may be collision-free if the pedestrian waits, but unsafe if they step into the vehicle’s path. Further, the information available at planning time might miss occluded actors and not provide sufficient signal to enable accurate forecasting of the scene’s evolution. Even seemingly robust driving policies can demonstrate degraded performance when operating in unfamiliar scenarios — Fail2Drive reports substantial drops in closed-loop success under controlled scenario shifts [5].

We study the problem of designing a safety monitor that examines each candidate trajectory before execution, independently of the quality of the underlying planning module. That monitor should flag trajectories that would collide within a fixed time horizon, while avoiding unnecessary alarms hindering task progress. In general, a safety monitor might need to reason about the candidate motion and the scene jointly, while considering uncertainty in the true unknown outcome. While vision-language driving systems, such as SimLingo, incorporate scene understanding with action-taking [17], we aim to construct a monitor that can be applied post-hoc over any base planner. We seek a theoretically-informed way to probabilistically limit the number of missed unsafe trajectories.

Safety monitors. Driving pipelines often forecast the motion of nearby actors before checking a planned trajectory for collision [16]. Model-based safety checks can account for uncertain actor motion and dynamics [9], but their assurances depend on assumed scene knowledge. When an actor is missed entirely or its response to the ego vehicle is incorrectly predicted, using a collision check may fail. We thus investigate the complementary problem of assessing candidate trajectories using front-facing images, so that a vision-language model (VLM) can assess hard to model risks. A VLM’s raw safe/unsafe prediction provides no assurances of low-rate missed collisions, since VLM spatial reasoning can be unreliable [18]. Hence, we aim to augment VLMs’ common-sense reasoning with a post-hoc calibration layer providing probabilistic guarantees on their prediction performance.

Fig. 1: Uncertainty in VLM-based trajectory safety predictions (safe or unsafe) varies across driving scenes. SLLCP upweights calibration examples (bottom) near each test input (top) in the projected embedding space and calibrates each safety label separately to address class imbalance in the collected data (e.g., due to unsafe cases being rarer). Trajectory footprints are shown in blue, with yellow waypoints spaced at 0.5\,s intervals.

Conformal prediction in robotics. Conformal prediction (CP) transforms approximate predictions into prediction sets with finite-sample guarantees under exchangeability [19, 1]. In robotics, CP has calibrated: forecasts in dynamic scenes for social navigation [9, 3]; how dynamics uncertainty varies over a system’s state–action space [11]; action uncertainty for Lie-algebraic systems [12]; and motion uncertainty across contact modes [13].

In this extended abstract, we propose a monitor calibrating a base VLM safety prediction in a scene-adaptive manner. Our key contributions are:

  • We develop Split Label-Localized Conformal Prediction (SLLCP), a post-hoc method over a planner-agnostic generated trajectory image and a frozen VLM backbone, that provides probabilistic guarantees on a low probability of missing unsafe trajectories.
  • We empirically validate the tradeoff between missed unsafe trajectories and false alarms on edge-case CARLA scenarios, demonstrating significant improvements in the calibration of two open-weight VLMs, using the same hyperparameter set.

II Problem Statement

Let x_{t} denote the state of an autonomous driving agent at time t\in\mathbb{N}, whose planner generates candidate H-step-long trajectories \tau:=\{x_{t},x_{t+1},\ldots,x_{t+H-1}\}. We consider the problem of provably monitoring the safety of said finite-horizon trajectories given a black-box planner of unknown quality. We assume that executing \tau without replanning can produce one of two, initially unknown, outcomes Y: either a collision occurs within H steps (Y=\textsc{unsafe}), or it does not (Y=\textsc{safe}). While conventional collision-based monitors could be implemented by projecting \tau into the environment observed so far and performing geometric checks, the validity of their predictions is dependent on an accurate forecast of the motion of other dynamic entities in the scene, such as pedestrians and other vehicles. However, these forecasts are notoriously difficult to estimate accurately, even more so given that the ego motion can influence the behaviours of nearby actors. In this work, we consider vision-language models (VLMs) with fixed parameters as the base of our safety monitor, due to their common-sense reasoning capabilities. However, VLM-estimated safety likelihood \hat{p}(Y\mid X) is not directly suitable for safety-critical applications, since it might be overconfident. Our VLM-based monitor receives as input X, consisting of a front-facing RGB camera image of the scene overlaid with the proposed trajectory, as well as a prompt describing the safety monitoring task.

Problem. Given a user-specified acceptable failure rate \upalpha\in(0,1), we seek a set-valued monitor m:\mathcal{X}\to 2^{\{\textsc{safe},\textsc{unsafe}\}} whose probability of missing an unsafe trajectory is at most \upalpha. Formally, we require

\mathbb{P}(\textsc{unsafe}\notin\widehat{C}(X)\mid Y=\textsc{unsafe})\leq\upalpha,
(1)

and further want to minimize the false positive rate (FPR)

\operatorname{FPR}=\mathbb{P}(\textsc{unsafe}\in\widehat{C}(X)\mid Y=\textsc{safe}),
(2)

since high FPR implies unnecessary alarms which can hinder the agent’s task performance. To make this problem tractable, we make the standard assumption in the conformal prediction literature of having access to a held-out calibration set of labeled examples that are iid with the test data, i.e.:

Assumption 1

We have VLM inputs-outcome pairs D_{\mathrm{cal}}=\{(X_{i},Y_{i})\}_{i=1}^{n} that are iid with the test pair (X_{n+1},Y_{n+1}).

Following literature, D_{\mathrm{cal}} is uncorrupted.

III Background: Conformal Prediction

We briefly introduce relevant conformal prediction (CP) concepts below and refer to [19, 1] for deeper insight. Let X\in\mathcal{X} denote the available information at the time of prediction and Y\in\mathcal{Y}=\{\textsc{safe},\textsc{unsafe}\} the unknown true label. Given a fixed classifier, the estimated unsafe probability is \widehat{Y}(x)=\hat{p}(\textsc{unsafe}\mid x). The nonconformity score function r:[0,1]\times\mathcal{Y}\to\mathbb{R} measures disagreement between a model prediction and the true label, with higher values indicating higher prediction uncertainty. For binary classification, a conventional choice is r(\widehat{Y}(x),y)=1-\hat{p}(y\mid x).

Split conformal prediction (SplitCP). SplitCP is a computationally efficient CP variant that uses an exchangeable calibration set of labeled examples (Ass. 1) to provide probabilistic guarantees on test-time prediction uncertainty. Given D_{\mathrm{cal}}, we can evaluate the nonconformity score on each example to obtain a list of scores \{R_{i}\}_{i=1}^{n}, where R_{i}=r(\widehat{Y}(X_{i}),Y_{i}). Then the k=\lceil(n+1)(1-\upalpha)\rceilth smallest score in \{R_{i}\}_{i=1}^{n}\cup\{\infty\}, serves as a probabilistic threshold \widehat{q} for the test-time nonconformity score r(\widehat{Y}(X_{n+1}),Y_{n+1}). Using exchangeability [19], the following prediction set

\widehat{C}(x)=\{y\in\mathcal{Y}:r(\widehat{Y}(x),y)\leq\widehat{q}\},
(3)

is guaranteed to provide marginal coverage, i.e., satisfy \mathbb{P}\{Y\in\widehat{C}(X)\}\geq 1-\upalpha on average.

Label-conditional prediction. Marginal coverage alone does not account for imbalanced datasets. Since the guarantees are based on average uncertainty, underrepresented labels in D_{\mathrm{cal}} that present higher predictive uncertainty can be severely undercovered. For instance, D_{\mathrm{cal}} may contain many more safe than unsafe examples, and unsafe might be more difficult for the VLM to estimate. To address this issue, Label-conditional CP instead calibrates each class label separately [19]. Let I_{y}=\{i:Y_{i}=y\} be the calibration indices for class y, n_{y}=|I_{y}| the number of calibration examples y, and let \widehat{q}_{y} be the \lceil(n_{y}+1)(1-\upalpha)\rceilth smallest score in \{R_{i}:i\in I_{y}\}\cup\{\infty\}. Replacing the SplitCP label-shared threshold \widehat{q} in Eq. (3) by the per-label threshold \widehat{q}_{y} gives rise to the prediction set

\displaystyle\widehat{C}(x) \\ \displaystyle=\{y\in\mathcal{Y}:r(\widehat{Y}(x),y)\leq\widehat{q}_{y}\},

which satisfies label-conditional coverage, i.e., implies Eq. (1). This guarantee holds for finite-sized D_{\mathrm{cal}} in a distribution-free manner, i.e., without making any assumptions on the underlying score distribution. The prediction region \widehat{C} may now contain safe, unsafe, both labels, or the empty set. If \widehat{C} includes unsafe, the monitor should raise a safety alarm.

Split localized conformal prediction (SLCP). While each label-dependent threshold \widehat{q}_{y} is input-independent, prediction difficulty may vary across inputs x. For example, some driving scenes may be simpler to classify as unsafe than others, depending on the size, color, and proximity of nearby obstacles. Hence, we desire a threshold that can adapt to the model uncertainty at each input. Unfortunately, input-conditional coverage \mathbb{P}(Y\in\widehat{C}(X)\mid X=x)\geq 1-\upalpha has been shown to be impossible with finite data, without making additional distributional assumptions [8]. SLCP aims to approximate input-conditional coverage by fitting a local score quantile \widehat{q}_{\mathrm{loc}}(x) which up-weights calibration examples with inputs “near” the test-time input x. The key idea is that nearby inputs might share similar prediction difficulty, and thus their scores should be considered as more informative than the scores of faraway calibration examples. Since on its own \widehat{q}_{\mathrm{loc}}(x) does not provide any finite-sample guarantees, SLCP then adds a held-out correction \widehat{q}_{\mathrm{corr}} to obtain a final composed threshold \widehat{q}_{\mathrm{SLCP}}(x) [6, 7]. For exchangeability to hold, D_{\mathrm{cal}} must be initially split into two disjoint sets D_{\mathrm{cal}}^{\mathrm{local}} and D_{\mathrm{cal}}^{\mathrm{CP}} used for local fitting and held-out correction, respectively. Let I_{\mathrm{local}} and I_{\mathrm{CP}} index D_{\mathrm{cal}}^{\mathrm{local}} and D_{\mathrm{cal}}^{\mathrm{CP}}, respectively, and let the local distance between a calibration input X_{i} and a test input x be d_{i}(x)=d(\phi(X_{i}),\phi(x)) where \phi is a feature mapping. A kernel \kappa and bandwidth h>0 define the local weights

w_{i}(x)=\frac{\kappa(d_{i}(x)/h)}{\sum_{j\in I_{\mathrm{local}}}\kappa(d_{j}(x)/h)},
(5)

and \widehat{q}_{\mathrm{loc}}(x) is the weighted (1-\upalpha) quantile of D_{\mathrm{cal}}^{\mathrm{local}} scores. The held-out correction \widehat{q}_{\mathrm{corr}} is the SplitCP quantile of \{R_{j}-\widehat{q}_{\mathrm{loc}}(X_{j}):j\in I_{\mathrm{CP}}\}, giving the SLCP composite threshold of

\widehat{q}_{\mathrm{SLCP}}(x)=\widehat{q}_{\mathrm{loc}}(x)+\widehat{q}_{\mathrm{corr}}.
(6)

Despite possibly better empirical performance, SLCP only provides marginal coverage due to \widehat{q}_{\mathrm{corr}}’s construction [7].

IV Method: SLLCP

We propose Split Label-Localized Conformal Prediction (SLLCP), which builds on SLCP by combining it with label-conditional conformal prediction. Instead of using the raw VLM nonconformity score, we learn a nonconformity score to better inform the local quantile fit. Figure 2 summarizes our method. Following SLCP, we split D_{\mathrm{cal}}=D_{\mathrm{cal}}^{\mathrm{local}}\sqcup D_{\mathrm{cal}}^{\mathrm{CP}}, using D_{\mathrm{cal}}^{\mathrm{local}} to both learn r and fit \widehat{q}_{\mathrm{loc}}, while D_{\mathrm{cal}}^{\mathrm{CP}} now informs per-label conformal corrections \widehat{q}_{\mathrm{corr},y} for each label y\in\mathcal{Y}.

Fig. 2: SLLCP overview. Monitor inputs consist of a camera image with the proposed trajectory overlaid and a fixed safety prompt. Offline: using D_{\mathrm{cal}}^{\mathrm{local}}, we fit a logistic probe to obtain safety scores and a UMAP projection for localization, then define per-label local score quantiles. The disjoint D_{\mathrm{cal}}^{\mathrm{CP}} provides per-label corrections from score residuals. Online: given a test-time image–prompt pair, we combine its local quantiles with the fitted corrections to construct \widehat{C}(x). The monitor raises an alarm when \textsc{unsafe}\in\widehat{C}(x).

Probe score and local representation with D_{\mathrm{cal}}^{\mathrm{local}}. Let \mathbf{e}_{\mathrm{VLM}}(x) be the VLM’s hidden state before the Boolean answer token (safe, unsafe) and after the frozen language model’s final RMSNorm. A raw VLM-based score could be computed from the VLM’s Boolean-token probabilities. However, we have empirically observed that this score may not expose all safety-relevant information contained in the VLM hidden state. Therefore, we fit a linear logistic probe \widehat{Y}(x)=\sigma(\mathbf{w}^{\top}\mathbf{e}_{\mathrm{VLM}}(x)+b) to \mathbf{e}_{\mathrm{VLM}}(x) using the ground-truth labels in D_{\mathrm{cal}}^{\mathrm{local}}. This simple classifier provides an alternative way to estimate the unsafe probability \widehat{Y}(x), which we then use instead of Boolean token probabilities to compute the standard binary nonconformity score of Sec. III. This is cheap and does not require updating any VLM weights. For localization, since the raw VLM embedding is very high-dimensional, we first fit a two-dimensional UMAP [14] projection g to \{\mathbf{e}_{\mathrm{VLM}}(X_{i}):i\in I_{\mathrm{local}}\}, yielding a lower-dimensional latent representation of the input \mathbf{z}(x)=g(\mathbf{e}_{\mathrm{VLM}}(x)). This \mathbf{z}(x) is then used as the feature mapping \phi(x) in Eq. (5), leading to the local distance d(u,v)=\|u-v\|_{2} for projected vectors u:=\mathbf{z}(X_{i}),v:=\mathbf{z}(x). Both the classifier and UMAP fits are fixed independently of D_{\mathrm{cal}}^{\mathrm{CP}}.

Label-conditional local quantiles and correction. For each label y\in\mathcal{Y}, let I_{y}^{\mathrm{local}}=\{i\in I_{\mathrm{local}}:Y_{i}=y\}. We obtain the label-conditional thresholds \widehat{q}_{\mathrm{loc},y}(x) by computing the weighted (1-\upalpha) quantile of the scores R_{i}=r(\widehat{Y}(X_{i}),y) for i\in I_{y}^{\mathrm{local}}, using Eq. (5). Given correction examples j\in I_{\mathrm{CP}} with Y_{j}=y, their residual quantile gives \widehat{q}_{\mathrm{corr},y} similarly to SLCP. The \widehat{C} then is

\widehat{C}(x)=\{y\in\mathcal{Y}:r(\widehat{Y}(x),y)\leq\widehat{q}_{\mathrm{loc},y}(x)+\widehat{q}_{\mathrm{corr},y}\}.
(7)

Our safety monitor raises an alarm when \textsc{unsafe}\in\widehat{C}(x), to match the objective in Sec. II. Assuming within-label exchangeability, the proposed method provides label-conditional marginal coverage. While this assumption might not hold in our experiments, we demonstrate that our theory-driven method still provides adequate empirical coverage in practice.

V Experimental Results

We evaluate if the proposed safety monitor can detect collision-causing trajectories in the CARLA simulator [4]. We report unsafe-label coverage, the monitor’s false-positive rate on safe trajectories, and the fraction of produced prediction sets containing both labels. The latter indicates “ambiguity” in the monitor’s decision, and could be used to trigger a different recovery policy than unsafe-alarms. We use \upalpha=0.1 for all experiments. Coverage is a random variable that, under exchangeability, follows a Beta-binomial distribution given a fixed calibration set and a finite test set [10]. We report if the observed empirical coverage falls within the 90\% confidence interval of the nominal coverage 1-\upalpha, which would indicate if the applied procedure mitigates the nonexchangeable nature of the data.

We compare SLLCP with several relevant baselines: two uncalibrated raw-VLM rules and four component ablations. Raw VLM (binary) emits one safe/unsafe decision: the higher-scoring candidate label. Raw VLM (0.1/0.9) instead uses a \upalpha-aware scoring and a two-label prediction set. This would be the natural monitor if the raw VLM score were well-calibrated. The ablations remove, one at a time: locality (global, input-independent label-conditional CP); the linear probe (using original VLM score); the UMAP projection (raw embedding distances); the per-label correction (use label-agnostic uncertainty threshold). This analysis aims to isolate the impact of our choices of score, latent space representation, and label-aware correction.

We collect trajectories and front-facing images using the SimLingo planner [17]. A trajectory is deemed unsafe if tracking it for 2.5\,s with a PID controller causes a collision with a vehicle, pedestrian, or other object. In total, we obtain 40,455 frames from 90 episodes across 30 underlying scenario IDs in three Fail2Drive families [5]: HardBrake, PedestriansOnRoad, and Wall (seen in Figure 2). As base VLMs, we evaluated Qwen3-VL-8B-Instruct [2] and Cosmos-Reason2-8B [15]. We relied on open-weight VLMs so that we could access the hidden embeddings needed for the linear probe and for doing localized CP in embedding-space. Neither VLM was fine-tuned and both received the same fixed binary safety prompt and trajectory image.

We split the entire data into D_{\mathrm{cal}} and an evaluation set. All seeds of each scenario ID stay on the same side of this split. This prevents leakage of information from the evaluation set into D_{\mathrm{cal}} — each test scenario was not seen during the offline calibration phase. For some methods, D_{\mathrm{cal}} is further divided into D_{\mathrm{cal}}^{\mathrm{local}} and D_{\mathrm{cal}}^{\mathrm{CP}}. This results in |D_{\mathrm{cal}}^{\mathrm{local}}|= 19,528 frames (2,615 unsafe), |D_{\mathrm{cal}}^{\mathrm{CP}}|= 5,606 frames (856 unsafe), and an evaluation set with 15,321 frames (2,489 unsafe; 12,832 safe). SLLCP projected VLM embeddings to UMAP-2 (100 neighbors; Euclidean distance), then used a Laplace kernel \kappa(u)=\exp(-u), with per-label bandwidth h_{y}=0.4 times the label-specific median distance in D_{\mathrm{cal}}^{\mathrm{local}}. Hyperparameters were shared across the two VLMs, yet this results in model-specific conformal thresholds. Table I reports the three evaluation metrics over the 15,321 frames in the evaluation set.

TABLE I: Trajectory-safety results on unseen scenario IDs.
  • Raw VLM (binary)
    VLM
    Qwen
    UNSAFE coverage (%) \uparrow
    4.6
    Alarm FPR (%) \downarrow
    0.3
    Ambiguity (%) \downarrow
    0.0
  • —
    VLM
    Cosmos
    UNSAFE coverage (%) \uparrow
    39.1
    Alarm FPR (%) \downarrow
    1.9
    Ambiguity (%) \downarrow
    0.0
  • Raw VLM (0.1/0.9)
    VLM
    Qwen
    UNSAFE coverage (%) \uparrow
    20.1
    Alarm FPR (%) \downarrow
    2.1
    Ambiguity (%) \downarrow
    4.9
  • —
    VLM
    Cosmos
    UNSAFE coverage (%) \uparrow
    78.1
    Alarm FPR (%) \downarrow
    37.3
    Ambiguity (%) \downarrow
    43.9
  • Ablation w/o locality
    VLM
    Qwen
    UNSAFE coverage (%) \uparrow
    77.5
    Alarm FPR (%) \downarrow
    27.7
    Ambiguity (%) \downarrow
    18.6
  • —
    VLM
    Cosmos
    UNSAFE coverage (%) \uparrow
    80.5
    Alarm FPR (%) \downarrow
    30.4
    Ambiguity (%) \downarrow
    21.3
  • Ablation w/o logistic probe
    VLM
    Qwen
    UNSAFE coverage (%) \uparrow
    99.2
    Alarm FPR (%) \downarrow
    95.6
    Ambiguity (%) \downarrow
    89.4
  • —
    VLM
    Cosmos
    UNSAFE coverage (%) \uparrow
    99.6
    Alarm FPR (%) \downarrow
    99.0
    Ambiguity (%) \downarrow
    89.3
  • Ablation w/o projection
    VLM
    Qwen
    UNSAFE coverage (%) \uparrow
    84.3
    Alarm FPR (%) \downarrow
    41.6
    Ambiguity (%) \downarrow
    28.3
  • —
    VLM
    Cosmos
    UNSAFE coverage (%) \uparrow
    82.4
    Alarm FPR (%) \downarrow
    37.9
    Ambiguity (%) \downarrow
    26.8
  • Ablation w/o label-conditional CP
    VLM
    Qwen
    UNSAFE coverage (%) \uparrow
    70.9
    Alarm FPR (%) \downarrow
    15.4
    Ambiguity (%) \downarrow
    6.1
  • —
    VLM
    Cosmos
    UNSAFE coverage (%) \uparrow
    66.7
    Alarm FPR (%) \downarrow
    11.9
    Ambiguity (%) \downarrow
    5.2
  • SLLCP (ours)
    VLM
    Qwen
    UNSAFE coverage (%) \uparrow
    89.6
    Alarm FPR (%) \downarrow
    51.8
    Ambiguity (%) \downarrow
    37.6
  • —
    VLM
    Cosmos
    UNSAFE coverage (%) \uparrow
    88.4
    Alarm FPR (%) \downarrow
    44.5
    Ambiguity (%) \downarrow
    33.6

Raw VLM (binary) alarms included unsafe on only 4.6\% of Qwen’s and 39.1\% of Cosmos’s unsafe frames, failing to meet the user-set safety requirements. Similarly, the safety-biased raw rule also remained well below the nominal 90\% target. Our approach, SLLCP, increased unsafe-label coverage to 89.6\% and 88.4\% staying within the finite-sample Beta-binomial confidence interval of the user-set requirement. However, the alarm FPRs rose to 51.8\% and 44.5\% and both-label ambiguity to 37.6\% and 33.6\%, respectively. This indicates an efficiency–coverage tradeoff. In safety-critical applications, where consequences can be severe, one may prefer to reduce missed unsafe trajectories at the cost of more false alarms.

The ablations motivate our design decisions. Without the logistic probe, the local method includes nearly all unsafe frames but raises false alarms on at least 95.6\% of safe frames, making it impractical for real-world applications. Without UMAP, unsafe coverage falls to 84.3\% and 82.4\%. Using a label-independent threshold lowers coverage further to 70.9\% and 66.7\%. These comparisons suggest that the score, local representation, and label-specific correction each impact the coverage–false-alarm tradeoff.

Limitations. Results are sensitive to the UMAP projection and kernel hyperparameters. The original dataset includes video frames collected along full-episode SimLingo trajectories. Hence, frames are correlated, breaking exchangeability. The scenario shift from D_{\mathrm{cal}} also impacts empirical performance.

VI Conclusion

On unseen Fail2Drive scenarios in CARLA, SLLCP achieved 89.6\% and 88.4\% unsafe-label coverage for Qwen and Cosmos, improvements of 85.0 and 49.3 percentage points, respectively, over the raw VLM performance. In this extended abstract, we considered trajectory monitoring as a two-label prediction-set problem. We combined a learned score, from frozen VLM representations, with local calibration in a projected embedding space and label-conditional calibration. Compared with the no-probe ablation, our approach reduces the alarm FPR by 43.8 and 54.5 percentage points for Qwen and Cosmos, respectively, while keeping unsafe-label coverage near 90\%. SLLCP provides a planner-agnostic way to balance safety monitoring and false alarms without requiring any VLM fine-tuning. Future work could consider applications to semantic safety violations, such as failing to yield or obey traffic signals. Yet, obtaining reliable ground-truth labels in these settings is nontrivial.

References

  1. [1] A. N. Angelopoulos and S. Bates (2023) Conformal prediction: a gentle introduction. Found. Trends Mach. Learn.. External Links: Document
  2. [2] S. Bai, Y. Cai, R. Chen, et al. (2025) Qwen3-VL technical report. Note: arXiv preprint arXiv:2511.21631
  3. [3] A. Dixit, L. Lindemann, S. X. Wei, M. Cleaveland, G. J. Pappas, and J. W. Burdick (2023) Adaptive conformal prediction for motion planning among dynamic agents. In L4DC,
  4. [4] A. Dosovitskiy, G. Ros, F. Codevilla, A. Lopez, and V. Koltun (2017) CARLA: an open urban driving simulator. In CoRL,
  5. [5] S. Gerstenecker, A. Geiger, and K. Renz (2026) Fail2Drive: benchmarking closed-loop driving generalization. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS),
  6. [6] L. Guan (2023) Localized conformal prediction: a generalized inference framework for conformal prediction. Biometrika. External Links: Document
  7. [7] X. Han, Z. Tang, J. Ghosh, and Q. Liu (2022) Split localized conformal prediction. Note: arXiv preprint arXiv:2206.13092
  8. [8] J. Lei and L. Wasserman (2014) Distribution-free prediction bands for non-parametric regression. J. R. Stat. Soc. Ser. B. External Links: Document
  9. [9] L. Lindemann, M. Cleaveland, G. Shim, and G. J. Pappas (2023) Safe planning in dynamic environments using conformal prediction. IEEE Robot. Autom. Lett.. External Links: Document
  10. [10] P. C. Marques F. (2025) Universal distribution of the empirical coverage in split conformal prediction. Statist. Probab. Lett.. External Links: Document
  11. [11] L. Marques and D. Berenson (2026) Quantifying aleatoric and epistemic dynamics uncertainty via local conformal calibration. In WAFR, External Links: Document
  12. [12] L. Marques, M. Ghaffari, and D. Berenson (2026) Lies we can trust: quantifying action uncertainty with inaccurate stochastic dynamics through conformalized nonholonomic Lie groups. IEEE Robot. Autom. Lett.. External Links: Document
  13. [13] L. Marques, K. Popov, and D. Berenson (2026) Particle-based conformal prediction for contact-aware uncertainty calibration in stratified configuration spaces. In COPA,
  14. [14] L. McInnes, J. Healy, N. Saul, and L. Großberger (2018) UMAP: uniform manifold approximation and projection. JOSS. External Links: Document
  15. [15] NVIDIA (2026) Cosmos-Reason2. Note: NVIDIA Cosmos documentationAccessed September 24, 2026
  16. [16] B. Paden, M. Čáp, S. Z. Yong, D. Yershov, and E. Frazzoli (2016) A survey of motion planning and control techniques for self-driving urban vehicles. IEEE Trans. Intell. Veh.. External Links: Document
  17. [17] K. Renz, L. Chen, E. Arani, and O. Sinavski (2025) SimLingo: vision-only closed-loop autonomous driving with language-action alignment. In CVPR, External Links: Document
  18. [18] X. Tian, J. Gu, B. Li, Y. Liu, Y. Wang, Z. Zhao, K. Zhan, P. Jia, X. Lang, and H. Zhao (2025) DriveVLM: the convergence of autonomous driving and large vision-language models. In CoRL,
  19. [19] V. Vovk, A. Gammerman, and G. Shafer (2005) Algorithmic learning in a random world. Springer, New York, NY, USA. External Links: Document