MEKAWIRE
专业arXiv

SLLCP:用局部标签条件 conformal prediction 校准视觉语言模型,提升自动驾驶碰撞轨迹检测

Luís Marques, Rong Fang, Disha Kamale, Dmitry Berenson
30 秒速读

本文为 arXiv 预印本,未经同行评审。作者提出 SLLCP,在冻结的视觉语言模型之上增加后处理校准层,输出带有概率保证的安全预测集。在 CARLA 未见场景的 15321 帧测试中,SLLCP 将 Qwen 基底的碰撞轨迹检出率从 4.6% 提升至 89.6%,将 Cosmos 基底的检出率从 39.1% 提升至 88.4%,但代价是虚警率分别升至 51.8% 和 44.5%。

研究问题

能否在不微调视觉语言模型的情况下,为自动驾驶规划轨迹的安全监测提供概率保证,从而在用户指定的漏检率内检测碰撞轨迹?

问题

自动驾驶车辆在执行规划轨迹前需要判断是否安全,但经典碰撞检查依赖对周围动态物体运动的准确预测,而该预测本身受自车运动影响且难以精确。视觉语言模型具有常识推理能力,但原始安全预测不可靠、可能过度自信,无法提供低漏检率的保证。

既有方法

此前方法包括基于预测的碰撞检查和原始 VLM 安全判断;conformal prediction 中的 SplitCP 提供边际覆盖,Label-conditional CP 处理类别不平衡但阈值与输入无关,SLCP 通过局部加权近似输入条件覆盖,但其修正项只能保证边际覆盖,不能提供标签条件覆盖。

新方法

SLLCP 将标签条件 conformal prediction 与局部校准结合:在冻结 VLM 的隐藏状态上训练线性 logistic probe 作为非一致性得分,使用 UMAP 将高维嵌入投影到二维空间以计算局部权重,分别对安全和不安全标签拟合局部分位数,再用独立的校正集计算每个标签的校正阈值。最终预测集若包含 unsafe 则触发警报。该方法不更新 VLM 权重,可叠加在任何规划器之上。

结果

在 CARLA 仿真中,使用 SimLingo 规划器收集 40455 帧,按场景 ID 划分,评估集包含 15321 帧(2489 危险,12832 安全)。设定 α=0.1。原始 VLM 二分类在 Qwen 上仅检出 4.6% 的碰撞轨迹,Cosmos 为 39.1%。SLLCP 使 Qwen 的碰撞轨迹检出率达到 89.6%,Cosmos 为 88.4%,均落在名义覆盖率 90% 的 90% Beta-binomial 置信区间内;但虚警率分别升至 51.8% 和 44.5%,双标签歧义率分别为 37.6% 和 33.6%。消融实验显示:去掉 logistic probe 后虚警率超过 95.6%;去掉 UMAP 投影后覆盖率降至 84.3% 和 82.4%;去掉标签条件校正后覆盖率进一步降至 70.9% 和 66.7%。

局限

作者承认结果对 UMAP 投影和核超参数敏感;数据来自完整 episode 的视频帧,帧间相关破坏交换性假设;校准集与评估集存在场景偏移。此外,评估仅在 CARLA 仿真中进行,未在真实车辆或真实传感器数据上验证;虚警率约 50%,可能阻碍实际部署;需要访问 VLM 隐藏层嵌入,仅适用于开放权重模型;论文未报告算力成本。

产业影响

该方法可应用于自动驾驶公司的安全监测模块,作为规划器之外的独立碰撞风险过滤器,尤其适用于边缘场景和仿真测试。若能在真实数据和更低虚警率下验证,预期 1 至 3 年可能进入量产开发流程;前提是解决交换性假设、降低虚警率,并在多种 VLM 和真实路测数据上验证。

论文全文

Localized Conformal Safety Monitoring with Vision-Language Models for Autonomous Driving

Luís Marques, Rong Fang, Disha Kamale, Dmitry Berenson

本文依据 CC BY 4.0 许可发布,经署名转载,原文见 arXiv:2610.02765(PDF)。

Abstract

Monitoring planned driving trajectories requires accurately estimating the collision likelihood with actors whose motion is itself impacted by the ego motion. Existing classical approaches are often limited by the quality of their forecasting model. Vision-language models (VLMs) have shown promise in reasoning about the consequences of high-level actions, yet their approximate predictions are unsuitable for safety-critical applications such as autonomous driving. Conformal prediction (CP) has emerged as a data-driven framework for quantifying the uncertainty of black-box model predictions. We propose Split Label-Localized Conformal Prediction (SLLCP), a post-hoc calibration layer over frozen VLMs that transforms their unreliable predictions into probabilistically calibrated safety prediction sets. We consider how the ability to estimate safety can depend on the observed driving scene and introduce a localized procedure that upweights relevant past experience when calculating uncertainty thresholds. We provide label-conditional finite-sample distribution-free coverage under exchangeability. Evaluated over 15k CARLA trajectories from unseen scenarios, SLLCP correctly flags 89.6\% of collision-causing trajectories with a Qwen backbone and 88.4\% with a Cosmos backbone, while the base VLMs only flagged 4.6\% and 39.1\% of the collision-causing trajectories, respectively. These results indicate that local, label-conditional calibration can reduce missed unsafe trajectories.

Index Terms:

Conformal prediction, safety monitoring, uncertainty quantification, vision-language models, autonomous driving

I Introduction and Related Work

Before self-driving cars execute planned trajectories, they must assess whether the intended motion can be safely executed, considering the surrounding scene. This is nontrivial: pedestrians and other vehicles move in response to the ego vehicle. For example, a trajectory passing a pedestrian near the curb may be collision-free if the pedestrian waits, but unsafe if they step into the vehicle’s path. Further, the information available at planning time might miss occluded actors and not provide sufficient signal to enable accurate forecasting of the scene’s evolution. Even seemingly robust driving policies can demonstrate degraded performance when operating in unfamiliar scenarios — Fail2Drive reports substantial drops in closed-loop success under controlled scenario shifts [5].

We study the problem of designing a safety monitor that examines each candidate trajectory before execution, independently of the quality of the underlying planning module. That monitor should flag trajectories that would collide within a fixed time horizon, while avoiding unnecessary alarms hindering task progress. In general, a safety monitor might need to reason about the candidate motion and the scene jointly, while considering uncertainty in the true unknown outcome. While vision-language driving systems, such as SimLingo, incorporate scene understanding with action-taking [17], we aim to construct a monitor that can be applied post-hoc over any base planner. We seek a theoretically-informed way to probabilistically limit the number of missed unsafe trajectories.

Safety monitors. Driving pipelines often forecast the motion of nearby actors before checking a planned trajectory for collision [16]. Model-based safety checks can account for uncertain actor motion and dynamics [9], but their assurances depend on assumed scene knowledge. When an actor is missed entirely or its response to the ego vehicle is incorrectly predicted, using a collision check may fail. We thus investigate the complementary problem of assessing candidate trajectories using front-facing images, so that a vision-language model (VLM) can assess hard to model risks. A VLM’s raw safe/unsafe prediction provides no assurances of low-rate missed collisions, since VLM spatial reasoning can be unreliable [18]. Hence, we aim to augment VLMs’ common-sense reasoning with a post-hoc calibration layer providing probabilistic guarantees on their prediction performance.

Fig. 1: Uncertainty in VLM-based trajectory safety predictions (safe or unsafe) varies across driving scenes. SLLCP upweights calibration examples (bottom) near each test input (top) in the projected embedding space and calibrates each safety label separately to address class imbalance in the collected data (e.g., due to unsafe cases being rarer). Trajectory footprints are shown in blue, with yellow waypoints spaced at 0.5\,s intervals.

Conformal prediction in robotics. Conformal prediction (CP) transforms approximate predictions into prediction sets with finite-sample guarantees under exchangeability [19, 1]. In robotics, CP has calibrated: forecasts in dynamic scenes for social navigation [9, 3]; how dynamics uncertainty varies over a system’s state–action space [11]; action uncertainty for Lie-algebraic systems [12]; and motion uncertainty across contact modes [13].

In this extended abstract, we propose a monitor calibrating a base VLM safety prediction in a scene-adaptive manner. Our key contributions are:

  • We develop Split Label-Localized Conformal Prediction (SLLCP), a post-hoc method over a planner-agnostic generated trajectory image and a frozen VLM backbone, that provides probabilistic guarantees on a low probability of missing unsafe trajectories.
  • We empirically validate the tradeoff between missed unsafe trajectories and false alarms on edge-case CARLA scenarios, demonstrating significant improvements in the calibration of two open-weight VLMs, using the same hyperparameter set.

II Problem Statement

Let x_{t} denote the state of an autonomous driving agent at time t\in\mathbb{N}, whose planner generates candidate H-step-long trajectories \tau:=\{x_{t},x_{t+1},\ldots,x_{t+H-1}\}. We consider the problem of provably monitoring the safety of said finite-horizon trajectories given a black-box planner of unknown quality. We assume that executing \tau without replanning can produce one of two, initially unknown, outcomes Y: either a collision occurs within H steps (Y=\textsc{unsafe}), or it does not (Y=\textsc{safe}). While conventional collision-based monitors could be implemented by projecting \tau into the environment observed so far and performing geometric checks, the validity of their predictions is dependent on an accurate forecast of the motion of other dynamic entities in the scene, such as pedestrians and other vehicles. However, these forecasts are notoriously difficult to estimate accurately, even more so given that the ego motion can influence the behaviours of nearby actors. In this work, we consider vision-language models (VLMs) with fixed parameters as the base of our safety monitor, due to their common-sense reasoning capabilities. However, VLM-estimated safety likelihood \hat{p}(Y\mid X) is not directly suitable for safety-critical applications, since it might be overconfident. Our VLM-based monitor receives as input X, consisting of a front-facing RGB camera image of the scene overlaid with the proposed trajectory, as well as a prompt describing the safety monitoring task.

Problem. Given a user-specified acceptable failure rate \upalpha\in(0,1), we seek a set-valued monitor m:\mathcal{X}\to 2^{\{\textsc{safe},\textsc{unsafe}\}} whose probability of missing an unsafe trajectory is at most \upalpha. Formally, we require

\mathbb{P}(\textsc{unsafe}\notin\widehat{C}(X)\mid Y=\textsc{unsafe})\leq\upalpha,
(1)

and further want to minimize the false positive rate (FPR)

\operatorname{FPR}=\mathbb{P}(\textsc{unsafe}\in\widehat{C}(X)\mid Y=\textsc{safe}),
(2)

since high FPR implies unnecessary alarms which can hinder the agent’s task performance. To make this problem tractable, we make the standard assumption in the conformal prediction literature of having access to a held-out calibration set of labeled examples that are iid with the test data, i.e.:

Assumption 1

We have VLM inputs-outcome pairs D_{\mathrm{cal}}=\{(X_{i},Y_{i})\}_{i=1}^{n} that are iid with the test pair (X_{n+1},Y_{n+1}).

Following literature, D_{\mathrm{cal}} is uncorrupted.

III Background: Conformal Prediction

We briefly introduce relevant conformal prediction (CP) concepts below and refer to [19, 1] for deeper insight. Let X\in\mathcal{X} denote the available information at the time of prediction and Y\in\mathcal{Y}=\{\textsc{safe},\textsc{unsafe}\} the unknown true label. Given a fixed classifier, the estimated unsafe probability is \widehat{Y}(x)=\hat{p}(\textsc{unsafe}\mid x). The nonconformity score function r:[0,1]\times\mathcal{Y}\to\mathbb{R} measures disagreement between a model prediction and the true label, with higher values indicating higher prediction uncertainty. For binary classification, a conventional choice is r(\widehat{Y}(x),y)=1-\hat{p}(y\mid x).

Split conformal prediction (SplitCP). SplitCP is a computationally efficient CP variant that uses an exchangeable calibration set of labeled examples (Ass. 1) to provide probabilistic guarantees on test-time prediction uncertainty. Given D_{\mathrm{cal}}, we can evaluate the nonconformity score on each example to obtain a list of scores \{R_{i}\}_{i=1}^{n}, where R_{i}=r(\widehat{Y}(X_{i}),Y_{i}). Then the k=\lceil(n+1)(1-\upalpha)\rceilth smallest score in \{R_{i}\}_{i=1}^{n}\cup\{\infty\}, serves as a probabilistic threshold \widehat{q} for the test-time nonconformity score r(\widehat{Y}(X_{n+1}),Y_{n+1}). Using exchangeability [19], the following prediction set

\widehat{C}(x)=\{y\in\mathcal{Y}:r(\widehat{Y}(x),y)\leq\widehat{q}\},
(3)

is guaranteed to provide marginal coverage, i.e., satisfy \mathbb{P}\{Y\in\widehat{C}(X)\}\geq 1-\upalpha on average.

Label-conditional prediction. Marginal coverage alone does not account for imbalanced datasets. Since the guarantees are based on average uncertainty, underrepresented labels in D_{\mathrm{cal}} that present higher predictive uncertainty can be severely undercovered. For instance, D_{\mathrm{cal}} may contain many more safe than unsafe examples, and unsafe might be more difficult for the VLM to estimate. To address this issue, Label-conditional CP instead calibrates each class label separately [19]. Let I_{y}=\{i:Y_{i}=y\} be the calibration indices for class y, n_{y}=|I_{y}| the number of calibration examples y, and let \widehat{q}_{y} be the \lceil(n_{y}+1)(1-\upalpha)\rceilth smallest score in \{R_{i}:i\in I_{y}\}\cup\{\infty\}. Replacing the SplitCP label-shared threshold \widehat{q} in Eq. (3) by the per-label threshold \widehat{q}_{y} gives rise to the prediction set

\displaystyle\widehat{C}(x) \\ \displaystyle=\{y\in\mathcal{Y}:r(\widehat{Y}(x),y)\leq\widehat{q}_{y}\},

which satisfies label-conditional coverage, i.e., implies Eq. (1). This guarantee holds for finite-sized D_{\mathrm{cal}} in a distribution-free manner, i.e., without making any assumptions on the underlying score distribution. The prediction region \widehat{C} may now contain safe, unsafe, both labels, or the empty set. If \widehat{C} includes unsafe, the monitor should raise a safety alarm.

Split localized conformal prediction (SLCP). While each label-dependent threshold \widehat{q}_{y} is input-independent, prediction difficulty may vary across inputs x. For example, some driving scenes may be simpler to classify as unsafe than others, depending on the size, color, and proximity of nearby obstacles. Hence, we desire a threshold that can adapt to the model uncertainty at each input. Unfortunately, input-conditional coverage \mathbb{P}(Y\in\widehat{C}(X)\mid X=x)\geq 1-\upalpha has been shown to be impossible with finite data, without making additional distributional assumptions [8]. SLCP aims to approximate input-conditional coverage by fitting a local score quantile \widehat{q}_{\mathrm{loc}}(x) which up-weights calibration examples with inputs “near” the test-time input x. The key idea is that nearby inputs might share similar prediction difficulty, and thus their scores should be considered as more informative than the scores of faraway calibration examples. Since on its own \widehat{q}_{\mathrm{loc}}(x) does not provide any finite-sample guarantees, SLCP then adds a held-out correction \widehat{q}_{\mathrm{corr}} to obtain a final composed threshold \widehat{q}_{\mathrm{SLCP}}(x) [6, 7]. For exchangeability to hold, D_{\mathrm{cal}} must be initially split into two disjoint sets D_{\mathrm{cal}}^{\mathrm{local}} and D_{\mathrm{cal}}^{\mathrm{CP}} used for local fitting and held-out correction, respectively. Let I_{\mathrm{local}} and I_{\mathrm{CP}} index D_{\mathrm{cal}}^{\mathrm{local}} and D_{\mathrm{cal}}^{\mathrm{CP}}, respectively, and let the local distance between a calibration input X_{i} and a test input x be d_{i}(x)=d(\phi(X_{i}),\phi(x)) where \phi is a feature mapping. A kernel \kappa and bandwidth h>0 define the local weights

w_{i}(x)=\frac{\kappa(d_{i}(x)/h)}{\sum_{j\in I_{\mathrm{local}}}\kappa(d_{j}(x)/h)},
(5)

and \widehat{q}_{\mathrm{loc}}(x) is the weighted (1-\upalpha) quantile of D_{\mathrm{cal}}^{\mathrm{local}} scores. The held-out correction \widehat{q}_{\mathrm{corr}} is the SplitCP quantile of \{R_{j}-\widehat{q}_{\mathrm{loc}}(X_{j}):j\in I_{\mathrm{CP}}\}, giving the SLCP composite threshold of

\widehat{q}_{\mathrm{SLCP}}(x)=\widehat{q}_{\mathrm{loc}}(x)+\widehat{q}_{\mathrm{corr}}.
(6)

Despite possibly better empirical performance, SLCP only provides marginal coverage due to \widehat{q}_{\mathrm{corr}}’s construction [7].

IV Method: SLLCP

We propose Split Label-Localized Conformal Prediction (SLLCP), which builds on SLCP by combining it with label-conditional conformal prediction. Instead of using the raw VLM nonconformity score, we learn a nonconformity score to better inform the local quantile fit. Figure 2 summarizes our method. Following SLCP, we split D_{\mathrm{cal}}=D_{\mathrm{cal}}^{\mathrm{local}}\sqcup D_{\mathrm{cal}}^{\mathrm{CP}}, using D_{\mathrm{cal}}^{\mathrm{local}} to both learn r and fit \widehat{q}_{\mathrm{loc}}, while D_{\mathrm{cal}}^{\mathrm{CP}} now informs per-label conformal corrections \widehat{q}_{\mathrm{corr},y} for each label y\in\mathcal{Y}.

Fig. 2: SLLCP overview. Monitor inputs consist of a camera image with the proposed trajectory overlaid and a fixed safety prompt. Offline: using D_{\mathrm{cal}}^{\mathrm{local}}, we fit a logistic probe to obtain safety scores and a UMAP projection for localization, then define per-label local score quantiles. The disjoint D_{\mathrm{cal}}^{\mathrm{CP}} provides per-label corrections from score residuals. Online: given a test-time image–prompt pair, we combine its local quantiles with the fitted corrections to construct \widehat{C}(x). The monitor raises an alarm when \textsc{unsafe}\in\widehat{C}(x).

Probe score and local representation with D_{\mathrm{cal}}^{\mathrm{local}}. Let \mathbf{e}_{\mathrm{VLM}}(x) be the VLM’s hidden state before the Boolean answer token (safe, unsafe) and after the frozen language model’s final RMSNorm. A raw VLM-based score could be computed from the VLM’s Boolean-token probabilities. However, we have empirically observed that this score may not expose all safety-relevant information contained in the VLM hidden state. Therefore, we fit a linear logistic probe \widehat{Y}(x)=\sigma(\mathbf{w}^{\top}\mathbf{e}_{\mathrm{VLM}}(x)+b) to \mathbf{e}_{\mathrm{VLM}}(x) using the ground-truth labels in D_{\mathrm{cal}}^{\mathrm{local}}. This simple classifier provides an alternative way to estimate the unsafe probability \widehat{Y}(x), which we then use instead of Boolean token probabilities to compute the standard binary nonconformity score of Sec. III. This is cheap and does not require updating any VLM weights. For localization, since the raw VLM embedding is very high-dimensional, we first fit a two-dimensional UMAP [14] projection g to \{\mathbf{e}_{\mathrm{VLM}}(X_{i}):i\in I_{\mathrm{local}}\}, yielding a lower-dimensional latent representation of the input \mathbf{z}(x)=g(\mathbf{e}_{\mathrm{VLM}}(x)). This \mathbf{z}(x) is then used as the feature mapping \phi(x) in Eq. (5), leading to the local distance d(u,v)=\|u-v\|_{2} for projected vectors u:=\mathbf{z}(X_{i}),v:=\mathbf{z}(x). Both the classifier and UMAP fits are fixed independently of D_{\mathrm{cal}}^{\mathrm{CP}}.

Label-conditional local quantiles and correction. For each label y\in\mathcal{Y}, let I_{y}^{\mathrm{local}}=\{i\in I_{\mathrm{local}}:Y_{i}=y\}. We obtain the label-conditional thresholds \widehat{q}_{\mathrm{loc},y}(x) by computing the weighted (1-\upalpha) quantile of the scores R_{i}=r(\widehat{Y}(X_{i}),y) for i\in I_{y}^{\mathrm{local}}, using Eq. (5). Given correction examples j\in I_{\mathrm{CP}} with Y_{j}=y, their residual quantile gives \widehat{q}_{\mathrm{corr},y} similarly to SLCP. The \widehat{C} then is

\widehat{C}(x)=\{y\in\mathcal{Y}:r(\widehat{Y}(x),y)\leq\widehat{q}_{\mathrm{loc},y}(x)+\widehat{q}_{\mathrm{corr},y}\}.
(7)

Our safety monitor raises an alarm when \textsc{unsafe}\in\widehat{C}(x), to match the objective in Sec. II. Assuming within-label exchangeability, the proposed method provides label-conditional marginal coverage. While this assumption might not hold in our experiments, we demonstrate that our theory-driven method still provides adequate empirical coverage in practice.

V Experimental Results

We evaluate if the proposed safety monitor can detect collision-causing trajectories in the CARLA simulator [4]. We report unsafe-label coverage, the monitor’s false-positive rate on safe trajectories, and the fraction of produced prediction sets containing both labels. The latter indicates “ambiguity” in the monitor’s decision, and could be used to trigger a different recovery policy than unsafe-alarms. We use \upalpha=0.1 for all experiments. Coverage is a random variable that, under exchangeability, follows a Beta-binomial distribution given a fixed calibration set and a finite test set [10]. We report if the observed empirical coverage falls within the 90\% confidence interval of the nominal coverage 1-\upalpha, which would indicate if the applied procedure mitigates the nonexchangeable nature of the data.

We compare SLLCP with several relevant baselines: two uncalibrated raw-VLM rules and four component ablations. Raw VLM (binary) emits one safe/unsafe decision: the higher-scoring candidate label. Raw VLM (0.1/0.9) instead uses a \upalpha-aware scoring and a two-label prediction set. This would be the natural monitor if the raw VLM score were well-calibrated. The ablations remove, one at a time: locality (global, input-independent label-conditional CP); the linear probe (using original VLM score); the UMAP projection (raw embedding distances); the per-label correction (use label-agnostic uncertainty threshold). This analysis aims to isolate the impact of our choices of score, latent space representation, and label-aware correction.

We collect trajectories and front-facing images using the SimLingo planner [17]. A trajectory is deemed unsafe if tracking it for 2.5\,s with a PID controller causes a collision with a vehicle, pedestrian, or other object. In total, we obtain 40,455 frames from 90 episodes across 30 underlying scenario IDs in three Fail2Drive families [5]: HardBrake, PedestriansOnRoad, and Wall (seen in Figure 2). As base VLMs, we evaluated Qwen3-VL-8B-Instruct [2] and Cosmos-Reason2-8B [15]. We relied on open-weight VLMs so that we could access the hidden embeddings needed for the linear probe and for doing localized CP in embedding-space. Neither VLM was fine-tuned and both received the same fixed binary safety prompt and trajectory image.

We split the entire data into D_{\mathrm{cal}} and an evaluation set. All seeds of each scenario ID stay on the same side of this split. This prevents leakage of information from the evaluation set into D_{\mathrm{cal}} — each test scenario was not seen during the offline calibration phase. For some methods, D_{\mathrm{cal}} is further divided into D_{\mathrm{cal}}^{\mathrm{local}} and D_{\mathrm{cal}}^{\mathrm{CP}}. This results in |D_{\mathrm{cal}}^{\mathrm{local}}|= 19,528 frames (2,615 unsafe), |D_{\mathrm{cal}}^{\mathrm{CP}}|= 5,606 frames (856 unsafe), and an evaluation set with 15,321 frames (2,489 unsafe; 12,832 safe). SLLCP projected VLM embeddings to UMAP-2 (100 neighbors; Euclidean distance), then used a Laplace kernel \kappa(u)=\exp(-u), with per-label bandwidth h_{y}=0.4 times the label-specific median distance in D_{\mathrm{cal}}^{\mathrm{local}}. Hyperparameters were shared across the two VLMs, yet this results in model-specific conformal thresholds. Table I reports the three evaluation metrics over the 15,321 frames in the evaluation set.

TABLE I: Trajectory-safety results on unseen scenario IDs.
  • Raw VLM (binary)
    VLM
    Qwen
    UNSAFE coverage (%) \uparrow
    4.6
    Alarm FPR (%) \downarrow
    0.3
    Ambiguity (%) \downarrow
    0.0
  • —
    VLM
    Cosmos
    UNSAFE coverage (%) \uparrow
    39.1
    Alarm FPR (%) \downarrow
    1.9
    Ambiguity (%) \downarrow
    0.0
  • Raw VLM (0.1/0.9)
    VLM
    Qwen
    UNSAFE coverage (%) \uparrow
    20.1
    Alarm FPR (%) \downarrow
    2.1
    Ambiguity (%) \downarrow
    4.9
  • —
    VLM
    Cosmos
    UNSAFE coverage (%) \uparrow
    78.1
    Alarm FPR (%) \downarrow
    37.3
    Ambiguity (%) \downarrow
    43.9
  • Ablation w/o locality
    VLM
    Qwen
    UNSAFE coverage (%) \uparrow
    77.5
    Alarm FPR (%) \downarrow
    27.7
    Ambiguity (%) \downarrow
    18.6
  • —
    VLM
    Cosmos
    UNSAFE coverage (%) \uparrow
    80.5
    Alarm FPR (%) \downarrow
    30.4
    Ambiguity (%) \downarrow
    21.3
  • Ablation w/o logistic probe
    VLM
    Qwen
    UNSAFE coverage (%) \uparrow
    99.2
    Alarm FPR (%) \downarrow
    95.6
    Ambiguity (%) \downarrow
    89.4
  • —
    VLM
    Cosmos
    UNSAFE coverage (%) \uparrow
    99.6
    Alarm FPR (%) \downarrow
    99.0
    Ambiguity (%) \downarrow
    89.3
  • Ablation w/o projection
    VLM
    Qwen
    UNSAFE coverage (%) \uparrow
    84.3
    Alarm FPR (%) \downarrow
    41.6
    Ambiguity (%) \downarrow
    28.3
  • —
    VLM
    Cosmos
    UNSAFE coverage (%) \uparrow
    82.4
    Alarm FPR (%) \downarrow
    37.9
    Ambiguity (%) \downarrow
    26.8
  • Ablation w/o label-conditional CP
    VLM
    Qwen
    UNSAFE coverage (%) \uparrow
    70.9
    Alarm FPR (%) \downarrow
    15.4
    Ambiguity (%) \downarrow
    6.1
  • —
    VLM
    Cosmos
    UNSAFE coverage (%) \uparrow
    66.7
    Alarm FPR (%) \downarrow
    11.9
    Ambiguity (%) \downarrow
    5.2
  • SLLCP (ours)
    VLM
    Qwen
    UNSAFE coverage (%) \uparrow
    89.6
    Alarm FPR (%) \downarrow
    51.8
    Ambiguity (%) \downarrow
    37.6
  • —
    VLM
    Cosmos
    UNSAFE coverage (%) \uparrow
    88.4
    Alarm FPR (%) \downarrow
    44.5
    Ambiguity (%) \downarrow
    33.6

Raw VLM (binary) alarms included unsafe on only 4.6\% of Qwen’s and 39.1\% of Cosmos’s unsafe frames, failing to meet the user-set safety requirements. Similarly, the safety-biased raw rule also remained well below the nominal 90\% target. Our approach, SLLCP, increased unsafe-label coverage to 89.6\% and 88.4\% staying within the finite-sample Beta-binomial confidence interval of the user-set requirement. However, the alarm FPRs rose to 51.8\% and 44.5\% and both-label ambiguity to 37.6\% and 33.6\%, respectively. This indicates an efficiency–coverage tradeoff. In safety-critical applications, where consequences can be severe, one may prefer to reduce missed unsafe trajectories at the cost of more false alarms.

The ablations motivate our design decisions. Without the logistic probe, the local method includes nearly all unsafe frames but raises false alarms on at least 95.6\% of safe frames, making it impractical for real-world applications. Without UMAP, unsafe coverage falls to 84.3\% and 82.4\%. Using a label-independent threshold lowers coverage further to 70.9\% and 66.7\%. These comparisons suggest that the score, local representation, and label-specific correction each impact the coverage–false-alarm tradeoff.

Limitations. Results are sensitive to the UMAP projection and kernel hyperparameters. The original dataset includes video frames collected along full-episode SimLingo trajectories. Hence, frames are correlated, breaking exchangeability. The scenario shift from D_{\mathrm{cal}} also impacts empirical performance.

VI Conclusion

On unseen Fail2Drive scenarios in CARLA, SLLCP achieved 89.6\% and 88.4\% unsafe-label coverage for Qwen and Cosmos, improvements of 85.0 and 49.3 percentage points, respectively, over the raw VLM performance. In this extended abstract, we considered trajectory monitoring as a two-label prediction-set problem. We combined a learned score, from frozen VLM representations, with local calibration in a projected embedding space and label-conditional calibration. Compared with the no-probe ablation, our approach reduces the alarm FPR by 43.8 and 54.5 percentage points for Qwen and Cosmos, respectively, while keeping unsafe-label coverage near 90\%. SLLCP provides a planner-agnostic way to balance safety monitoring and false alarms without requiring any VLM fine-tuning. Future work could consider applications to semantic safety violations, such as failing to yield or obey traffic signals. Yet, obtaining reliable ground-truth labels in these settings is nontrivial.

References

  1. [1] A. N. Angelopoulos and S. Bates (2023) Conformal prediction: a gentle introduction. Found. Trends Mach. Learn.. External Links: Document
  2. [2] S. Bai, Y. Cai, R. Chen, et al. (2025) Qwen3-VL technical report. Note: arXiv preprint arXiv:2511.21631
  3. [3] A. Dixit, L. Lindemann, S. X. Wei, M. Cleaveland, G. J. Pappas, and J. W. Burdick (2023) Adaptive conformal prediction for motion planning among dynamic agents. In L4DC,
  4. [4] A. Dosovitskiy, G. Ros, F. Codevilla, A. Lopez, and V. Koltun (2017) CARLA: an open urban driving simulator. In CoRL,
  5. [5] S. Gerstenecker, A. Geiger, and K. Renz (2026) Fail2Drive: benchmarking closed-loop driving generalization. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS),
  6. [6] L. Guan (2023) Localized conformal prediction: a generalized inference framework for conformal prediction. Biometrika. External Links: Document
  7. [7] X. Han, Z. Tang, J. Ghosh, and Q. Liu (2022) Split localized conformal prediction. Note: arXiv preprint arXiv:2206.13092
  8. [8] J. Lei and L. Wasserman (2014) Distribution-free prediction bands for non-parametric regression. J. R. Stat. Soc. Ser. B. External Links: Document
  9. [9] L. Lindemann, M. Cleaveland, G. Shim, and G. J. Pappas (2023) Safe planning in dynamic environments using conformal prediction. IEEE Robot. Autom. Lett.. External Links: Document
  10. [10] P. C. Marques F. (2025) Universal distribution of the empirical coverage in split conformal prediction. Statist. Probab. Lett.. External Links: Document
  11. [11] L. Marques and D. Berenson (2026) Quantifying aleatoric and epistemic dynamics uncertainty via local conformal calibration. In WAFR, External Links: Document
  12. [12] L. Marques, M. Ghaffari, and D. Berenson (2026) Lies we can trust: quantifying action uncertainty with inaccurate stochastic dynamics through conformalized nonholonomic Lie groups. IEEE Robot. Autom. Lett.. External Links: Document
  13. [13] L. Marques, K. Popov, and D. Berenson (2026) Particle-based conformal prediction for contact-aware uncertainty calibration in stratified configuration spaces. In COPA,
  14. [14] L. McInnes, J. Healy, N. Saul, and L. Großberger (2018) UMAP: uniform manifold approximation and projection. JOSS. External Links: Document
  15. [15] NVIDIA (2026) Cosmos-Reason2. Note: NVIDIA Cosmos documentationAccessed September 24, 2026
  16. [16] B. Paden, M. Čáp, S. Z. Yong, D. Yershov, and E. Frazzoli (2016) A survey of motion planning and control techniques for self-driving urban vehicles. IEEE Trans. Intell. Veh.. External Links: Document
  17. [17] K. Renz, L. Chen, E. Arani, and O. Sinavski (2025) SimLingo: vision-only closed-loop autonomous driving with language-action alignment. In CVPR, External Links: Document
  18. [18] X. Tian, J. Gu, B. Li, Y. Liu, Y. Wang, Z. Zhao, K. Zhan, P. Jia, X. Lang, and H. Zhao (2025) DriveVLM: the convergence of autonomous driving and large vision-language models. In CoRL,
  19. [19] V. Vovk, A. Gammerman, and G. Shafer (2005) Algorithmic learning in a random world. Springer, New York, NY, USA. External Links: Document