When to Adapt: Multi-Signal Domain Shift Detection for Efficient Training-Free Adaptation in Open-Vocabulary Segmentation

Abstract

Robust and reliable perception is essential for autonomous robots operating in real-world environments, particularly in long-term missions where environmental conditions may change significantly over time. Although recent advances in Visual Foundation Models (VFMs) have improved open-vocabulary semantic segmentation, these models can still suffer from domain shift, which can significantly degrade performance if they are not adapted to the current environment. Training-free domain adaptation is a relevant paradigm for adaptation, consisting of adjusting the model online using lightweight adapters. Recent approaches apply this on a per-frame basis, which is impractical for deployments on resource-constrained robotic hardware. To tackle this, we propose a multi-signal domain shift detection method for training-free continual test-time adaptation (TF-CTTA) in open-vocabulary segmentation. Our method leverages temporal coherence across consecutive frames by monitoring and combining complementary aspects of domain shift (visual change, adapter mismatch, and semantic drift) to trigger adaptation only when needed. We validate our approach on a benchmark including indoor and outdoor environments and using real robotic data. We demonstrate that our approach maintains segmentation accuracy while substantially reducing adaptations, making training-free adaptation practical and feasible for long-term, real-world robotic deployments.

I Introduction

For autonomous robots operating in real-world environments, robust perception is essential, especially during long-term missions such as last-mile delivery, service robotics, or mobile assistance [1]. In these scenarios, a robot must navigate and understand scenes reliably across multiple and changing environments (e.g., alternating indoor and outdoor scenarios, new buildings, and varying illumination and weather). Such variability generates the well-known domain shift, which causes performance degradation if the model is not adapted to the current operational environment [2].

Recent advances in Visual Foundation Models (VFMs), particularly for open-vocabulary semantic segmentation [3], have significantly improved robustness and generalization in out-of-distribution settings. Despite these zero-shot capabilities, VFMs can still suffer notable performance degradation under domain shift. Changes in scene type, appearance, lighting, or layout can lead to a noticeable decrease in the model’s accuracy, making adaptation necessary during deployment [4].

Although domain adaptation for semantic segmentation has been widely explored, a core challenge still remains: how to adapt models online without introducing computational burden (e.g., operational cost or system complexity) while maintaining task performance. A potential solution is to rely on Continual Test-Time Adaptation (CTTA) approaches [5, 6], consisting of fine-tuning the model online after each inference step. However, this paradigm can be impractical in real-world robotic deployments because online fine-tuning via backpropagation introduces computational overhead, latency, and memory demands on resource-constrained devices. A promising alternative is training-free adaptation in which the model is adjusted at inference time without gradient-based updates. This paradigm has recently been introduced by SemLA [7], which proposes to dynamically fuse low-rank adapters into the model at test time, based on their proximity to the target domain.

In this paper, we analyze SemLA for real-world robotic deployments, finding important practical limitations. The method requires per-frame adapters retrieval and model updates, thus increasing energy consumption and introducing substantial runtime overhead, affecting real-time operation. Furthermore, evaluations have mostly focused on general-purpose datasets, leaving evaluation on sequential robot data largely unexplored.

To avoid per-frame overhead, we propose a multi-signal domain-shift detector for identifying significant visual and semantic changes in image streams to trigger adaptation only when needed. While previous approaches relying on a single indicator (prediction entropy [8] or feature statistics [9]) tend to saturate quickly and miss gradual changes, our approach detects shifts by monitoring three complementary aspects: (i) visual change, which captures abrupt scene transitions, (ii) adapter mismatch, which determines if the loaded adapters become inadequate for the current domain, and (iii) semantic drift, which identifies the appearance of novel semantic content. These signals are combined through a hierarchical fusion strategy that triggers adaptation only when multiple signals detect a domain shift.

Our framework, illustrated in Fig. 1, integrates domain shift detection into a training-free continual test-time adaptation (TF-CTTA) pipeline for open-vocabulary segmentation used by mobile robots operating across indoor and outdoor environments. Rather than swapping adapters every frame like SemLA [7], our solution exploits temporal coherence across consecutive frames, triggering adaptation only when a domain shift is detected. This reduces adaptation events by two orders of magnitude while maintaining segmentation accuracy, making training-free adaptation practical for long-term robotic deployment. In summary, our contributions are as follows:

• We analyze the performance of SemLA [7] on real-world robotic sequences, with a focus on indoor and alternating indoor-outdoor scenarios, identifying practical limitations of per-frame adaptation.
• We introduce a domain shift detector that combines three complementary signals to guide the adaptation, reducing the overhead caused by per-frame adaptation while maintaining semantic segmentation quality.
• We propose a continual test-time adaptation benchmark tailored to robotics, capturing alternating and temporally correlated domain shifts1. Using this, we provide a comprehensive evaluation comparing our method for TF-CTTA with relevant baselines.
• We demonstrate our approach on real-world sequences acquired with a robotic sensor across indoor and outdoor environments and benchmark inference on NVIDIA Jetson hardware.

II Related Work

Continual test-time adaptation (CTTA) is a challenging task that focuses on enabling models to adapt in real-time to dynamic, changing environments, without access to the source data. This task was initially introduced in the seminal work of [6], where a neural network is fine-tuned online with the supervision of a teacher, which is updated as an exponential moving average of the student’s weights. EcoTTA [10] improves the teacher-student framework using lightweight meta networks that are attached to the original frozen model, while ViDA [11] leverages visual domain adapters added to a source-pretrained model. Moving toward more realistic deployment scenarios, [12] studies multi-modality test-time adaptation for semantic segmentation, leveraging multiple sensing modalities to improve robustness under domain shifts and highlighting the relevance of methods tailored for robotic perception. Despite attractive, the aforementioned solutions present important limitations for real-world deployment, in terms of both inference speed and memory consumption, as they require backpropagation and weight sharing for EMA accumulations between two large architectures.

Training-free continual test-time adaptation (TF-CTTA) is a different paradigm that avoids expensive parameter updates by combining pretrained models relevant to the current domain, avoiding any additional training. [13] proposes a model merging technique, including parameter and buffer merging for multi-target domain adaptation tasks, applicable to any single-target domain adaptation model. Most closely related to our work, [7] introduces SemLA, a training-free test-time framework for open-vocabulary segmentation that leverages a library of LoRA-based adapters indexed with Contrastive Language-Image Pretraining (CLIP) embeddings [14]. The CLIP embeddings guide adapter selection, and subsequently, the most relevant adapters are merged based on their proximity to the target domain in the embedding space. Despite being intriguing, updating the model every frame strongly compromises real-time inference, remaining not practical for real-world robotic deployments.

Domain shift detection represents a valuable technique for reducing the model updates by identifying substantial changes in the robot’s perception stream that can cause performance degradation. In continual test-time adaptation, a natural solution is to leverage the discrepancy between the teacher-student predictions. The works of [5] and [15] compare the consistency loss between teacher and student accumulated with EMA with a variable threshold. [16] models domain shift directions via Concept Activation Vectors (CAVs) and incorporates domain shift and class shift controllers to prevent category drifting.

Other approaches detect domain shifts by performing uncertainty estimation using a model trained on the source domain. In ViDA [11], each sample is processed multiple times using dropout in the linear layers, and uncertainty is calculated as the variance across the outputs. Similarly, [17] computes a threshold using an EMA by aggregating the maximum confidence values for each pixel calculated using multiple versions of the same sample obtained with data augmentation. The approaches of [18] and [19] estimate uncertainty using the average class-wise confidence of a source model accumulated in a sliding window with a threshold to identify domain changes. Teacher-student and source-model uncertainty methods are impractical in our scenario, as deploying an additional segmentation model or running multiple forward passes for uncertainty estimation significantly compromises computational and memory efficiency.

Domain shifts can also be detected by monitoring features or prediction statistics directly, with no secondary model. [9] proposes a domain shift signal based on the cosine distance between mean feature representations of consecutive batches from the source model, triggering when the signal exceeds a multiple of its moving average. More recently, [8] targets resource-constrained embodied devices, monitoring prediction entropy via EMA and adapting only when entropy deviates past a threshold. Despite being more efficient, these methods are not robust enough for long-term sequences of robotic perceptions. Relying on a single indicator, they tend to saturate quickly as they cannot differentiate between types of domain shift. Our method, instead, combines multiple complementary signals for capturing different types of shifts and, relying on CLIP features, is specifically designed for the training-free domain adaptation framework of [7].

Positioning our work. Our work brings training-free continual test-time adaptation (TF-CTTA) to real-world robotic deployments. Considering the sequential image streams perceived by robots in the wild, we propose a multi-signal domain shift detector that leverages temporal coherence to trigger adaptation only when a domain shift is detected, improving efficiency while maintaining segmentation accuracy.

III Preliminaries

III-A Problem formulation

In this paper, we consider a robot performing open-vocabulary semantic segmentation from images. More formally, this task can be defined as Y^{*}=f_{\theta_{p}}(I), where a neural network f correlates an image I to a dense mask Y^{*}, where each pixel is assigned to an object category. The network’s parameters \theta_{p} are pre-trained using a labeled source dataset \mathcal{P}=(\mathbf{I}^{p},\mathbf{Y}^{p},\mathbf{O}^{p}) containing a list of images (\mathbf{I}^{p}) and their corresponding ground truth masks (\mathbf{Y}^{p}), where object categories come from \mathbf{O}^{p}. Inside the same mission, a robot needs to operate in a sequence of target environments \mathcal{T}=[\mathbf{E}_{t}\>|\>t=1,\ldots,T] that can be both indoor or outdoor. Inside each environment \mathbf{E}_{t}=\{\mathbf{I}_{t},\mathbf{O}_{t}\}, the robot is tasked to segment a sequence of RGB images \mathbf{I}_{t} using a specific set of object categories \mathbf{O}_{t}. Note that we are working in open-vocabulary settings, so the classes used for initializing the models and those required at deployment time may differ.

Training-free continual test-time adaptation (TF-CTTA) aims to adapt the neural network f_{\theta_{p}}, initially pre-trained on the source dataset \mathcal{P}, to a sequence of target datasets \mathcal{T}, thus obtaining a new model f_{\theta_{t}} with improved performance in the domains seen by the robot during deployment. This process is constrained by three main challenges: (i) ground-truth labels of the target data are unavailable, (ii) the model must evolve online using the stream of images in \mathcal{T}, and (iii) the adaptation must be training-free, meaning that fine-tuning using backpropagation is not allowed.

Fig. 2 presents a general overview of the approach we propose for enabling training-free continual test-time adaptation in mobile robots. The framework builds on the training-free approach of SemLA [7], summarized in Section III-B. The core of our contribution, detailed in Section IV-A, is a domain shift detection method based on CLIP features to identify when adaptation is needed. By preventing unnecessary adaptation, our method reduces the number of adaptations while preserving segmentation performance, making the approach more efficient and practical for long-term, real-world deployments.

III-B Training-free adaptation via adapters merging

Our domain shift detection method adapts the training-free adaptation proposed by SemLA [7] to online robotic deployment. SemLA encodes domain-specific knowledge in adapters and dynamically merges the most relevant ones based on proximity in CLIP’s [14] embedding space.

Let us consider a pre-trained model f_{\theta_{p}} and a set \mathcal{S}=\{\mathbf{S}_{s}\>|\>\mathbf{S}_{s}=\{\mathbf{I}_{s},\mathbf{Y}_{s},\mathbf{O}_{s}\}\}_{s=1}^{S} containing S source datasets. SemLA is a library of adapters formally described as

\mathcal{L}=\{(C_{s},\theta_{s})\>|\>s=1,\ldots,S\},

where C_{s} is the descriptor for the dataset \mathbf{S}_{s} (i.e., the average of the CLIP features from the images \mathbf{I}_{s}), while \theta_{s} denotes the weights of a low-rank adapter [20] trained on \mathbf{S}_{s}. At deployment time, for each environment \mathbf{E}_{t}\in\mathcal{T} and each image I_{i}\in\mathbf{I}_{t}, SemLA first computes the CLIP embedding F_{i} for the test image. Then, this embedding is used to retrieve the top-K most relevant adapters from the library \mathcal{L}, namely those adapters whose centroids are closest to the image’s embedding in terms of Euclidean distance. The distances between the image and the adapters descriptors are then normalized using a softmax function and used to scale the adapters weights, that are then merged and attached to the pre-trained model before inference on I_{i}.

IV Multi-Signal Domain Shift Detection

The main limitation of applying SemLA on mobile robots is its per-frame adaptation. We observed that this requirement significantly compromises computational efficiency, as the model needs to be dynamically updated every iteration. Since robotic perceptions operating in an environment are highly correlated, we overcome this limitation by proposing a multi-signal domain shift detection to trigger adaptation only when needed. Specifically, our method monitors three signals capturing complementary aspects of domain shift: the visual change (S_{1}), whether the loaded adapters still match the current domain (S_{2}), and the drift in the semantic objects appearing in the stream (S_{3}). The signals are fused hierarchically to trigger adaptation only when substantial evidence is detected across them, thus avoiding costly per-frame adaptation and making training-free adaptation practically deployable in robotic hardware.

IV-A Visual change signal (S_{1})

The visual change signal S_{1} captures global appearance changes by monitoring how CLIP embeddings of incoming frames diverge from the current domain representation. It answers the question: has the environment changed visually?

Consider the ordered stream of images contained in all the target environments of \mathcal{T}, defined as \mathbf{I}_{\mathcal{T}}, with length |\mathbf{I}_{\mathcal{T}}|=\sum_{t=1}^{T}|I_{t}| equal to the sum of the number of images of all environments \mathbf{E}_{t}\in\mathcal{T}.

At each frame i, we use CLIP to compute a feature embedding F_{i} for the image I_{i} and measure its proximity to a feature memory \mathcal{F}_{i-1}, which encodes the visual appearance of the image stream up to step i-1. The memory is updated using an exponential moving average (EMA) of the incoming features until the latest step i-1, formally \mathcal{F}_{i-1}=(1-\alpha)\mathcal{F}_{i-2}+\alpha F_{i-1}, where \alpha=10^{-2}. The proximity score between the memory \mathcal{F}_{i-1} and the current feature F_{i} is calculated as P_{i}=1-\text{cos}(\mathcal{F}_{i-1},F_{i}), where \text{cos}(\cdot,\cdot) is the cosine similarity.

The signal works by tracking the temporal evolution of P_{i} over time: an increasing trend suggests a domain change, while a decreasing trend indicates a stable domain. Let P_{1\ldots i} denote the proximity scores from the first to the current frame i. At first, we apply a low-pass filter to obtain a smoothed signal, formally \tilde{P}_{1\ldots i}=G_{(\sigma,w)}\ast P_{1\ldots i}, where (\ast) is the convolution operation and G_{(\sigma,w)} is a Gaussian kernel with standard deviation \sigma=6 and amplitude w. Then, at the current frame i, we compute the derivative of \tilde{P}_{w\ldots i} considering only the last i-w+1 frames. It is calculated as D_{i}=S_{(w)}\ast\tilde{P}_{w\ldots i}, where S_{(w)} is a Sobel kernel applied in window w. Since single derivatives are noisy, the trend is encoded in a memory variable \mathcal{D}, which accumulates derivatives using EMA (with \alpha=10^{-2}).

The trend of the signal is monitored using the derivative memory \mathcal{D}_{i-1} until the last frame i-1. When the domain is changing, two conditions must be satisfied. First, the memory must be positive (\mathcal{D}_{i-1}>0), indicating an increasing trend. Secondly, the current derivative must substantially exceed the memory, i.e., D_{i}>\rho\mathcal{D}_{i-1}, where \rho is a sensitivity hyperparameter (fixed to \rho=10). Instead of triggering the signal immediately, we wait until the derivative memory becomes negative (\mathcal{D}_{i-1}<0), indicating that the signal trend is decreasing. The visual change S_{1}\in\{0,1\} is then a binary signal.

IV-B Adapter mismatch signal (S_{2})

A visual change does not necessarily mean that the active adapters are inadequate: a scene may change in appearance while remaining within the adapters’ competence. Conversely, gradual shifts in layout or viewpoint can cause an adapter mismatch without any detectable visual change. S_{2} asks: do the active adapters still match the current domain?

To tackle this, we introduce an adapter mismatch signal that monitors how far the current domain has moved from the active adapters’ competence regions. This score is formally computed as \delta_{i}=\min_{k=1}^{K}1-\text{cos}(\bar{F}_{i},C_{k}), i.e., the lowest inverse cosine distance score between the representation of the current domain \bar{F} (the average of the CLIP features from the last w frames) and the top-K loaded adapters. The rationale is that if \delta_{i} is low, then there is at least one adapter well aligned with the current domain, while it increases only when all the adapters become unsuitable. We record the \delta_{\text{base}}=\delta_{i} at the first frame after each adaptation and we define S_{2}(i)=\tfrac{\delta_{i}}{\delta_{\mathrm{base}}}>\tau_{2}. When S_{2}(i)>\tau_{2}, the adapters no longer cover the current domain as well as they did right after being loaded. Normalizing by \delta_{\mathrm{base}} makes \tau_{2} independent of absolute embedding distances, so the same threshold works across environments.

IV-C Semantic drift dignal (S_{3})

S_{1} and S_{2} operate on CLIP image embeddings and ignore the model’s predictions. Two visually distinct scenes may share the same semantic composition, while two visually similar ones may differ in object content. S_{3} answers a different question: has the semantic content changed relative to what the active adapters were trained for?

For each frame i, we compute a semantic descriptor \mathbf{s}_{i}=\text{norm}_{2}(\mathbf{c}_{i}^{\top}\mathbf{E}), where \text{norm}_{2}(\cdot) is L_{2}-normalization, \mathbf{c}_{i}\in\mathbb{R}^{|\mathbf{O}|} is the vector of normalized pixel frequencies for each object class in \mathbf{O} for the prediction i, while \mathbf{E}\in\mathbb{R}^{|\mathbf{O}|\times d} is the matrix of pre-computed L_{2}-normalized CLIP text embeddings for the object categories \mathbf{O} (d is the embedding dimension). We maintain a memory bank \mathcal{M}=\{(\mathbf{s}^{m},\,\mathcal{O}^{m})\}_{m=1}^{M} storing the semantic context of M adaptation events (we fix \text{M}=50). Each entry consists of: (i) a semantic descriptor \mathbf{s}^{m} computed as the mean of the W_{s} semantic scores after adaptation, and (ii) a class set \mathcal{O}^{m}, the union of the k_{c} most frequent predicted classes from each frame over W_{s}. Aggregating a few frames after adaptation improves stability. We set W_{s}=k_{c}=5.

The signal combines two sub-signals. Embedding drift d_{\mathrm{emb}}(i) measures how far the current content is from relevant past contexts. Given \bar{\mathbf{s}}_{i} as the mean of the last W_{s} semantic scores,

d_{\mathrm{emb}}(i)=\frac{1}{|\mathcal{N}_{(\bar{\mathbf{s}}_{i},\mathcal{M})}|}\sum_{s^{j}\in\mathcal{N}_{(\bar{\mathbf{s}}_{i},\mathcal{M})}}\Big(1-\text{cos}\big(\bar{\mathbf{s}}_{i},\mathbf{s}^{j}\big)\Big),

where \mathcal{N}_{(\bar{\mathbf{s}}_{i},\mathcal{M})} denotes the J-nearest memory entries. Using J-nearest neighbors rather than the global minimum prevents saturation in long sequences.

Class novelty d_{\mathrm{nov}}(i) detects object categories absent from relevant past contexts. Let \mathcal{O}_{i} be the union of the top-k_{c} predicted classes across the current window:

d_{\mathrm{nov}}(i)=\frac{\left|\mathcal{O}_{i}\setminus\{\mathcal{O}^{j}|(s^{j},\mathcal{O}^{j})\in\mathcal{N}_{(\bar{\mathbf{s}}_{i},\mathcal{M})}\}\right|}{|\mathcal{O}_{i}|}.

The final semantic drift signal combines both: S_{3}(i)=\min\!\Big(1,\;\frac{1}{2}\big(d_{\mathrm{emb}}(i)+d_{\mathrm{nov}}(i)\big)\Big). When S_{3}>\tau_{3}, the semantic content has diverged enough from known contexts, indicating that the adapters are likely inadequate.

IV-D Hierarchical Fusion

The three aforementioned signals are combined in a hierarchical logic defined as:

\mathcal{H}(S_{1},S_{2},S_{3})=\begin{cases}(S_{2}>\tau_{2})\lor(S_{3}>\tau_{3})&\text{if }S_{1}=1\\[4.0pt]
(S_{2}>\tau_{2})\land(S_{3}>\tau_{3})&\text{if }S_{1}=0\end{cases}

The rationale behind this formulation is that S_{1} is binary and highly sensitive, as minor variations in the CLIP embeddings immediately activate the signal (e.g., a person approaching the camera). To tackle this, when S_{1} fires, a confirmation from at least another signal is required. Conversely, when it is silent (no abrupt visual change), both S_{2} and S_{3} must agree. Upon triggering, we retrieve the top-K adapters using the mean CLIP embeddings over the last w frames, which is more robust than SemLA’s single-frame descriptor.

V Experimental Evaluation

V-A Experimental setup and benchmark design

To evaluate our TF-CTTA approach in open-vocabulary semantic segmentation under domain shift, we design a benchmark that combines a variety of widely used indoor and outdoor datasets. The benchmark covers diverse scene types, label sets, and environmental changes (e.g., weather conditions) to better reflect long-term robotic perception under evolving domain shifts. In addition, we perform evaluation on real-world robotic data.

Datasets. Our benchmark is constructed from two indoor and two outdoor datasets.

We use Scannet [21] and Scannet++ [22], two large-scale indoor scene understanding datasets with diverse sensing modalities in multiple environment types, such as apartment, office, classroom, bedroom, etc. As in previous works [23, 24], each environment is treated as a separate domain, resulting in 707 and 906 unique domains for Scannet and Scannet++, respectively. For evaluation, we use the NYU40 label space [25] for Scannet and the top-100 classes for Scannet++.

For outdoor contexts, we use ACDC [26] and MUSES [27], two popular driving datasets where domains are represented by different weather and illumination conditions. More precisely, ACDC defines 5 domains: sun, night, rain, fog, and snow. MUSES extends this to 8 domains: 4 weather conditions (clear, rain, fog, snow) in 2 different light conditions (day, night). Both datasets are evaluated using their 19 standard object categories.

Implementation details. We use SED [3] as the base model for open-vocabulary semantic segmentation. Following [7], we attach a LoRA [20] adapter to each linear layer of the model (except the text encoder), training one adapter per domain (1626 total adapters, {\sim}10 MB each). Each adapter is trained until convergence. Since Scannet++ provides data from two hardware configurations (iPhone and DSLR camera), we train adapters using only iPhone data.

Evaluation protocol and metrics. To validate our approach, we propose a benchmark in which the model is tested through a sequence of domains, reflecting our continual test-time scenario described in Section III-A.

We define a sequence \mathcal{T} alternating 16 domains \mathbf{E}_{t} randomly chosen from the 4 indoor/outdoor datasets. We compose a new label space containing all the object categories needed by the 4 datasets (removing repetitions). This reflects a real-world deployment where the model is prompted with all the objects a robot can find while operating. Inside the sequence \mathcal{T}, we evaluate in a leave-one-out fashion: when running in \mathbf{E}_{t}, the corresponding adapter trained in \mathbf{E}_{t} is discarded, so the model is always tested on unseen scenes. This is a common setup in domain adaptation [23, 24] where each environment is considered a different domain. To ensure fairness, the evaluation is repeated over 50 randomly sampled sequences.

Segmentation accuracy is computed using mIoU, while efficiency is measured considering, for each domain, how often the adaptation is triggered (N{}_{\text{A}}). Minimizing this indicator directly improves inference speed as the adaptation cost is avoided.

Hyper-parameters. The parameters of the signals introduced in Section IV are set to w=21, \tau_{2}=2.0, \tau_{3}=0.15, and J=10, and remain fixed across all datasets.

Comparative methods. To evaluate our approach for domain shift detection in TF-CTTA, we compare with the following strategies for triggering adaptation:

• Zero-shot: the base model SED [3] without adaptation.
• Entropy [8]: adapts when the exponential moving average of the prediction entropy deviates from a post-adaptation baseline beyond a threshold.
• DSS [9]: triggers adaptation when a domain shift signal (cosine distance between mean feature representations of consecutive batches of images) exceeds a multiple of its moving average. We keep the threshold of the original paper but lower the batch size to 50 frames to make the method more reactive.
• Periodic: adaptation is triggered every \Delta_{f}=120 frames, tuned to match the average number of domain changes (N{}_{\text{A}}) found by our method.
• SemLA [7]: the adaptation process is triggered at each frame, with no mechanisms for domain shift detection.

V-B Results on training-free continual adaptation

Table I reports the results of our method and the baselines tested in a continual test-time manner. Overall, our method achieves segmentation accuracy (mIoU) close to the SemLA approach that, by performing a per-frame adaptation, represents a performance plateau. At the same time, our approach is substantially more efficient in terms of the number of triggered adaptations (N{}_{\text{A}}). With only a marginal drop of 1.4 mIoU points compared to SemLA, our method triggers adaptation only 0.8% of the time.

Compared to all these baselines, our method achieves the best performance in terms of segmentation accuracy in the majority of the splits (13 out of 16), except for \mathbf{E^{I}_{3}}, \mathbf{E^{I}_{10}}, and \mathbf{E^{I}_{15}}, in which Periodic leads by a margin of 0.3–0.5 mIoU. This demonstrates that our approach is capable of identifying domain shifts in a stream of robotic sensor data for effectively triggering adaptation. Notably, our method triggers roughly the same number of adaptations as Periodic but achieves higher segmentation accuracy, as it fires at actual domain transitions instead of fixed intervals. This confirms that when to adapt is as important as how often. It is also important to specify that this is an unrealistic baseline, as the frequency for adaptation has been set to mimic the efficiency reached by our method in this specific benchmark.

The results also demonstrate a substantial improvement of our method over the baselines for domain shift detection, which show poor performance on long-term sequences of robotic perception. Specifically, Entropy [8] triggers adaptation a few times at the beginning but saturates quickly. This is because entropy is not a reliable signal in a training-free adaptation setting: since there is no online fine-tuning, the model does not progressively gain confidence on the current data, and a single peak in entropy can prevent subsequent adaptation. In contrast, DSS [9] identifies clear domain shifts, such as transitions from indoor to outdoor environments and vice versa, but it completely misses smaller domain shifts occurring within the same environment type, which our method captures by combining multiple signals.

We perform an additional evaluation in out-of-distribution settings, namely when testing involves a camera model different from those used to train the adapters. This reflects real-world robotic deployment, where the robot’s camera often differs from the one used during training. We set up a library of adapters trained on Scannet and Scannet++ with iPhone data, with MUSES as the only outdoor dataset, and test them on Scannet++ using DSLR data (rectified images from a fisheye camera), and on ACDC for outdoor scenes. The results, shown in Table II, first confirm that training-free adaptation remains beneficial in out-of-distribution deployments (+4.4 mIoU over zero-shot). Furthermore, our method achieves performance comparable to per-frame adaptation, with a drop of only 0.4 mIoU, while triggering adaptation for a small fraction of the time (3.2 vs 960).

V-C Signal analysis

Fig. 4 shows the accuracy trade-off of individual signals and their combinations. Performance is evaluated as above, averaging over 50 sequences composed of 8 domains instead of 16. Individually, S_{1} and S_{3} detect shifts frequently (N_{A}=10.8 and 12.4, mIoU 44.9 and 45.3), while S_{2} barely triggers (N_{A}=1, mIoU 43.2), as adapter mismatch builds gradually and needs corroboration from the other signals. Pairwise combinations expose the role of each signal and how they complement one another. S_{1}+S_{2} is the most conservative combination (N_{A}=1, mIoU 42.9), and is dominated by S_{2} alone, without S_{3} the gradual semantic drift is never caught. S_{1}+S_{3} triggers more often (N_{A}=3.1) but obtains only a marginal improvement (mIoU 43.5). S_{2}+S_{3} achieves the highest mIoU (45.5) but over-triggers (N_{A}=14.4) since visual change is not considered. Our full system achieves the best accuracy-to-adaptation trade-off (N_{A}=4.1, mIoU 44.7), staying within 1.3 mIoU of per-frame adaptation (46.0) while triggering 3.5\times fewer adaptations than S_{2}+S_{3}. This confirms that the signals are individually sensitive but jointly selective, discarding spurious triggers while preserving detection coverage.

V-D Real-World Deployment

We profile the pipeline on an NVIDIA Jetson AGX Orin (32GB, NVMe SSD) mounted on our Spot robot, comparing per-frame adaptation (SemLA) against our triggered adaptation. Both are tested at two power levels (60W and 30W) and two model sizes to reflect different deployment scenarios. The image size is set to 224\times224 and 384\times384 for CLIP and SED, respectively. Table III reports the per-component latency: domain navigation computed using CLIP (Domain nav.), the inference with the segmentation model (Seg. model), and adaptation cost (Adapt. cost). The domain shift signals introduce no additional model inference, operating on embeddings and predictions already available in the pipeline. The results clearly show that adaptation is the bottleneck. While inference time can be reduced using smaller backbones, the adaptation cost is harder to cut as it is bounded by weight loading and merging. Even with our optimized adapter loading procedure compared to the original implementation [7], which directly swaps adapters without creating new layers for storing them, each adaptation still takes on average 550ms at full power and 700ms at low power, more than CLIP and SED inference combined (except for the configuration with large models and low power). As a result, per-frame adaptation cuts throughput by 1.6–3.3\times, i.e., from 2.6 to 1.1 FPS at full power and from 0.8 to 0.5 FPS at low power with the large model. Since this cost is largely architecture-independent, minimizing the number of adaptations is a necessary condition for real-time deployment: with our triggered adaptation and the small model, the pipeline reaches 4.2 FPS at full power (3.2\times faster compared to the per-frame).

We deploy our method on a real-world sequence captured using the Odin sensor [28] while navigating indoor/outdoor in our university campus. The adapter library is trained on the benchmark datasets, and all signal parameters remain identical to the benchmark experiments, except \tau_{2}=1.5 (lowered from 2.0), as the Odin image embeddings lie further from all adapters, raising \delta_{\mathrm{base}} so that the adapter mismatch signal never triggers at the default value. The camera captures at 10 FPS, but as the pipeline throughput on Jetson is 4.2 FPS (Table III), we subsample the stream by 2, processing 2735 frames to simulate real-time operation.

Fig. 3 shows S_{1} and the adaptation events over the full sequence. S_{1} alone triggers 42 times (red dashed), but the hierarchical fusion confirms only 12 (green), rejecting 71\% of S_{1} activations as false positives during stable periods. S_{3} (semantic drift) confirms most triggers, while S_{2} (adapter mismatch) confirms them when the active adapters become inadequate for the current scene. The confirmed adaptations correspond to visually distinct domain transitions: corridor changes, indoor-to-outdoor passages, and room type shifts (office, cinema, and laboratory). On the Jetson hardware, 12 adaptations add only 7s of overhead compared to 25 minutes under per-frame adaptation. This experiment validates that the detector generalizes to unseen environments and a different sensor with a single parameter change, complementing the quantitative out-of-distribution evaluation in Table II, whose test images are of the same type as the Odin ones (rectified from a fisheye camera).

VI Conclusions

In this paper, we propose a pipeline for training-free domain adaptation with a multi-signal domain shift detector to trigger adaptation only when needed, substantially improving efficiency (3.3\times FPS) while maintaining accuracy.

In future work, we plan to investigate new strategies for indexing and merging adapters to improve segmentation accuracy and further reduce adaptation cost.

References

1. [1] L. Kunze, N. Hawes, T. Duckett, M. Hanheide, and T. Krajník (2018) Artificial intelligence for long-term robot autonomy: a survey. IEEE ROBOT AUTOM LETT 3 (4), pp. 4023–4030. External Links: Document
2. [2] M. Schwonberg, J. Niemeijer, J. Termöhlen, J. P. schäfer, N. M. Schmidt, H. Gottschalk, and T. Fingscheidt (2023) Survey on unsupervised domain adaptation for semantic segmentation for visual perception in automated driving. IEEE Access 11 (), pp. 54296–54336.
3. [3] B. Xie, J. Cao, J. Xie, F. S. Khan, and Y. Pang (2024) SED: a simple encoder-decoder for open-vocabulary semantic segmentation. In Proc. of CVPR, pp. 3426–3436.
4. [4] M. Wortsman, G. Ilharco, J. W. Kim, M. Li, S. Kornblith, R. Roelofs, R. G. Lopes, H. Hajishirzi, A. Farhadi, H. Namkoong, and L. Schmidt (2022) Robust fine-tuning of zero-shot models. In Proc. of CVPR, pp. 7959–7971.
5. [5] H. Park, H. Park, J. Ko, and D. Min (2025) Hybrid-tta: continual test-time adaptation via dynamic domain shift detection. In Proc. of ICCV, pp. 2877–2886.
6. [6] Q. Wang, O. Fink, L. Van Gool, and D. Dai (2022) Continual test-time domain adaptation. In Proc. of CVPR, Vol. , pp. 7191–7201.
7. [7] R. Qorbani, G. Villani, T. Panagiotakopoulos, M. B. Colomer, L. Härenstam-Nielsen, M. Segu, P. L. Dovesi, J. Karlgren, D. Cremers, F. Tombari, and M. Poggi (2025) Semantic library adaptation: lora retrieval and fusion for open-vocabulary semantic segmentation. In Proc. of CVPR, pp. 9804–9815.
8. [8] X. Ma, Y. D. Kwon, and D. Ma (2025) EmbodiTTA: resource-efficient test-time adaptation for embodied visual systems.
9. [9] G. Chakrabarty, M. Sreenivas, and S. Biswas (2023) A simple signal for domain shift. In Proc. of ICCVW, Vol. , pp. 3569–3576.
10. [10] J. Song, J. Lee, I. S. Kweon, and S. Choi (2023) Ecotta: memory-efficient continual test-time adaptation via self-distilled regularization. In Proc. of CVPR, pp. 11920–11929.
11. [11] J. Liu, S. Yang, P. Jia, R. Zhang, M. Lu, Y. Guo, W. Xue, and S. Zhang (2024) ViDA: homeostatic visual domain adapter for continual test time adaptation. In Proc. of ICLR, B. Kim, Y. Yue, S. Chaudhuri, K. Fragkiadaki, M. Khan, and Y. Sun (Eds.), pp. 48396–48417.
12. [12] Y. Liu, H. Zhu, Y. Zhang, Y. Lei, and Y. Guo (2025) Multi-modality test-time adaptation for semantic segmentation in robotic perception. In Proc. of ICRA, Vol. , pp. 16963–16969. External Links: Document
13. [13] W. Li, H. Gao, M. Gao, B. Tian, R. Zhi, and H. Zhao (2024) Training-free model merging for multi-target domain adaptation. In Proc. of ECCV, pp. 419–438.
14. [14] A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, G. Krueger, and I. Sutskever (2021) Learning transferable visual models from natural language supervision. In Proc. of ICML, Vol. 139, pp. 8748–8763.
15. [15] M. B. Colomer, P. L. Dovesi, T. Panagiotakopoulos, J. F. Carvalho, L. Härenstam-Nielsen, H. Azizpour, H. Kjellström, D. Cremers, and M. Poggi (2023) To adapt or not to adapt? real-time adaptation for semantic segmentation. In Proc. of ICCV, pp. 16548–16559.
16. [16] Z. Shi, F. Lyu, Y. Liu, F. Shang, F. Hu, W. Feng, Z. Zhang, and L. Wang (2025) Controllable continual test-time adaptation. In Proc. of ICME, pp. 1–6.
17. [17] Y. Wang, J. Hong, A. Cheraghian, S. Rahman, D. Ahmedt-Aristizabal, L. Petersson, and M. Harandi (2024) Continual test-time domain adaptation via dynamic sample selection. In Proc. of WACV,
18. [18] T. Panagiotakopoulos, P. L. Dovesi, L. Härenstam-Nielsen, and M. Poggi (2022) Online domain adaptation for semantic segmentation in ever-changing conditions. In Proc. of ECCV,
19. [19] Y. Tian, K. Li, T. He, L. Wan, P. Heng, and W. Feng (2026) Dual domain-attribute learning framework with asynchronous adapters for continual test-time adaptation. IEEE Transactions on Image Processing 35 (), pp. 376–387.
20. [20] E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, W. Chen, et al. (2022) Lora: low-rank adaptation of large language models.. In Proc. of ICLR, Vol. 1, pp. 3.
21. [21] A. Dai, A. X. Chang, M. Savva, M. Halber, T. Funkhouser, and M. Niessner (2017) ScanNet: richly-annotated 3d reconstructions of indoor scenes. In Proc. of CVPR,
22. [22] C. Yeshwanth, Y. Liu, M. Nießner, and A. Dai (2023) ScanNet++: a high-fidelity dataset of 3d indoor scenes. In Proc. of ICCV, pp. 12–22.
23. [23] J. Frey, H. Blum, F. Milano, R. Siegwart, and C. Cadena (2022) Continual adaptation of semantic segmentation using complementary 2d-3d data representations. IEEE ROBOT AUTOM LETT.
24. [24] M. Antonazzi, L. Signorelli, M. Luperto, and N. Basilico (2026) Instance-guided unsupervised domain adaptation for robotic semantic segmentation. In Proc. of ICRA,
25. [25] N. Silberman, D. Hoiem, P. Kohli, and R. Fergus (2012) Indoor segmentation and support inference from rgbd images. In Proc. of ECCV, pp. 746–760.
26. [26] C. Sakaridis, D. Dai, and L. Van Gool (2021) ACDC: the adverse conditions dataset with correspondences for semantic driving scene understanding. In Proc. of ICCV, pp. 10765–10775.
27. [27] T. Brödermann, D. Bruggemann, C. Sakaridis, K. Ta, O. Liagouris, J. Corkill, and L. Van Gool (2024) Muses: the multi-sensor semantic perception dataset for driving under uncertainty. In Proc. of ECCV, pp. 21–38.
28. [28] M. TechManifold odin 1(Website) External Links: Link