MEKAWIRE
ExpertarXiv

Smart Fusion for Adaptive Visual-IMU Weighting in cuVSLAM Sparse Bundle Adjustment

Wonju Lee, Dara Molloy, Abhilash Nand Kumar, Onias Castelo, Petar Trslic, David Adley, Conor Ferguson, Hesam Rabeti
From the abstract

In cuVSLAM, visual and inertial measurement unit (IMU) residuals are fused in sparse bundle adjustment (SBA) with a weighting fixed by pre-calibrated noise covariances, leaving no mechanism to adapt the visual-inertial balance at runtime. We introduce a scalar α≥1 that upscales the visual information matrix relative to the IMU preintegration constraint.

From the abstract. Our summary is in progress.

Full paper

Smart Fusion for Adaptive Visual-IMU Weighting in cuVSLAM Sparse Bundle Adjustment

Wonju Lee, Dara Molloy, Abhilash Nand Kumar, Onias Castelo, Petar Trslic, David Adley, Conor Ferguson, Hesam Rabeti

Published under CC BY 4.0. Reproduced with attribution; original at arXiv:2610.20226 (PDF).

Abstract

In cuVSLAM, visual and inertial measurement unit (IMU) residuals are fused in sparse bundle adjustment (SBA) with a weighting fixed by pre-calibrated noise covariances, leaving no mechanism to adapt the visual-inertial balance at runtime. We introduce a scalar \alpha\geq 1 that upscales the visual information matrix relative to the IMU preintegration constraint. Across the candidate set \{1.0,2.0,5.0,10.0,20.0\}, the resulting trajectories form two empirical performance regimes. The imu-trust regime contains \alpha\in\{1,2\} near the calibrated baseline, while the vision-trust regime contains \alpha\in\{10,20\}. The intermediate value \alpha=5 is treated as a transition case. Differences within each regime are small relative to those observed between the regimes. This structure motivates reducing \alpha-selection to a binary decision. We train a lightweight Visual Quality Estimator (VQE), implemented as a multilayer perceptron with 914 trainable parameters. It maps a 10-dimensional visual quality feature vector assembled from statistics already maintained by the tracker to a per-SBA regime prediction. The proposed method reduces mean RMSE APE by 27.7\% and by 14.9\% under leave-one-out cross-validation (LOOCV) on EuRoC MAV and TUM-VI Room, respectively.

Fig. 1: Feature quality contrast on EuRoC Machine Hall (MH_01 vs. MH_05). Corners are detected with GFTT on the raw image. The fraction of green points is chosen to match the feature survival rate recorded in the SBA log under the greedy-reference assignment. The coloring represents a per-frame aggregate and should not be interpreted as a per-point tracking outcome. The greedy reference assigns \alpha\!=\!10 (vision-trust) to the MH_01 segment and \alpha\!=\!2 (imu-trust) to the MH_05 segment.

I INTRODUCTION

Tightly-coupled visual-inertial odometry (VIO) fuses reprojection residuals and IMU preintegration constraints in a joint optimization [1, 2, 3]. The relative weight assigned to the two residual types affects the influence of each sensor modality on the optimized trajectory. Increasing the visual weight under degraded observations can increase the effect of visual measurement errors. Conversely, a low visual weight may underuse geometric constraints when the observations are reliable. An appropriate balance can depend on scene and motion conditions, yet most systems determine it during calibration and retain it during operation.

In the original cuVSLAM [1] formulation, visual and IMU residuals are combined under a fixed weighting derived from pre-calibrated noise covariances (corresponding to \alpha\!=\!1.0), with no mechanism to adjust this balance at runtime. This fixed weighting is appropriate when the calibrated noise model remains representative of the measurements. It may be less appropriate when scene conditions such as sparse texture or large depth variation increase reprojection error, or when motion blur and illumination changes temporarily degrade tracking. Under these conditions, retaining the visual weight near its nominal calibrated level can limit the influence of degraded measurements, but the original formulation does not perform this adaptation. Fig. 1 illustrates contrasting visual-quality conditions and the corresponding regimes selected by the greedy reference.

We present Smart Fusion, which introduces a global fusion weight \alpha to scale the visual information matrix relative to the IMU constraint. Smart Fusion uses a lightweight Visual Quality Estimator (VQE), implemented as an MLP, to map tracker-derived visual quality features to a regime prediction and set \alpha before each SBA call. The design is motivated by the empirical grouping observed over \mathcal{A}=\{1.0,2.0,5.0,10.0,20.0\}. The imu-trust regime contains \alpha\in\{1,2\} and the vision-trust regime contains \alpha\in\{10,20\}. On many evaluated sequences, within-regime differences are small relative to between-regime differences. On EuRoC MH_01, the APE difference between the two regimes is 47\%, while the tested values within each regime produce similar APE. This observation motivates the binary formulation used in Smart Fusion (Fig. 3). The VQE operates on a 10-dimensional visual quality feature vector assembled from statistics already maintained by the tracker. Training labels are generated by a sequential non-causal greedy reference that evaluates final full-trajectory APE for each sequence segment.

We make two contributions. First, we characterize the two empirical performance regimes observed when varying \alpha in cuVSLAM SBA and formulate their selection as binary classification. Second, we present Smart Fusion, a 914-parameter MLP that predicts a regime at each SBA call from tracker-internal visual quality features. The model is trained on sequential greedy-reference labels derived from trajectory APE. Smart Fusion reduces mean RMSE APE by 27.7\% on EuRoC MAV and by 14.9\% on TUM-VI Room, both under leave-one-out cross-validation.

II RELATED WORK

Optimization-based VIO

Tightly-coupled sliding-window optimizers such as VINS-Mono [2], ORB-SLAM3 [3], OKVIS [12], and cuVSLAM [1] minimize a joint cost over reprojection and IMU preintegration residuals. The relative weighting between visual and inertial terms is typically determined by pre-calibrated noise covariances and held fixed during operation.

Noise model and weight adaptation

Several works adjust sensor noise models online. Kelly and Sukhatme [10] perform online visual-inertial localization, mapping, and sensor-to-sensor extrinsic self-calibration, and Cong et al. [13] adapt per-feature visual covariances for UAV VIO. VINS-Mah [14] and Sibley et al. [11] propose sliding-window estimators with robust cost functions to down-weight individual high-residual observations. These approaches operate at the per-feature or per-residual level. Smart Fusion instead adjusts the global visual-to-IMU ratio at each SBA call and therefore provides a complementary form of adaptation.

Learned components in VIO

Nascivera et al. [15] adapt front-end hyperparameters via an image-conditioned learned policy, and AdaGate [9] uses an oracle-supervised MLP to adapt MSCKF feature-gating thresholds from filter state. Smart Fusion follows a similar oracle-supervised approach but operates on the optimization-based backend and uses a binary representation of the observed \alpha regimes. Methods such as DROID-SLAM [16] incorporate learned recurrent update operators into dense bundle adjustment. This differs in scope from the lightweight backend parameter adaptation considered here.

Fig. 2: cuVSLAM front-end pipeline with Smart Fusion. The 2D Frontend tracks features and selects keyframes. The 3D Module initializes landmarks via stereo triangulation using both 2D tracks and IMU state. SBA runs asynchronously minimizing the joint visual-inertial cost of (1). The Visual Quality Estimator (VQE) within Smart Fusion predicts \alpha from 10-dimensional visual quality features before each SBA call, replacing the fixed default \alpha\!=\!1.0.

III Smart Fusion

Fig. 2 shows the cuVSLAM front-end and the point at which Smart Fusion is applied. SBA is triggered asynchronously at each new keyframe. Immediately before each optimization call, Smart Fusion assembles a visual quality feature vector from tracker-internal statistics and queries the VQE to set \alpha for that call. The cuVSLAM solver structure is unchanged. Smart Fusion adds a scalar multiplier to the visual information matrix and one forward pass through a 914-parameter network.

III-A Alpha in cuVSLAM Bundle Adjustment

cuVSLAM minimizes the following cost over a sliding window of states and landmarks.

(\hat{\mathbf{S}}_{0:K},\{\hat{\mathbf{p}}_{j}\})=\arg\min_{\mathbf{S}_{0:K},\{\mathbf{p}_{j}\}}\left[\sum_{i}\bigl\|\mathbf{r}^{\mathrm{imu}}_{i}\bigr\|^{2}_{\boldsymbol{\Omega}_{\mathrm{IMU}}}+\alpha\sum_{i,j,k}\bigl\|\mathbf{r}^{\mathrm{repr}}_{ijk}\bigr\|^{2}_{\boldsymbol{\Omega}_{\mathrm{vis}}}\right]
(1)

where \|\mathbf{r}\|^{2}_{\boldsymbol{\Omega}}=\mathbf{r}^{\top}\boldsymbol{\Omega}\mathbf{r} is the squared Mahalanobis norm, \mathbf{S}_{0:K}=\{\mathbf{T}^{wb}_{i},\mathbf{v}_{i},\mathbf{b}_{i}\} is the sliding-window state (world-from-body poses, velocities, IMU biases), \mathbf{p}_{j}\in\mathbb{R}^{3} is the world position of landmark j, \mathbf{r}^{\mathrm{imu}}_{i} is the IMU preintegration residual between consecutive states [4] with information matrix \boldsymbol{\Omega}_{\mathrm{IMU}}, \mathbf{r}^{\mathrm{repr}}_{ijk}=\pi(\mathbf{T}^{c_{k}b}\mathbf{T}^{bw}_{i}\mathbf{p}_{j})-\mathbf{o}_{ijk} is the reprojection residual of landmark j in camera k at keyframe i (\mathbf{T}^{bw}_{i}\!\in\!SE(3) body-from-world pose, \mathbf{T}^{c_{k}b} fixed camera-k-from-body extrinsic, \mathbf{o}_{ijk} observed keypoint), and \boldsymbol{\Omega}_{\mathrm{vis}}=\sigma^{-2}\mathbf{I} is the fixed pixel-noise precision (\sigma\!\approx\!1\,\text{px}).

The IMU term aggregates raw measurements between keyframes into a (\Delta\mathbf{R},\Delta\mathbf{v},\Delta\mathbf{p}) preintegrated constraint [4] with a 9\!\times\!9 information matrix. \alpha\!=\!1.0 weights the two terms according to their nominal noise covariances. Increasing \alpha above 1.0 raises the relative influence of visual observations. The candidate set explored in this work is \mathcal{A}=\{1.0,2.0,5.0,10.0,20.0\}, which restricts \alpha to values at or above the nominal setting and therefore never down-weights the visual term relative to the baseline. Values below \alpha\!=\!1.0 were not explored in this work. The greedy reference produces similar full-trajectory APE for \alpha\!=\!1.0 and \alpha\!=\!2.0 across the evaluated sequences. We therefore treat these values as members of the same performance regime. The behavior of values below \alpha\!=\!1.0 is not examined in this work.

The preferred regime depends on the quality of the visual observations. When the features are abundant, well distributed, and stable, the vision-trust regime assigns greater influence to the available geometric constraints. When visual quality degrades through sparse observations, high reprojection error, or low feature survival, a large \alpha can increase the effect of visual measurement errors and shift the optimized trajectory away from the IMU-predicted state. Under such conditions, the imu-trust regime may be preferable.

Fig. 3: Conceptual APE landscape as a function of \alpha and visual quality. The imu-trust regime at low \alpha and the vision-trust regime at high \alpha are separated by a transition region near \alpha\!=\!5. Under good-quality conditions the fixed baseline \alpha\!=\!1.0 (\times) sits in the imu-trust regime while the greedy-reference selection lies in the vision-trust regime. Smart Fusion aims to select the appropriate regime (\bigstar) based on the observed visual quality.

III-B Performance Regime Structure

On the EuRoC and TUM-VI sequences tested, varying \alpha over the candidate set \mathcal{A}=\{1.0,2.0,5.0,10.0,20.0\} produces outputs that cluster into two groups.

  • imu-trust (\alpha\in\{1.0,2.0\}). The visual weight is at most twice the calibrated baseline, so the IMU constraint retains greater relative influence. The two values produce similar trajectories on most evaluated sequences.
  • vision-trust (\alpha\in\{10.0,20.0\}). The visual reprojection cost receives a higher relative weight. The two values produce similar trajectories within this group and differ from the imu-trust group on many evaluated sequences. \alpha\!=\!5.0 falls in the transition region between the two regimes and is treated separately (see Sec. III-C).

The observed grouping is consistent with the change in relative influence between the visual and inertial terms. In the imu-trust range, the IMU constraint has greater relative influence, and varying \alpha within \{1.0,2.0\} produces similar converged trajectories. In the vision-trust range, the visual cost has greater relative influence, and varying \alpha within \{10.0,20.0\} also produces similar trajectories. The tested intermediate candidate \alpha\!=\!5.0 exhibits transition behavior and is treated separately from the two groups.

On EuRoC MH_01, both \alpha\in\{1.0,2.0\} yield 0.138\,\text{m} APE, while both values in \{10.0,20.0\} yield approximately 0.073\,\text{m}. This corresponds to a 47\% difference between the two regimes and a small difference within each regime. Similar grouping is observed on most evaluated sequences, which motivates the binary prediction used in this work. Fig. 3 illustrates this structure conceptually.

For the output \alpha at inference, we use \alpha\!=\!2.0 for imu-trust and \alpha\!=\!10.0 for vision-trust. These values are selected as representative outputs and are the candidates nearest the transition region within their respective groups.

III-C Sequential Greedy Label Generation

We generate per-segment binary labels using a sequential non-causal greedy reference based on final full-trajectory APE. For each training sequence,

  1. Divide into S\!=\!10 equal-length segments by camera frame count.
  2. Initialize the weight of every segment to the released default \alpha\!=\!1.0.
  3. For segment k, retain the values already selected for segments 1,\ldots,k-1, evaluate every candidate \alpha\in\mathcal{A} for segment k over N\!=\!3 independent tracker runs, and keep future segments k+1,\ldots,S at \alpha\!=\!1.0. Record the mean final full-trajectory APE for each candidate. Averaging over three runs reduces the effect of CUDA non-determinism when candidates produce similar APE values.
  4. Fix the candidate \alpha_{k}^{*} with the lowest mean final APE and continue to segment k\!+\!1. Map the selected value to a binary label. The label is imu-trust if \alpha_{k}^{*}\in\{1.0,2.0\} and vision-trust if \alpha_{k}^{*}\in\{10.0,20.0\}. Segments where \alpha_{k}^{*}\!=\!5.0 (the transition region) are excluded from the training set as uncertain.

This forward sequential procedure accounts for interactions with previously selected segments and for the downstream effect of the current choice under the default future schedule. In a deterministic evaluation, retaining \alpha\!=\!1.0 in the candidate set allows each step to preserve the current partial schedule. Averaging over three CUDA runs reduces, but does not eliminate, small deviations caused by non-determinism. It is nevertheless not guaranteed to find the globally optimal assignment. Exhaustive joint optimization would require evaluating all |\mathcal{A}|^{S}=5^{10}\approx 9.8\times 10^{6} combinations per sequence. We therefore use it as a non-causal greedy reference for label generation, rather than as a strict upper bound.

At inference, imu-trust maps to \alpha\!=\!2.0 and vision-trust maps to \alpha\!=\!10.0. On EuRoC, among all SBA calls with a definite binary label (approximately 9{,}500 of 13{,}700 total), the greedy reference assigns imu-trust to approximately 76\% and vision-trust to 24\%. A separate 31\% of all SBA calls fall in the transition region (\alpha_{k}^{*}\!=\!5.0) and are excluded from training. At inference, the MLP always outputs one of the two binary classes. Calls from segments for which \alpha\!=\!5.0 would be near-optimal are therefore assigned according to the learned decision boundary, potentially leading to a suboptimal regime selection.

III-D Visual Quality Features

At each SBA call we assemble a 10-dimensional feature vector \mathbf{x}\in\mathbb{R}^{10} from statistics already available in the tracker as

\mathbf{x}=[\underbrace{\ln n_{\mathrm{obs}},\;\ln n_{\mathrm{lm}}}_{\text{quantity}},\;\underbrace{\tilde{\sigma}_{u},\;\tilde{\sigma}_{v},\;\ln\Delta t_{\mathrm{imu}}}_{\text{coverage \& timing}},\;\underbrace{\ln\bar{d},\;\mathrm{CV}_{d}}_{\text{geometry}},\;\underbrace{r_{\mathrm{surv}},\;\ln\bar{e},\;\ln s_{e}}_{\text{stability}}]^{\!\top}
(2)

All features are drawn from quantities already maintained by the tracker, adding no extra sensing. The added computation consists of one MLP forward pass per SBA call. All features reflect tracker state accumulated prior to the current SBA call. Outputs from the current SBA call are not included in the feature vector. Observation and landmark counts (\ln n_{\mathrm{obs}}, \ln n_{\mathrm{lm}}) describe the number of constraints contributing to the visual cost. Pixel scatter (\tilde{\sigma}_{u}, \tilde{\sigma}_{v}) describes the image-space distribution of features. The preintegration interval (\ln\Delta t_{\mathrm{imu}}) relates to the duration over which IMU noise accumulates. Mean depth (\ln\bar{d}) and its spread (\mathrm{CV}_{d}) affect the scale and conditioning of reprojection errors relative to the IMU term. Survival rate (r_{\mathrm{surv}}) and reprojection residual statistics (\ln\bar{e}, \ln s_{e}) provide indicators of tracking quality. Low survival or elevated residuals may be associated with the imu-trust regime. Log transforms are applied for numerical stability.

III-E Visual Quality Estimator

We train a feedforward network f_{\theta}\colon\mathbb{R}^{10}\to\Delta^{1} (probability simplex over two classes) with architecture 10\to 32\to 16\to 2 (ReLU activations, softmax output, 914 trainable parameters including biases).

Training

The greedy procedure produces 110 segment-level alpha assignments across the 11 EuRoC sequences (10 per sequence), of which 103 receive a definite binary label (9 imu-trust, 94 vision-trust) and 7 fall in the transition region (\alpha^{*}\!=\!5.0) and are excluded. The larger proportion of vision-trust labels at the segment level (91\% of labeled segments) is consistent with the visual conditions represented in the EuRoC sequences. The same greedy procedure applied to TUM-VI Room gives a more balanced distribution. Approximately 57\% of labeled segments are imu-trust and 43\% are vision-trust. The difference between the two datasets supports the use of a classifier rather than a single fixed regime. Assigning each EuRoC segment label to all SBA calls within that segment yields approximately 13{,}700 feature vectors in total, of which approximately 9{,}500 have a definite binary label (the remaining \approx\!31\% fall in the transition region and are excluded from training). The segment-level and call-level class proportions differ because each segment label is repeated according to the number of SBA calls recorded within that segment. Calls within the same segment share a label and are therefore not independent. The effective label count for EuRoC is consequently 103. Features are collected from a common baseline (\alpha\!=\!1.0) rollout to provide a consistent input distribution across training sequences. Note that at inference the trajectory may differ from the baseline rollout due to prior MLP decisions, which is a known limitation of this approach. Training minimizes focal loss [5] with \gamma\!=\!2.0 as

\mathcal{L}=-\sum_{k}w_{y_{k}}(1-p_{y_{k}})^{\gamma}\log p_{y_{k}}
(3)

Per-class weights are set to inverse class frequency. Because imu-trust is the majority class (76\% of labeled calls), inverse frequency assigns it a lower base weight than vision-trust. Before normalization, the imu-trust weight is additionally multiplied by 2. This adjustment increases its value relative to the unmodified inverse-frequency weight and reflects a design preference against predicting vision-trust under poor visual conditions. After normalization the final weights are w_{\mathrm{imu}}\!\approx\!0.78 and w_{\mathrm{vis}}\!\approx\!1.22. The model is optimized with Adam (learning rate 1\!\times\!10^{-3}, weight decay 1\!\times\!10^{-4}, batch size 64) for 500 epochs with cosine annealing. Features are standardized with training-set mean and variance before both training and inference.

Inference

f_{\theta} runs once per SBA call immediately before the optimization step, consuming only the 10-dimensional feature vector assembled from tracker state prior to the current call. The predicted class determines \alpha\in\{2.0,\,10.0\} for that call. Each SBA call is classified independently using the per-call feature vector. The segment-level supervision used during training means that calls within the same oracle segment share a label, while at inference the prediction can vary freely across calls. No temporal smoothing is applied.

IV EXPERIMENTS

TABLE I: EuRoC MAV RMSE APE [m] (mean over N\!=\!3 runs). Greedy Ref. is the sequential non-causal S\!=\!10 reference. Smart Fusion uses LOOCV (11-fold). \Delta denotes relative change over \alpha\!=\!1.0, computed from unrounded means.
  • MH_01
    \alpha{=}1.0
    0.138
    Greedy Ref.
    0.049
    Smart Fusion
    0.073
    \Delta [%]
    -47.0
  • MH_02
    \alpha{=}1.0
    0.113
    Greedy Ref.
    0.064
    Smart Fusion
    0.078
    \Delta [%]
    -30.6
  • MH_03
    \alpha{=}1.0
    0.162
    Greedy Ref.
    0.078
    Smart Fusion
    0.105
    \Delta [%]
    -35.4
  • MH_04
    \alpha{=}1.0
    0.191
    Greedy Ref.
    0.150
    Smart Fusion
    0.160
    \Delta [%]
    -16.4
  • MH_05
    \alpha{=}1.0
    0.239
    Greedy Ref.
    0.131
    Smart Fusion
    0.144
    \Delta [%]
    -40.0
  • V1_01
    \alpha{=}1.0
    0.090
    Greedy Ref.
    0.090
    Smart Fusion
    0.090
    \Delta [%]
    \phantom{-}0.0
  • V1_02
    \alpha{=}1.0
    0.167
    Greedy Ref.
    0.089
    Smart Fusion
    0.099
    \Delta [%]
    -40.6
  • V1_03
    \alpha{=}1.0
    0.237
    Greedy Ref.
    0.114
    Smart Fusion
    0.119
    \Delta [%]
    -49.7
  • V2_01
    \alpha{=}1.0
    0.157
    Greedy Ref.
    0.128
    Smart Fusion
    0.143
    \Delta [%]
    -8.9
  • V2_02
    \alpha{=}1.0
    0.145
    Greedy Ref.
    0.106
    Smart Fusion
    0.115
    \Delta [%]
    -20.3
  • V2_03
    \alpha{=}1.0
    0.585
    Greedy Ref.
    0.331
    Smart Fusion
    0.481
    \Delta [%]
    -17.7
  • Mean
    \alpha{=}1.0
    0.202
    Greedy Ref.
    0.121
    Smart Fusion
    0.146
    \Delta [%]
    \mathbf{-27.7}

IV-A Setup

Datasets

EuRoC MAV [6] contains 11 stereo-inertial sequences in two indoor environments (Machine Hall MH_01–05 and Vicon Room V1_01–03, V2_01–03). We evaluate on all 11 sequences. V1_01 is included but its greedy-reference APE change is 0%, as varying \alpha over \mathcal{A} produces the same APE (0.090\,\text{m}) regardless of which value is selected. TUM-VI Room [7] provides five fisheye stereo-inertial room sequences (room1–5). We evaluate them under LOOCV to measure performance on a held-out sequence within the dataset. Room6 is excluded as its ground truth coverage does not span the full trajectory in our setup.

Metric

RMSE APE [m] with Sim(3) Umeyama alignment [8]. The \Delta column in all tables is the per-sequence relative change over the \alpha\!=\!1.0 baseline, computed from unrounded means. The Mean row reports arithmetic mean APE across sequences, and its \Delta value is the relative change between the baseline and Smart Fusion mean APEs. All results are averaged over N\!=\!3 independent runs to account for CUDA non-determinism in cuVSLAM.

Baseline and greedy reference

The primary baseline is the released cuVSLAM configuration with fixed \alpha\!=\!1.0. This comparison measures the change obtained by Smart Fusion relative to the standard fixed-weight configuration. The sequential greedy procedure in Sec. III-C is reported as a non-causal reference for the candidate set \mathcal{A}, not as a globally optimal upper bound. Each alpha candidate in the greedy search is evaluated over N\!=\!3 independent tracker runs, and the mean final APE is used for selection.

Evaluation protocol

For EuRoC, we use LOOCV over 11 sequences. In each fold, the MLP is trained on greedy-reference labels from 10 sequences and evaluated on the held-out sequence. For TUM-VI, we use LOOCV over 5 sequences. In each fold, the MLP is trained on greedy-reference labels generated from 4 TUM-VI sequences and evaluated on the held-out sequence, using that fold’s training statistics for standardization.

IV-B EuRoC MAV

Table I reports results for all 11 sequences. Mean APE decreases from 0.202\,\text{m} to 0.146\,\text{m} (-27.7\%), against the greedy-reference result of 0.121\,\text{m} (-40.2\%). V1_01 shows no improvement in either direction, as its APE is identical across all \alpha values.

The largest per-sequence reductions are observed on V1_03 (-49.7\%), MH_01 (-47.0\%), V1_02 (-40.6\%), and MH_05 (-40.0\%). The reduction achieved by Smart Fusion corresponds to 69\% of the difference between the baseline and the greedy reference. The remaining difference may arise from two sources. The per-SBA feature vector contains only local statistics, while the reference uses full-trajectory APE. In addition, some visual conditions in a held-out fold may be underrepresented in the corresponding training set. V2_03 is a notable case where the gap between Smart Fusion (0.481\,\text{m}) and the greedy reference (0.331\,\text{m}) is largest in absolute terms. V2_03 has the highest baseline APE among the evaluated EuRoC sequences. No sequence in the corresponding training fold has a comparable baseline APE.

IV-C TUM-VI Room

Table II shows results on TUM-VI Room under LOOCV (5 sequences). Mean APE decreases from 0.270\,\text{m} to 0.230\,\text{m} (-14.9\%), while the greedy-reference result is 0.222\,\text{m} (-18.0\%).

Room1 and room3 improve by 29.4\% and 21.3\%, respectively. Room4 and room5 change by less than 1\%. The corresponding greedy-reference changes are 6.3\% and 0.5\%, indicating limited improvement over the fixed baseline on these sequences. The results on held-out TUM-VI sequences suggest that the tracker-internal feature set transfers across sequences within TUM-VI Room without dataset-specific feature definitions.

TABLE II: TUM-VI Room RMSE APE [m] (mean over N\!=\!3 runs). Smart Fusion uses LOOCV (5-fold). \Delta denotes relative change over \alpha\!=\!1.0, computed from unrounded means.
  • room1
    \alpha{=}1.0
    0.286
    Greedy Ref.
    0.194
    Smart Fusion
    0.202
    \Delta [%]
    -29.4
  • room2
    \alpha{=}1.0
    0.148
    Greedy Ref.
    0.138
    Smart Fusion
    0.143
    \Delta [%]
    -3.2
  • room3
    \alpha{=}1.0
    0.525
    Greedy Ref.
    0.391
    Smart Fusion
    0.413
    \Delta [%]
    -21.3
  • room4
    \alpha{=}1.0
    0.104
    Greedy Ref.
    0.098
    Smart Fusion
    0.103
    \Delta [%]
    -0.7
  • room5
    \alpha{=}1.0
    0.289
    Greedy Ref.
    0.287
    Smart Fusion
    0.289
    \Delta [%]
    \phantom{-}0.0
  • Mean
    \alpha{=}1.0
    0.270
    Greedy Ref.
    0.222
    Smart Fusion
    0.230
    \Delta [%]
    \mathbf{-14.9}

V CONCLUSIONS

Smart Fusion formulates per-SBA \alpha selection as binary classification based on two empirical performance regimes observed over the tested candidate set. A 914-parameter MLP trained on sequential greedy labels from tracker-internal features reduces mean RMSE APE by 27.7\% on EuRoC (LOOCV, 11 sequences) and 14.9\% on TUM-VI Room (LOOCV, 5 sequences). These reductions correspond to 69\% and 83\% of the respective differences between the baseline and the greedy reference. The remaining differences may be related to the mismatch between the local per-SBA feature vector and the full-trajectory APE objective, as well as limited training coverage of difficult operating conditions. Future work could examine temporal features and a broader range of training conditions.

ACKNOWLEDGMENT

The authors thank the cuVSLAM team at NVIDIA for making the system available as open source.

References

  1. [1] A. Korovko, D. Slepichev, A. Efitorov, A. Dzhumamuratova, V. Kuznetsov, J. Biswas, H. Rabeti, and S. Pouya, “cuVSLAM: CUDA Accelerated Visual Odometry and Mapping,” arXiv:2506.04359v3, 2025.
  2. [2] T. Qin, P. Li, and S. Shen, “VINS-Mono: A robust and versatile monocular visual-inertial state estimator,” IEEE Trans. Robot., vol. 34, no. 4, pp. 1004–1020, 2018.
  3. [3] C. Campos, R. Elvira, J. J. G. Rodríguez, J. M. M. Montiel, and J. D. Tardós, “ORB-SLAM3: An accurate open-source library for visual, visual-inertial, and multimap SLAM,” IEEE Trans. Robot., vol. 37, no. 6, pp. 1874–1890, 2021.
  4. [4] C. Forster, L. Carlone, F. Dellaert, and D. Scaramuzza, “On-manifold preintegration for real-time visual-inertial odometry,” IEEE Trans. Robot., vol. 33, no. 1, pp. 1–21, 2017.
  5. [5] T.-Y. Lin, P. Goyal, R. Girshick, K. He, and P. Dollár, “Focal loss for dense object detection,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 42, no. 2, pp. 318–327, 2020.
  6. [6] M. Burri, J. Nikolic, P. Gohl, T. Schneider, J. Rehder, S. Omari, M. W. Achtelik, and R. Siegwart, “The EuRoC micro aerial vehicle datasets,” Int. J. Robot. Res., vol. 35, no. 10, pp. 1157–1163, 2016.
  7. [7] D. Schubert, T. Goll, N. Demmel, V. Usenko, J. Stueckler, and D. Cremers, “The TUM VI benchmark for evaluating visual-inertial odometry,” in Proc. IEEE/RSJ IROS, 2018, pp. 1680–1687.
  8. [8] S. Umeyama, “Least-squares estimation of transformation parameters between two point patterns,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 13, no. 4, pp. 376–380, 1991.
  9. [9] W. Lee, D. Adley, and C. Ferguson, “State-adaptive MSCKF feature gating for robust UAV visual-inertial navigation systems,” unpublished manuscript, 2026.
  10. [10] J. Kelly and G. Sukhatme, “Visual-inertial sensor fusion: Localization, mapping and sensor-to-sensor self-calibration,” Int. J. Robot. Res., vol. 30, no. 1, pp. 56–79, 2011.
  11. [11] G. Sibley, L. Matthies, and G. Sukhatme, “Sliding window filter with application to planetary landing,” J. Field Robot., vol. 27, no. 5, pp. 587–608, 2010.
  12. [12] S. Leutenegger, S. Lynen, M. Bosse, R. Siegwart, and P. Furgale, “Keyframe-based visual-inertial odometry using nonlinear optimization,” Int. J. Robot. Res., vol. 34, no. 3, pp. 314–334, 2015.
  13. [13] Y. Cong, W. Su, N. Jiang, W. Zong, L. Li, Y. Xu, T. Xu, and P. Wu, “Adaptive covariance matrix for UAV-based visual-inertial navigation systems using Gaussian formulas,” Sensors, vol. 25, no. 15, art. 4745, 2025.
  14. [14] Y. Hu and A. Gardi, “VINS-Mah: A robust monocular visual inertial state estimator for dynamic environments,” IEEE Robot. Autom. Lett., vol. 11, no. 3, DOI: 10.1109/LRA.2026.3656724, 2026.
  15. [15] S. Nascivera, L. Bauersfeld, J. Delaune, and D. Scaramuzza, “Image-conditioned adaptive parameter tuning for visual odometry frontends,” arXiv:2603.21785, 2026.
  16. [16] Z. Teed and J. Deng, “DROID-SLAM: Deep visual SLAM for monocular, stereo, and RGB-D cameras,” in Advances in Neural Information Processing Systems (NeurIPS), 2021.