FUSEye: Training-Light Fisheye Detection with Overlapping Views and Zero-Initialized Adapters
This preprint introduces FUSEye, a training-light method that adapts a frozen COCO-pretrained YOLO26-x detector to fisheye camera images using overlapping views, zero-initialized residual adapters, and learned cross-view fusion. On WoodScape, it lifts mAP50 from 14.80 to 26.61, reaching 84.3% of full fine-tuning accuracy while training only about 227k added parameters. This matters because it lets robots adapt off-the-shelf detectors to wide-angle cameras without expensive full retraining or calibration.
Can a COCO-pretrained object detector be adapted to fisheye imagery without full-model fine-tuning while jointly handling boundary compression, feature misalignment, and cross-view detection conflicts under a limited labeling and training budget?
Fisheye cameras give robots broad coverage, but COCO-pretrained YOLO models transfer poorly because radial distortion warps local appearance and compresses peripheral objects to near-invisible sizes. For example, YOLO26-x scores only 14.80 mAP50 on WoodScape MVR for car/person/bus without adaptation, while full fine-tuning reaches 31.56, but full fine-tuning needs abundant fisheye labels and compute.
Prior work used calibration-dependent distortion-aware kernels, learned rectification networks that add a preprocessing stage, or deformable convolutions requiring full-model retraining. Generic parameter-efficient adapters are not fisheye-specific, and multi-crop inference methods such as SAHI or WBF fuse detections without explicitly modeling cross-projection agreement.
FUSEye keeps the YOLO26-x backbone frozen. It adds GridViews, which creates the full 1280x966 image plus four overlapping 857x647 corner crops resized to 640x640 and remaps boxes to original coordinates. Zero-initialized residual Z-Adapters are inserted after P4, P5, and C2PSA, adding 222,480 parameters while remaining identity at initialization; the detection head and adapters are trained for eight epochs. AgreeFusion clusters remapped per-class boxes at IoU at least 0.5, uses a 12-to-96-to-32-to-1 MLP scorer with about 4.4k parameters to re-score multi-view clusters, fuses boxes with weighted box fusion, and applies class-wise NMS.
On WoodScape MVR, FUSEye raises YOLO26-x from 14.80 to 26.61 mAP50 and from 10.96 to 17.26 mAP50:95, recovering 70.5% and 53.1% of the full fine-tuning gain; full fine-tuning achieves 31.56 and 22.82. With three random 25% training subsets of 1,520 images each, FUSEye averages 25.97 +/- 0.34 mAP50 and 16.84 +/- 0.25 mAP50:95, retaining 97.6% of full-label performance. In cross-model tests, FUSEye improves mAP50 by 5.80-8.29 points over COCO-pretrained baselines for YOLOv8-x, YOLOv9-e, YOLOv10-x, and YOLO11-x, while bus AP drops by 0.25-0.75 points in every pair.
The authors note bus detection remains weak and FUSEye's bus mAP50:95 falls below direct transfer on YOLO26-x. AP50:95 recovery is lower than AP50, indicating a localization gap versus full fine-tuning. Evaluation is limited to WoodScape car/person/bus, no real-robot or deployment timing tests are reported, and inference requires five forward passes. Code is stated as forthcoming rather than measured in the paper.
Mobile robot, ADAS/surround-view, drone, and security camera makers using fisheye or omnidirectional sensors could adapt COCO-pretrained detectors with far fewer labels and less training compute. Adoption in 1-3 years is plausible once the code is released and the recipe is validated on additional fisheye datasets and real robotic deployments; the five-pass inference overhead must also be weighed for embedded compute budgets.
The full text is not republished here because the paper's license does not allow it. Read the original on arXiv.