Abstract Sensor fusion—the synergistic integration of data from multiple sensing modalities—has evolved from a niche engineering practice into a foundational pillar of autonomous systems, robotics, and augmented reality. Recent breakthroughs in deep learning, probabilistic inference, and edge computing have enabled unprecedented robustness in perception under adverse conditions. This article reviews the latest advances in sensor fusion, focusing on cross-modal calibration, uncertainty-aware fusion architectures, and the emergence of foundation models for multimodal perception. We highlight key technical breakthroughs—including transformer-based fusion, differentiable Kalman filters, and neural implicit representations—and discuss persistent challenges such as domain shift, computational constraints, and the need for explainability. Finally, we outline future directions, including active sensing, collaborative fusion across distributed agents, and the integration of physics-informed priors.

1. Introduction The fundamental premise of sensor fusion is that no single sensor provides complete, reliable, and continuous information about the environment. Cameras offer rich semantic detail but fail in darkness or glare; LiDAR provides precise geometry but is sparse and expensive; radar is robust to weather but suffers from low angular resolution; inertial measurement units (IMUs) are drift-prone but high-rate. The fusion of these modalities has become critical for applications ranging from autonomous driving (AD) to surgical robotics. The past three years have witnessed a paradigm shift from hand-crafted fusion rules to learned, end-to-end differentiable systems that adapt to context. This article synthesizes recent progress, emphasizing works published between 2022 and 2025.

2. Cross-modal calibration and synchronization A prerequisite for effective fusion is accurate spatial and temporal alignment. Traditional target-based calibration methods are being replaced by target-less, deep-learning-based approaches. For instance,Cai et al. (2023)introduced a self-supervised framework that aligns LiDAR point clouds with camera images using photometric and geometric consistency losses, achieving sub-degree accuracy without external markers. Temporal synchronization has also advanced:Meyer et al. (2024)proposed a continuous-time fusion model that estimates per-sensor latency as a latent variable, enabling robust fusion even with asynchronous sensors. These methods significantly reduce deployment overhead in real-world systems.

3. Uncertainty-aware fusion architectures Modern fusion systems increasingly treat uncertainty as first-class information. Two dominant paradigms have emerged:

4. Transformer-based and attention-driven fusion The transformer architecture has become the de facto standard for fusing heterogeneous sequences. The key advantage is its ability to model long-range dependencies across both spatial and temporal dimensions.Li et al. (2024)proposedFusionTransformer, which uses cross-attention layers to align camera tokens with LiDAR voxel tokens. Unlike previous convolution-based fusions, FusionTransformer learns dynamic attention weights that shift depending on scene context—e.g., prioritizing radar in fog and camera in well-lit urban scenes. Evaluated on the nuScenes dataset, it achieved state-of-the-art 3D object detection (74.2 NDS) while being 2.3× more parameter-efficient than prior methods.

A parallel trend istoken-level fusion with mask modeling.Wang et al. (2025)introducedMaskFusion, which randomly masks 50% of each modality’s tokens during training and learns to reconstruct missing tokens from the other modality. This self-supervised pre-training strategy yields a fusion backbone that is highly robust to sensor dropout—a critical failure mode in real deployments.

5. Neural implicit representations for multi-sensor mapping Neural radiance fields (NeRFs) and 3D Gaussian splatting have been extended to fuse camera, LiDAR, and radar data into a single continuous scene representation.Zhu et al. (2024)presentedSensorNeRF, which jointly optimizes a radiance field and a radar-specific cross-section model. The result is a photorealistic 3D map that also encodes radar reflectivity, enabling simultaneous visual localization and radar-based mapping in GPS-denied environments. This work demonstrates a new direction: fusion not only for perception but forscene reconstructionas a unified product.

6. Foundation models and cross-modal transfer The rise of large multimodal foundation models (e.g., CLIP, GPT-4V) has inspired attempts to use them as priors for sensor fusion.Krishna et al. (2025)proposedFusionCLIP, which aligns LiDAR point clouds, camera images, and radar spectrograms into a shared semantic embedding space using contrastive learning. Once trained, FusionCLIP enables zero-shot fusion: given a radar pattern and a camera image, the model can reason about object semantics without task-specific fine-tuning. While still in early stages, this approach promises to drastically reduce the need for labeled fusion datasets.

7. Challenges and open problems Despite rapid progress, several issues remain:

8. Future outlook Looking ahead, three directions appear most promising:

1. Active sensor fusion: Future systems will dynamically control sensor parameters (e.g., LiDAR scan density, camera exposure) based on the current uncertainty map. This closes the loop between perception and sensing. 2. Collaborative fusion: Vehicle-to-everything (V2X) communication enables fusion across agents. Recent work byXu et al. (2025)onCoFusionshows that sharing raw feature maps between two vehicles can improve detection range by 30%, but raises privacy and bandwidth concerns. 3. Physics-informed fusion: Incorporating known physical constraints (e.g., radar Doppler shift, IMU preintegration) into neural architectures can improve generalization. Early results indicate that physics-based inductive biases reduce sample complexity by an order of magnitude.

Conclusion Sensor fusion has matured from a signal-processing afterthought into a core research area at the intersection of machine learning, robotics, and computer vision. The convergence of transformer architectures, uncertainty quantification, and neural scene representations is enabling systems that are not only more accurate but also more resilient to sensor failures. As we move toward fully autonomous and collaborative systems, the ability to fuse heterogeneous data with principled uncertainty will be a defining capability.

References