Through the local-global attention module to fuse the features of the three modalities of camera, gating and lidar to complete "3D object detection based on weather perception and attention"

Method

Local and Global Attention Network

Local Attention

The features of the three modalities pass through the Inception Block to calculate their respective local attention weights LA:


The features of the modality are multiplied by the corresponding attention map and then added to obtain the local attention map:

Global Attention

Similarly, the features of the three modalities are passed through the Inception Block to calculate their respective attention weights GA:


willThe local attention feature map obtained in the previous step is multiplied by the respective attention weights (through the Inception Block) and added to obtain the global attention feature map:

Experiments

Ablation Experiment: Accuracy Comparison of Different Modal Data and Fusion Method Models on DENSE Dataset

Accuracy comparison of different models on DENSE dataset:

Attention visualization: white ovals highlight the best detection features for each modality