TAMNet: A loss-balanced multi-task model for simultaneous detection and segmentation

The framework uses SSD and FCN as pipeline.

• Task-related Attention Module (TAM): Consider the features of two tasks at the same time and use the attention mechanism to weight
  
  1. After adding the features of the two branches, the weighted map is obtained through the attention module (CNN)
  2. The two features are weighted with the weighted map and then combined withAdd the original features to get the final feature
  Public expression: F represents the previous feature, M represents the weighted graph, i, c represent the spatial subscript and channel subscript respectively
  
• Optimization method
  Most of the practice is to manually set the weight W:
  
  This paper proposes that the difficulty of task t at step i+1 is:
  
  Define rewards (weights) based on the difficulty of each task:
  
  
  
  So: