Improving 3D object detection with channel wise transformer

title ：Improving 3D Object Detection with Channel-wise Transformer

iccv2021
Zhejiang University Aridamo house The famous CT3d
The article is on the point transformer, Previous articles have said that on a huge point cloud tranformer The amount of computation is enormous So this article is a little clever utilize transformer Design refinement net
Article based on utilization The characteristics of the original point are sufficient to box refinement Based on the assumption of
Old rules Upper figure

You can see trans It is mainly used in the stage of encoding and decoding
Firstly, the author uses the output of single-stage network proposals Make a size expansion of them The concrete operation is to proposal Expand into a large cylinder The radius of the cylinder is proposal The diagonal of The height does not limit the size In such a cylinder We randomly choose 256 individual point Conduct transformer

transformer There is no innovation in the structure of The main reason is that the author combines location coding with feature A deeper fusion Instead of working before pe Simple addition of .

And then 256 Superposition of points obtain NXD The characteristic dimension of

The following is the focus of the full text ：

decode link

Compared with the traditional decoder m individual query embedding Different We only have one here Because we only need to restore one proposal

Standard coding ：

vector qK Each value of can be regarded as a global aggregation of a single point ( That is, each key is embedded ), Follow up softmax The function assigns a decoded value to each point according to the probability in the normalized vector . therefore , The value of decoding weight vector is a simple global aggregation , Lack of local channel modeling , This is necessary to understand the three-dimensional surface structure of point cloud , Because different channels usually show a strong geometric relationship in the point cloud .

channel code

and softmax The calculated decoding weight is associated with each channel , Ignore the global aggregation of each point . therefore , We can come to a conclusion , The standard decoding scheme focuses on global aggregation , The channel weighting scheme focuses on channel local aggregation . In order to combine their characteristics , We propose an extended channel weighting scheme , As shown below .

So we propose to combine the two schemes Mainly for The output of the previous module Conduct repeatD Time Then with Kh Multiply result softmax Then linear mapping

As a result, it can be compared with point Multiply

Finally, we analyze each dimension reweight Handle Output 1XN Of feature The result is convenient value Weighted Get our final output
Send the final output to two different ffn Test in iou and regression.
experiment ：


ablation The influence of different pre single-stage detectors ：

pe Advantages of modules

And different decoding The way is good or bad

In fact, it has not been greatly improved ...
The main reason I think is for proposal Come on Global features are enough