[paper reading] [3D target detection] point transformer

Paper title ：Point Transformer

iccv2021
use transformer Do some cloud work
Because of the randomness of the point cloud transformer Just deal with such problems

however Obviously, for large-scale point clouds Use it directly transformer The amount of computation is enormous So the author puts forward a new transformer Processing form , That is knn Find nearby points .
First The author introduces transformer The background of Self attention operation can be divided into scalar and vector attention
scalar It's what we often call the attention mechanism

and vector attention Then consider two token The relationship between

β Is a relational function （ Subtracting the ） γ For one mapping function
and Proposed by the author point transformer layer It mainly makes use of vector attention Module Consider two token The relationship between At the same time, add the location code to value in It means considering the relationship between position and value ：


The specific operation is for the input primary and secondary points Let's start with linear Feature coding for Subtract them to get their relation, Then add the location code Conduct mlp After the operation, take their coded value Then proceed norm The weight matrix of Let the weight matrix be multiplied by the value function and position code to get the code k Output point characteristics of adjacent points Be careful Conduct transformer The point is that we first carried out a knn Of the query Select... Near the main point k Let's do it at one point value weighting .

The author's position coding adopts relative position Then a linear layer coding ：

The overall framework of the article is ：

You can see that the network framework is mainly pointnet++ Framework
SA The layer is replaced by tranformer Plus for knn Near the point max pooling Take the next sample And the upper sampling feeling is completely FP layer .

hinder ablation Comparison of the k Selection of adjacent points


Effectiveness of location coding

attention The necessity of the module

in general The innovation of the article is to put forward point transformer layer
However, this layer can mainly be operated indoors with dense point clouds For some automatic driving scenes The point cloud is very sparse Use... In these scenarios knn It's unwise to query the proximity point And the amount of calculation is huge .
The way of location coding can be learned ！