[paper reading] [3D object detection] voxel set transformer: a set to set approach to 3D object detection from point clouds

Paper title ：Voxel Set Transformer: A Set-to-Set Approach to 3D Object Detection from Point Clouds

cvpr2022
transformer It has gradually become a new trend to use it on point clouds This article is to use transformer do 3d object detection .
The author first analyzes some existing methods Directly point by point on the point cloud transformer It doesn't work Because it's so big And some existing methods For example, add a point cloud group Do later trans It's hard to avoid missing some points Convert the point cloud to voxel Conduct 3d Convolution words be relative to transformer Come on The receptive field is very small . that Is there a way to enjoy trans Bring global features It can reduce the amount of calculation This article is written in this context ：
Old rules Upper figure ！

The article is in point level do You can see The author puts forward a relatively new point in the article vsa layer , Let's take a closer look at this structure ：
This structure is for all point clouds Firstly, the author introduces set transformer The concept of , It is suggested that you can browse a little before reading this article nips An article on ：《Set transformer: A framework for attention-based permutation-invariant neural networks》
The main idea is similar to linformer I think the self attention matrix is a low rank module So for the self attention module We can use two cross attetion To reduce the rank , And then reduce the rank of attention matrix Multiply with our input matrix .

And for all point clouds We'll start from here , First, map the point cloud to key and value Let's do a linear mapping operation And then calculate key And a reduced rank matrix L（K*D） Between cross attention And again value Multiply after softmax Calculation value After the weight of value Multiply obtain hidden feature This hidden feature For every point We put them according to voxel The division of is assigned to each voxel in Make up each voxel Of feature depth ：


To constitute voxel chart We do deep convolution Extract different voxel Characteristics between This convolution is done twice Last but not least broadcast What will be obtained feature Project back to the original point For newly acquired feature We call them key and value Perform the similar with the low rank matrix crossattention After the operation Get new point feature

We can see Ahead encoder It's actually vfe The operation of

The above modules That is the focus of the full text vsa
Then the module is superimposed Keep extracting pointlevel The overall characteristics of The use of soft pool Project features onto bev

X Is that the bev voxel Characteristics of points within
The last part 2d Convolution Output detected head

The author believes that the network can also be extended to two stages The second stage The author makes use of cvpr2021 Of Lidar rcnn As refinement net


ablation The experiment made a replacement ffn And different latent codes And in pointrcnn Replace above Sa by transformer Here's how it works ：

You can see The important thing is Conduct voxel Between the conv Can greatly improve ap Compare with the original sa transformer It can also better extract local features .