1. Summary

subject ：Unsupervised Multi-Task Feature Learning on Point Clouds
The paper ：https://openaccess.thecvf.com/content_ICCV_2019/papers/Hassani_Unsupervised_Multi-Task_Feature_Learning_on_Point_Clouds_ICCV_2019_paper.pdf

2. motivation

Traditional manual design features mainly capture local or global statistical attributes , Unable to represent semantic properties .
Is there a common feature for tasks that require semantic features ？

So this article This paper puts forward a method which can be used for unsupervised multitasking Feature expression of the model , Encoder based on multiscale graph . End to end training .

3. Algorithm

For three unsupervised learning tasks ：

1. clustering
2. restructure
3. Self supervised classification

The overall framework ：

decoder ： Multi scale （ Graph convolution + Convolution + Pooling ）+ Perturbed Gaussian noise
Essentially applicable to multitasking , It's also because we used the loss of these tasks to train together , Naturally, with this function .
Definition of volume of drawing ： It's actually the residual between two points ;

Algorithm description ：

Training set ： S = { s 1 , s 2 , . . . , s N } S=\{s_1, s_2, ..., s_N\} S={ s1​,s2​,...,sN​},N A little bit .
One point ： s i = { p 1 i , p 2 i , . . , p M 1 } s_i =\{ p^i_1, p^i_2, .., p^1_M\} si​={ p1i​,p2i​,..,pM1​}, M A disordered point , $p_j^i=(x_j^i,y_j^i,z_j^i)$ Include coordinates only
Encoder ：$E_{\theta }:S({\mathbb{R}}^{M\times d_{in}})\to Z({\mathbb{R}}^{d_z})$, $d_z$ Far greater than $d_{i}n$

In order to learn more tasks than supervision $\theta$, Design the following three parameter functions ：
Clustering function ${\mathrm{T}}_c：Z\to y$, Classify hidden codes into $K$ Of the three categories , among $y=[y_1,y_2,...,y_n]$, y i ∈ { 0 , 1 } K y_i\in\{0,1\}^K yi​∈{ 0,1}K, And $y_n^T{1}_k=1$.
Classification function $f_{\psi }:Z\to \hat{y}$, Category prediction after clustering , In other words , The classification function maps implicit variables to K A prediction class $\hat{y}=[\widehat{y_1},\widehat{y_2},...,\widehat{y_n}]$, And y i ^ ∈ { 0 , 1 } K \hat{y_i}\in\{0,1\}^K yi​^​∈{ 0,1}K. The function uses the pseudo tag generated by the clustering function as the agent training data .
Decoder function ：$g_{\varphi }:Z({\mathbb{R}}^{d_z})\to \hat{S}({\mathbb{R}}^{M\times d_{in}})$, Reconstruct the implicit variable back to the point cloud . If only clustering loss is used, the features will be clustered into a single class , The function is designed to prevent the final aggregation into a single class .

Loss of training ：

1. Clustering loss ： In essence, it is learning the clustering center matrix ：$C\in {\mathbb{R}}^{d_z\times K}$, $z_n=E_{\theta }(sn)$,$y_n^T{1}_k=1$. among , initialization The random clustering matrix is the center , yes epoch The updated .
   

2. Classified loss ： Cross entropy measures , $y_n={\mathrm{T}}_c(z_n)$, ${\hat{y}}_n=f_{\psi }(z_n)$.
   

3. Refactoring loss ：CD distance . ${\hat{s}}_n=g_{\varphi }(z_n)$ Is the reconstructed point set ,$s_n$ yes GT. $N$: Number of training sets ;$M$: The number of points in each point set .
   
   The ultimate loss ： above 3 Weighted summation of losses
   

The specific description process is shown in the following figure 1 Shown ：

4. experiment

Model convergence ：88 Classes , actual 55 Category .

The Clustering Visualization diagram is as follows ：

On classified tasks , Comparison of unsupervised and supervised methods ： It works well in unsupervised methods ;

The effect of semi supervised segmentation task ： 5% The effect of the training set is still very good ;

Split task effect ：

Encoder The role of , Test on classification tasks ： In fact, it seems to be saying here , complex Encoder The improvement after refactoring is not particularly great , The improvement of multi class tasks is very big .

Ablation Experiment ： Refactoring is the key , Plus the effect of classification > The effect of clustering , Because clustering here is actually for further classification , Realize the so-called unsupervised .

5. Conclusion and thinking

Failure case list ：

1. apply K-Means To initialize the cluster center , But there is no improvement in the random clustering center method ;
2. stay decoder And classification model soft Shared parameter mechanism , It is found that the effect is reduced , So separate the two ;
3. Try to iterate over more layers , Recalculate the nearest neighbor at each layer , It is found that this is disadvantageous to classification and segmentation ;