1. Summary

TVCG 2021 Periodical , Point cloud completion
Address of thesis ：https://ieeexplore.ieee.org/abstract/document/9528986

2. motivation

For the scene of multi object combination Partial completion
Pay close attention to ： Whether the two objects are close in space in the scene
As shown in the figure ： That is, two objects with similar space refer to each other to complete

3. Method

Problem definition ：

Given a partial point cloud after segmentation , There are two parts , Belong to different objects
The goal is ： Complete each part

Assume ： Assume The input is obtained from a scan , This is more challenging than capturing scenes from multiple directions . In theory , Our method can also be used to construct multi-directional scanning .

Two way network ：

Why two-way network ： In fact, it's because of two possible incomplete objects $O_a$ and $O_b$, Let's see which of these objects is completed first , If it's first A Again B So it's the left branch in the figure below , If it's first B Again A So it's the right branch .

Optimize the network by using consistency constraints , I hope the more similar the results of the two branches, the better . It is ensured by the parameter mapping of the graph .
reference ：https://consistency.epfl.ch/
This way is also called ： Conditional completion .
The definition is as follows ：

Overall network framework ：

Network based on self encoder ：
The short answer is

1. Encode all input point cloud features first , Then maximize the pool to get A global feature $g_1$
2. Then extend $g_1$ And contain only the features of the object that need to be completed （ yellow ）, Then maximize the pool into global features $g_2$
3. Last use TopNet To complete what needs to be completed
   

Loss function ：

1. Shape loss ：EMD
2. Loss of consistency ： Defined in two different paths EMD distance .

Training process ：

1. Train first Step1 The Internet
2. Training Step2 The Internet ;
3. Train the whole network ;

4. experiment

Data sets ：

from [29] Build it , That is to say Interaction context (ICON): Towards a geometric functionality
descriptor The data set of this article .

contain 6 Data sets of different combination types , Here's the picture 4：

• desk-chair
• vase-flower
• hanger-clothes
• basket-object
• handcart-object
• stand-hat

Partial scan simulation ：

1. First, calculate the region of interest , Pictured 4 Shown , Use IBS[28] To extract
2. Then after extraction IBS Upper sampling point
3. Then, the bounding boxes of these sampling points are calculated as the interactive region of interest .
4. These bounding boxes are normalized by coordinates , The visitor's center is regarded as the central point of interaction .
5. Then randomly sample different camera positions at the center of the region of interest 1024 A point for each object , Remove the situation where one object obscures another .

Visualization ：

experimental result ：

Ablation Experiment ：

chart 2：
• v1: PCN-encoder + PCN-decoder
• v2: PCN-encoder + topnet-decoder
• v3: Our-encoder + PCN-decoder
• v4 (ours): Our-encoder + topnet-decoder

Yes no Verification of consistency loss ：

Robust to noise

5. Spatial relations

This is partly because I am interested in , Pay attention to .
[27] Use the correlation matrix to encode the spatial relationship
[28] Put forward Interaction Bisector Surface （IBS） To capture interactive information , Include Geometric and topological features ;
[29] IBS + IR（ Interaction area ） To encode more geometric features .
The above methods are used to complete 3D Of the scene

The following method is to capture the spatial relationship between point clouds ：
[30] Figure network , combination 2D and 3D Set information to guide the expression of association relationship
[31] The overlap of the border after mapping Separated by the nearest distance between two objects , Then calculate the spatial correlation .
[32] and [33]： The joint Gaussian distribution is used to express the spatial relationship of different objects in a scene .
[34] Methods based on deep learning , Measure relationships by nearest neighbors
[35] Use the attention mechanism to capture the relationship between a point and its adjacent points .

Corresponding literature ：
[27] M. Fisher, D. Ritchie, M. Savva, T. Funkhouser, and P. Hanrahan, “Example-based synthesis of 3d object arrangements,” ACM Trans. Graph., vol. 31, no. 6, pp. 135:1–135:11, 2012.
[28] X. Zhao, H. Wang, and T. Komura, “Indexing 3d scenes using the interaction bisector surface,” ACM Transactions on Graphics (TOG), vol. 33, no. 3, pp. 22:1–22:14, 2014.
[29] R. Hu, C. Zhu, O. van Kaick, L. Liu, A. Shamir, and H. Zhang, “Interaction context (ICON): Towards a geometric functionality descriptor,” ACM Transactions on Graphics, vol. 34, 2015.
[30] X. Qi, R. Liao, J. Jia, S. Fidler, and R. Urtasun, “3d graph neural networks for RGBD semantic segmentation,” in 2017 IEEE International Conference on Computer Vision (ICCV), 2017, pp. 5209–5218, ISSN: 2380-7504.
[31] Y. Song, Z. Sun, Y. Wu, and H. Li, “Coarse-to-fine segmentation for indoor scenes with progressive supervision,” Computer Aided Geometric Design, vol. 75, p. 101775, 2019.
[32] M. Alberti, P. Jensfelt, and J. Folkesson, “Relational approaches for joint object classification andscene similarity measurement in indoor environments,” in AAAI Spring Symposium Qualitative Representations for Robots March 24–26 2014, Palo Alto, USA. The AAAI Press, 2014.
[33] M. Sunkel, S. Jansen, M. Wand, and H.-P. Seidel, “A correlated parts model for object detection in large 3d scans,” in Computer Graphics Forum, vol. 32. Wiley Online Library, 2013, pp. 205–214.
[34] Y. Liu, B. Fan, S. Xiang, and C. Pan, “Relation-shape convolutional neural network for point cloud analysis,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2019, pp. 8895–8904.
[35] Z. Li, J. Zhang, G. Li, Y. Liu, and S. Li, “Graph attention neural networks for point cloud recognition,” in 2019 IEEE International Conference on Multimedia and Expo (ICME). IEEE, 2019-07, pp. 387–392.