Inventory clearing series , It's been a long time .

1. Summary

subject ：Implicit Functions in Feature Space for 3D Shape Reconstruction and Completion,CVPR 2020
The paper ：https://virtualhumans.mpi-inf.mpg.de/papers/chibane20ifnet/chibane20ifnet.pdf
Code ：https://virtualhumans.mpiinf.mpg.de/ifnets/.

2. motivation

The existing reconstruction of rigid objects is limited by two points ：

1. Cannot represent complex objects , For example, reconstruction often loses arms or legs ;
2. They cannot retain the details presented in the input data .

Reasons for possible limitations ：

1. Online learning in xyz Too strong a priori in coordinates destroys the inflexibility of clarity ;
2. The shape coding vector lacks three bit coding , This makes the code look more like a classification of shape prototypes , Instead of continuous regression . therefore , The current method is limited by the above 2 spot ;

This work is based on the above considerations , Propose hidden feature network , stay 5 Three dimensions have been improved .

The desired effect ：

3 Algorithm

In essence, it is decode The design of the , because encoder Is the basic feature extraction part ; and decoder In fact, whether the prediction point is inside or outside the object , Then the boundary between the two is the object surface we need . Finally, it is necessary to reproduce the continuous surface through other algorithms . Because the discrete value of the output ：0 or 1.

3.1 background ： Implicit surface learning

Definition occupancy The formula . Use invisible expression $z$ Encoding 3D shape . Then the continuous shape expression can be obtained by learning neural function ：

Surface pass [0,1] Set whether it belongs to the inside or outside of the object . In this case, the surface is the edge of decision-making . Continuous representation can be unrestricted by pixels .

Later, you can build mesh in marching cubes Algorithm to express object constraints , But there are two limitations ：

1. Unable to express complex objects , for example Jointed figures, etc .
2. Failed to keep the details of the input data .

3.2 Invisible feature network

Shape coding $g$： Used 3D Convolution , Multiscale depth feature grid $F$, as follows ：

In fact, the most important thing is to use occupancy Achieve multi-resolution output , In the process of knowledge, multi-scale is used to ensure the retention of details .

Shape decoding $f$：
It is different from the direct point cloud coordinates $\mathbf{p}$ To classify , By learning the coordinates of our features ${\mathbf{F}}_1(\mathbf{p}),...,{\mathbf{F}}_n(\mathbf{p})$. Because the feature grid is discrete , Therefore, cubic linear interpolation is used to obtain continuous 3D spot . This is to encode a point and its field points in the front small receptive field area , Cartesian coordinates are expressed as follows ：

among $d$ Is the point from the center .$e_i$ For the first time $i$ A Cartesian unit vector .
Input the encoder result to the decoder $f$, Include a full connection layer to predict points $\mathbf{p}$ Is it outside the surface or inside , It is shown in the following formula ：

So with the most basic implicit expression , The formula （1） contrast , The features here include local and global features , Not just coordinate information . Due to such a multi-scale coding mode , Details can also be well preserved .

In short, extracting feature level is omnipotent ： Multi scale to consider the overall situation + Local correlation information .

3.3 model training

Use To minimize the mini-batch To determine the loss of ： In essence, it is predicted that this point is 0 still 1, And then GT Of $o_i()$ comparison .

4. experimental result

Visualization ： There will be a smoother surface , Then include some details

completion ：

Apply to voxels ：

5. Conclusion and thinking

Nature is , Turn the learning of points into Classify problems to operate , Predict where is inside the surface , Where is the outer surface , use 0,1 To label . Well, in fact, these points are discrete , Therefore, post-processing is needed to realize curved surface .
Point based ： Just give each point a prediction ;
Voxel based ： Use marching cubes Algorithm implementation ;