dried food! Point based: differentiable Poisson solver

Click on the blue words

Pay attention to our

AI TIME Welcome to everyone AI Fans join in ！

In the last few years , Neural implicit expression is very popular because of its excellent expression ability and flexibility , But it is also limited by slower prediction time .

In our article , We revisited the classic but powerful explicit point cloud expression , A differentiable Poisson solver is proposed , It can efficiently transform the directed point cloud into dense grid through an implicit indicator function . 

The connection between such points and grids allows us to use simple , Lightweight and interpretable point clouds are expressed as shapes , This also accelerates the prediction time by an order of magnitude compared with neural implicit expression .

At the same time, compared with other display expressions , We can also model shapes with different topologies .

Last , We have shown effectiveness in the task of 3D surface reconstruction based on pure optimization and depth learning .

In this issue AI TIME PhD studio , We invited to Zurich Federal Institute of Technology (ETH Zurich) And the Institute of intelligent systems of Max Planck Institute in Germany (Max Planck Institute for Intelligent Systems) Doctor —— Peng Songyou , Bring us report sharing 《 In the shape of a point ： Differentiable Poisson solver 》.

Peng Songyou ：

Doctor is studying at the Federal Institute of technology in Zurich (ETH Zurich) And the Institute of intelligent systems of Max Planck Institute in Germany (Max Planck Institute for Intelligent Systems), from Marc Pollefeys and Andreas Geiger Jointly guide . His research interests are mainly in the intersection of three-dimensional vision and deep learning , Especially interested in studying the implicit and explicit neural expression of three-dimensional scenes , And their applications in 3D reconstruction and neural rendering .

Content abstract

For the shape of any object , We can express it in different forms . What we put forward in this study SAP It is a differentiable version of classical Poisson surface reconstruction . besides ,SAP It is also a new expression of object shape .

So what is a better three-dimensional shape expression ？

Some traditional forms of shape expression have already undergone sufficient research , Such as point cloud . Their advantage is in inference Very efficient at all times , But there are widespread discontinuities .

In recent years, there is a hot research direction ——Neural Implicit Representations, That is, the shape can be expressed as the output of the network . But it is inference A little slower .

therefore , We propose a hybrid expression . Or use points to represent shapes , But we associate the point with an implicit expression through a differentiable Poisson solver . On the one hand, you can get high-quality presentation , On the one hand, it can quickly inference.

Method

Differentiable Poisson solver DPSR：Differentiable Poisson Solver

The left side of the figure above is the definition field with normal vector provided by us , The Poisson solver can output a dense mesh from this point cloud , And can judge whether the points in the grid are inside or outside the object .

The key to solving Poisson equation , We provide two-dimensional examples to help understand .

First , We consider the circle as our shape Shape, Can be transformed into an indicator function Indicator Function（ Black represents the interior of an object , White represents the outside of the object ）, Then we can get the derivative of it Gradient; It is found that the derivative exists only on the surface ;

When we are on the surface sample A little bit , Vector method at the same time , The normal vector can be connected with the previous indicator function .

Solving Poisson's equation

Take the derivative of the divergence operator directly , Just pay attention to the boundary conditions .

Spectral methods To solve the Poisson equation .

• The derivative of the signal in the spectral domain is analytically calculated

• The fast Fourier transform (FFT) stay gpu / tpu Highly optimized on

• It takes only a small amount of code to complete

The differentiable Poisson solver proposed by us can be widely used in various fields task On .

be based on Unoriented Point 3D surface reconstruction of cloud

（ There is no normal vector ）

1）SAP for Optimization-based 3D Reconstruction

Pure optimized 3D reconstruction （ No deep learning network ）

Given the target cloud , Restore its 3D model . We first initialize a point cloud with a normal vector , To demonstrate flexibility and robustness , Let's initialize his vector and go to a point .

Then we can input the point cloud into DPSR Poisson solver to obtain a dense indication grid .

We can get one from the grid later march, And from march On sample A little bit ; Compare these points with target（ Target point cloud ） Make one tranfer distance obtain loss.

We can use the above methods to update Point cloud ,

Comparison

We can find that our method can restore the previous shape very well .

The classic method before —— Poisson reconstruction method is shown in the figure below ：

The problem is that it depends too much on whether the normal vector is accurate , A little noise can cause inaccurate results .

2）SAP for Learning-based 3D Reconstruction（ There is a deep learning network ）

Suppose we have a noisy The input of , The purpose is to learn the network shown in the figure above , This network can estimate how the current point cloud moves and what its normal vector is .

Then we input the output in the figure above into DPSR in .

We from DPSR Get from Indicator Function Indicator field . Here is GroundTruth Indication field acquisition process .

Last , We obtained it in the above two ways Indicator Function The indicator field obtains a loss, And pass DPSR Pass back the derivative to get Gradient Come on update Our network .

stay inference When , Finally get march The output process is as follows ：

Results

Benefit of Geometric Initialization

Although we can initialize the input point cloud geometrically , But what we found was that SAP converges Is the fastest .

Conclusions

We propose a Poisson solver SAP：

• SAP It's explicable , Lightweight and guaranteed HQ watertight meshes

• SAP It is also topology independent , Support fast inference

• Our Poisson solver is differentiable and GPU Accelerated

Limitation

Memory increases in cubic form , So it will be limited to smaller scenes .

carry

Wake up

Thesis title ：

Shape As Points: A Differentiable Poisson Solver

Thesis link ：

https://proceedings.neurips.cc/paper/2021/hash/6cd9313ed34ef58bad3fdd504355e72c-Abstract.html

Click on “ Read the original ”, You can watch this playback

Arrangement ： Lin     be

author ： Peng Songyou

Excellent articles in the past are recommended

Remember to pay attention to us ！ There is new knowledge every day ！

  About AI TIME 

AI TIME From 2019 year , It aims to carry forward the spirit of scientific speculation , Invite people from all walks of life to the theory of artificial intelligence 、 Explore the essence of algorithm and scenario application , Strengthen the collision of ideas , Link the world AI scholars 、 Industry experts and enthusiasts , I hope in the form of debate , Explore the contradiction between artificial intelligence and human future , Explore the future of artificial intelligence .

so far ,AI TIME Has invited 600 Many speakers at home and abroad , Held more than 300 An event , super 150 10000 people watch .

I know you.

Looking at

Oh

~

Click on Read the original   View playback ！