Fast and robust multi person 3D pose estimation from multiple views

Fast and Robust Multi-Person 3D Pose Estimation from Multiple Views

subject ：《 Fast robustness, multi view, multi person 3D Attitude estimation 》

author ：

 

source ：CVPR 2019

research contents :

         Many people - Multiview - Unsupervised

Innovation points ：

        ① A multi-channel matching algorithm is proposed , To find the... Detected across multiple views 2D The periodic correspondence of posture . The proposed matching algorithm can prune the error detection and deal with the partial overlap between views , Without knowing the real number of people in the scene ;

        ② Combine geometric and visual cues , To match the... Detected across views 2D posture .

Existing problems and technologies ：

        Through graph structure model [1]  Direct reasoning 3D China and 2D Detect all assumptions of geometric compatibility . Its disadvantage is that the structural state space of the problem is huge , When the number of cameras is small , The method is not robust （ Sensitive to the number of cameras ）, Therefore, this paper ： Multi channel matching algorithm based on convex optimization .

      Existing multi-channel matching 2D Attitude algorithm , Polar constraints are often used （ Judge 2 individual 2D Whether the posture is the same 3D Projection of attitude ）, The disadvantages are as follows ① For occlusion and truncation positions , Estimation inaccuracy ;② Circular consistency constraints may be violated for each pair of views [2] 

      The main challenge ： Find noise and incomplete 2D Find cross view correspondence in posture （ Who belongs to 2D Pose Match the view of ）

    In this paper, ：

        Multi channel matching algorithm based on convex optimization （ For all views detected 2D Pose In order to solve the problem of body clustering ）

                                                                 

                                  The same person with different views 2D Pose Consistent with key points

【 Periodic consistency constraint ： Solve the global consistency correspondence generated by multi view matching ; Geometric consistency + Physical similarity （ From this, we can get 3D Posture ） So as to reduce fuzzy matching 】

                                                                 

            For everyone and matching 2D Pose Deduce 3D Pose( The size of state space of multi attitude joint is reduced )

The framework of this paper ：

3.1 Multi view matching process

（1） Parameters involved ：

        ① Suppose there is V A camera ;

        ② In view i Bounding box detected in pi;

        ③ For a pair of views , Use Aij To represent the affinity matrix , Its elements represent affinity scores ;

        ④ The correspondence between the two sets of bounding boxes is estimated by the partial permutation matrix Pij Express ; 

        ⑤ By N Composed of two joints 2D posture .

（2） Problem description

         The affinity matrix Aij As input , Output the best partial permutation matrix Pij, To maximize the corresponding functions and cycle through multiple views （ Affinity matrix A It combines appearance similarity and geometric compatibility ）

（3） Implementation process ：
    A: Affinity matrix of fusion

        ① Use Re-ID The network obtains the descriptor of the bounding box （ Such as ： Black clothes 、 Long hair, etc ） after , And from “pool5” The feature vector is extracted as the descriptor of each bounding box . then , We calculate the Euclidean distance between the descriptors of the bounding box pair , And use sigmoid Function maps the distance to The value in is used as the appearance affinity score of this bounding box pair .

        ② Measure by distance ： Geometric consistency calculation （Xi,Xj）, Satisfy Dg It's very small .

 

Sum up , The fusion affinity matrix is obtained A by

 

    B： Find the partial permutation matrix P（ The estimated relationship between the two sets of bounding boxes ）

        ① Suppose the permutation matrix P Satisfy

          However P Can be broken down into P=YYT（Y Express 2D Bounding box and 3D The correspondence between ）

②

         Optimize it ： No longer required P Semi positive definite , Only required P symmetry

（if P Symmetrical and Pii=Ii,0<P<1）

         Solving this problem only ADMM Alternating method of algorithmic device , Introduce auxiliary variables Q Rewrite this question

Expand Lagrange

3.2 3DPS structure

（1） Parameters involved ：① The joints i Location ti;

                          ②2D View Vi

（2） Yes 3Dpose An estimate of the possibility ：

         Maximize P(T|I) General strategy ：
            Transform the state space into 3 Three dimensional mesh of dimensional space , Apply the maximum product method , But its complexity increases with the increase of grid space .

         This article maximizes P(T|I) Strategy ：

             use 3D The state space is 2D Triangulation of joint pairs 3D coordinate （ Just check the joints in both views , Its reality 3D The location is included in the proposal ）, Its state space parameters are reduced , And the accuracy increases .

Data sets ：

（1）Campus Data sets ：

        3 people , Outside ,3 The camera

        assessment ：PCP（ Correctly estimate the percentage of parts ）, Measure body parts 3D Location accuracy .

（2）Shelf Data sets

        4 people , Rack removal ,5 The camera , More occlusion .

（3）CMU Panoptic Data sets

        Many people , indoor , A hundred Cameras , The method used in this paper for qualitative evaluation .