Part-aware Measurement for Robust Multi-View Multi-Human 3D Pose Estimation and Tracking

Last update: Oct 27, 2022

Overview

Part-aware Measurement for Robust Multi-View Multi-Human 3D Pose Estimation and Tracking

Part-Aware Measurement for Robust Multi-View Multi-Human 3D Pose Estimation and Tracking
Hau Chu, Jia-Hong Lee, Yao-Chih Lee, Ching-Hsien Hsu, Jia-Da Li, Chu-Song Chen
2021 CVPR B-AMFG Workshop

Note: It's a project of AI^2 Lab. The code will be update in here while there is a new version.

Installation

Python 3.6+
Cuda 9.0
Cudnn 7
gcc 5 & g++ 5 (for Ubuntu 18.04)

$ sudo apt install gcc-5 g++-5
$ sudo ln -s /usr/bin/gcc-6 /usr/local/bin/gcc
$ sudo ln -s /usr/bin/g++-6 /usr/local/bin/g++

Conda Env

$ conda create -n venv python=3.6
$ conda activate venv
$ conda install pytorch==1.1.0 torchvision==0.3.0 cudatoolkit=9.0 -c pytorch
$ pip install tensorflow_gpu==1.9.0
$ pip install -r requirements.txt

$ sudo apt install git

Data preparation

Download datasets:

Campus (http://campar.in.tum.de/Chair/MultiHumanPose)
Shelf (http://campar.in.tum.de/Chair/MultiHumanPose)
CMU Panoptic (https://github.com/CMU-Perceptual-Computing-Lab/panoptic-toolbox)

Dataset's camera_parameter.pickle download

The directory tree should look like below:

${ROOT}
    |-- CatchImage
        |-- CampusSeq1
        |   |-- Camera0
        |   |-- Camera1
        |   |-- Camera2
        |   |-- camera_parameter.pickle
        |   |-- actorsGT.mat
        |-- Shelf
        |   |-- Camera0
        |   |-- ...
        |   |-- Camera4
        |   |-- camera_parameter.pickle
        |   |-- actorsGT.mat
        |-- Panoptic
        |   |-- 160906_pizza1
            |   |-- 00_03 # hdImgs folder of 03 camera
            |   |-- 00_06 # hdImgs folder of 06 camera
            |   |-- ...
            |   |-- camera_parameter.pickle
            |   |-- hdPose_stage1_coco19
            |-- ...
    |-- src

Backend Models

Backend models, which is not our works, are released codes from others. We only did some small modifications to fit the format of our input/output. Put models in {ROOT}/src/backend

YOLOv3
- origin: https://github.com/ZQPei/deep_sort_pytorch
- modification
HRNet
- origin: https://github.com/stefanopini/simple-HRNet
- modification

Run Codes

Demo

$cd src
python -W ignore testmodel.py --dataset CampusSeq1 # For Campus
python -W ignore testmodel.py --dataset Shelf # For Shelf
python -W ignore testmodel.py --dataset Panoptic # For Panoptic (sub-dataset can be modified in config)

Evaluation

$cd src
python -W ignore evalmodel.py --dataset CampusSeq1 
python -W ignore evalmodel.py --dataset Shelf

Campus PCP Score

Bone Group	Actor 0	Actor 1	Actor 2	Average
Head	100.00	100.00	100.00	100.00
Torso	100.00	100.00	100.00	100.00
Upper arms	98.98	100.00	100.00	99.66
Lower arms	92.86	68.78	91.30	84.31
Upper legs	100.00	100.00	100.00	100.00
Lower legs	100.00	100.00	100.00	100.00
Total	98.37	93.76	98.26	96.79

Shelf PCP Score

Bone Group	Actor 0	Actor 1	Actor 2	Average
Head	94.98	100.00	91.30	95.43
Torso	100.00	100.00	100.00	100.00
Upper arms	100.00	100.00	96.27	98.76
Lower arms	98.21	77.03	96.27	90.50
Upper legs	100.00	100.00	100.00	100.00
Lower legs	100.00	100.00	100.00	100.00
Total	99.14	95.41	97.64	97.39

Citation

@InProceedings{Chu_2021_CVPR,
    author    = {Chu, Hau and Lee, Jia-Hong and Lee, Yao-Chih and Hsu, Ching-Hsien and Li, Jia-Da and Chen, Chu-Song},
    title     = {Part-Aware Measurement for Robust Multi-View Multi-Human 3D Pose Estimation and Tracking},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops},
    month     = {June},
    year      = {2021},
    pages     = {1472-1481}
}

Part-aware Measurement for Robust Multi-View Multi-Human 3D Pose Estimation and Tracking

Related tags

Overview

Part-aware Measurement for Robust Multi-View Multi-Human 3D Pose Estimation and Tracking

Installation

Data preparation

Backend Models

Run Codes

Demo

Evaluation

Campus PCP Score

Shelf PCP Score

Citation

Owner

PyTorch implementation of DeepLab v2 on COCO-Stuff / PASCAL VOC

这是一个deeplabv3-plus-pytorch的源码，可以用于训练自己的模型。

A forwarding MPI implementation that can use any other MPI implementation via an MPI ABI

Bayesian optimization in PyTorch

Python Interview Questions

PyTorch implementation of MuseMorphose, a Transformer-based model for music style transfer.

Training data extraction on GPT-2

Cache Requests in Deta Bases and Echo them with Deta Micros

DRIFT is a tool for Diachronic Analysis of Scientific Literature.

Revisiting Weakly Supervised Pre-Training of Visual Perception Models

[3DV 2020] PeeledHuman: Robust Shape Representation for Textured 3D Human Body Reconstruction

A Joint Video and Image Encoder for End-to-End Retrieval

Thermal Control of Laser Powder Bed Fusion using Deep Reinforcement Learning

🍅🍅🍅YOLOv5-Lite: lighter, faster and easier to deploy. Evolved from yolov5 and the size of model is only 1.7M (int8) and 3.3M (fp16). It can reach 10+ FPS on the Raspberry Pi 4B when the input size is 320×320~

Text-to-SQL in the Wild: A Naturally-Occurring Dataset Based on Stack Exchange Data

Densely Connected Convolutional Networks, In CVPR 2017 (Best Paper Award).

A criticism of a recent paper on buggy image downsampling methods in popular image processing and deep learning libraries.

Pytorch implementation of "Neural Wireframe Renderer: Learning Wireframe to Image Translations"

[CVPR 2021] Semi-Supervised Semantic Segmentation with Cross Pseudo Supervision

Pyramid Scene Parsing Network, CVPR2017.