Code for "Multi-View Multi-Person 3D Pose Estimation with Plane Sweep Stereo"

Last update: Jan 04, 2023

Related tags

Overview

Multi-View Multi-Person 3D Pose Estimation with Plane Sweep Stereo

This repository includes the source code for our CVPR 2021 paper on multi-view multi-person 3D pose estimation. Please read our paper for more details at https://arxiv.org/abs/2104.02273. The project webpage is available here.

Bibtex:

@InProceedings{Lin_2021_CVPR,
    author    = {Lin, Jiahao and Lee, Gim Hee},
    title     = {Multi-View Multi-Person 3D Pose Estimation With Plane Sweep Stereo},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
    month     = {June},
    year      = {2021},
    pages     = {11886-11895}
}

Environment

Our code is tested on

Python 3.8.5
PyTorch 1.6.0 & torchvision 0.7.0
CUDA 11.2

Preparing Data

Download following data before using the code in this repository:

Annotations and 2D pose predictions for the Campus and the Shelf datasets can be downloaded here. Credit to VoxelPose.
Follow the instructions on the CMU Panoptic Github repo to download the annotations. 2D pose predictions can be downloaded here.
Pre-trained models can be downloaded here.

The data should be organized as follows:

    ROOTDIR/
        └── data/
            └── Campus/
                └── actorsGT.mat
                └── calibration_campus.json
                └── pred_campus_maskrcnn_hrnet_coco.pkl
            └── Shelf/
                └── actorsGT.mat
                └── calibration_shelf.json
                └── pred_shelf_maskrcnn_hrnet_coco.pkl
            └── Panoptic/
                └── 160224_haggling1/
                └── 160226_haggling1/
                └── ...
                └── keypoints_train_results.json
                └── keypoints_validation_results.json
            └── panoptic_training_pose.pkl
        └── output/
            └── campus_synthetic/mvmppe/config/model_best_pretrained.pth.tar
            └── shelf_synthetic/mvmppe/config/model_best_pretrained.pth.tar
            └── panoptic/mvmppe/config/model_best_pretrained.pth.tar
        └── ...

Training and Inference

Below are the commands for training our model on different datasets.

The Campus dataset:

    python run/train.py --cfg configs/campus/config.yaml

The Shelf dataset:

    python run/train.py --cfg configs/shelf/config.yaml

The CMU Panoptic dataset:

    python run/train.py --cfg configs/panoptic/config.yaml

Below are the commands for performing inference with our pre-trained models.

The Campus dataset:

    python run/validate.py --cfg configs/campus/config.yaml -t pretrained

The Shelf dataset:

    python run/validate.py --cfg configs/shelf/config.yaml -t pretrained

The CMU Panoptic dataset:

    python run/validate.py --cfg configs/panoptic/config.yaml -t pretrained

Code for "Multi-View Multi-Person 3D Pose Estimation with Plane Sweep Stereo"

Related tags

Overview

Multi-View Multi-Person 3D Pose Estimation with Plane Sweep Stereo

Environment

Preparing Data

Training and Inference

Owner

Jiahao Lin

A simple python module to generate anchor (aka default/prior) boxes for object detection tasks.

Picasso: A CUDA-based Library for Deep Learning over 3D Meshes

Zeyuan Chen, Yangchao Wang, Yang Yang and Dong Liu.

Usable Implementation of "Bootstrap Your Own Latent" self-supervised learning, from Deepmind, in Pytorch

Aspect-Sentiment-Multiple-Opinion Triplet Extraction (NLPCC 2021)

Robust fine-tuning of zero-shot models

Selene is a Python library and command line interface for training deep neural networks from biological sequence data such as genomes.

ULMFiT for Genomic Sequence Data

Blind visual quality assessment on 360° Video based on progressive learning

Python implementation of Wu et al (2018)'s registration fusion

Angora is a mutation-based fuzzer. The main goal of Angora is to increase branch coverage by solving path constraints without symbolic execution.

BrainGNN - A deep learning model for data-driven discovery of functional connectivity

official implemntation for "Contrastive Learning with Stronger Augmentations"

Learning RGB-D Feature Embeddings for Unseen Object Instance Segmentation

Code and experiments for "Deep Neural Networks for Rank Consistent Ordinal Regression based on Conditional Probabilities"

YOLOX Win10 Project

nfelo: a power ranking, prediction, and betting model for the NFL

SlotRefine: A Fast Non-Autoregressive Model forJoint Intent Detection and Slot Filling

Perturb-and-max-product: Sampling and learning in discrete energy-based models

Datasets, Transforms and Models specific to Computer Vision