Research code for CVPR 2021 paper "End-to-End Human Pose and Mesh Reconstruction with Transformers"

Overview

MeshTransformer โœจ

This is our research code of End-to-End Human Pose and Mesh Reconstruction with Transformers.

MEsh TRansfOrmer is a simple yet effective transformer-based method for human pose and mesh reconsruction from an input image. In this repository, we provide our research code for training and testing our proposed method for the following tasks:

  • Human pose and mesh reconstruction
  • Hand pose and mesh reconstruction

Installation

Check INSTALL.md for installation instructions.

Model Zoo and Download

Please download our pre-trained models and other relevant files that are important to run our code.

Check DOWNLOAD.md for details.

Quick demo

We provide demo codes to run end-to-end inference on the test images.

Check DEMO.md for details.

Experiments

We provide python codes for training and evaluation.

Check EXP.md for details.

Contributing

We welcome contributions and suggestions. Please check CONTRIBUTE and CODE_OF_CONDUCT for details.

Citations

If you find our work useful in your research, please consider citing:

@inproceedings{lin2021end-to-end,
author = {Lin, Kevin and Wang, Lijuan and Liu, Zicheng},
title = {End-to-End Human Pose and Mesh Reconstruction with Transformers},
booktitle = {CVPR},
year = {2021},
}

License

Our research code is released under the MIT license. See LICENSE for details.

We use submodules from third parties, such as huggingface/transformers and hassony2/manopth. Please see NOTICE for details.

We note that any use of SMPL models and MANO models are subject to Software Copyright License for non-commercial scientific research purposes. See SMPL-Model License and MANO License for details.

Acknowledgments

Our implementation and experiments are built on top of open-source GitHub repositories. We thank all the authors who made their code public, which tremendously accelerates our project progress. If you find these works helpful, please consider citing them as well.

huggingface/transformers

HRNet/HRNet-Image-Classification

nkolot/GraphCMR

akanazawa/hmr

MandyMo/pytorch_HMR

hassony2/manopth

hongsukchoi/Pose2Mesh_RELEASE

mks0601/I2L-MeshNet_RELEASE

open-mmlab/mmdetection

Owner
Microsoft
Open source projects and samples from Microsoft
Microsoft
A trashy useless Latin programming language written in python.

Codigum! The first programming langage in latin! (please keep your eyes closed when if you read the source code) It is pretty useless though. Document

Bic 2 Oct 25, 2021
ICLR21 Tent: Fully Test-Time Adaptation by Entropy Minimization

โ›บ๏ธ Tent: Fully Test-Time Adaptation by Entropy Minimization This is the official project repository for Tent: Fully-Test Time Adaptation by Entropy Mi

Dequan Wang 204 Dec 25, 2022
This program uses trial auth token of Azure Cognitive Services to do speech synthesis for you.

๐Ÿ—ฃ๏ธ aspeak A simple text-to-speech client using azure TTS API(trial). ๐Ÿ˜† TL;DR: This program uses trial auth token of Azure Cognitive Services to do s

Levi Zim 359 Jan 05, 2023
Towards Flexible Blind JPEG Artifacts Removal (FBCNN, ICCV 2021)

Towards Flexible Blind JPEG Artifacts Removal (FBCNN, ICCV 2021) Jiaxi Jiang, Kai Zhang, Radu Timofte Computer Vision Lab, ETH Zurich, Switzerland ๐Ÿ”ฅ

Jiaxi Jiang 282 Jan 02, 2023
BboxToolkit is a tiny library of special bounding boxes.

BboxToolkit is a light codebase collecting some practical functions for the special-shape detection, such as oriented detection

jbwang1997 73 Jan 01, 2023
Temporal Segment Networks (TSN) in PyTorch

TSN-Pytorch We have released MMAction, a full-fledged action understanding toolbox based on PyTorch. It includes implementation for TSN as well as oth

1k Jan 03, 2023
TensorFlow implementation of Elastic Weight Consolidation

Elastic weight consolidation Introduction A TensorFlow implementation of elastic weight consolidation as presented in Overcoming catastrophic forgetti

James Stokes 67 Oct 11, 2022
Unofficial JAX implementations of Deep Learning models

JAX Models Table of Contents About The Project Getting Started Prerequisites Installation Usage Contributing License Contact About The Project The JAX

107 Jan 05, 2023
The project is an official implementation of our paper "3D Human Pose Estimation with Spatial and Temporal Transformers".

3D Human Pose Estimation with Spatial and Temporal Transformers This repo is the official implementation for 3D Human Pose Estimation with Spatial and

Ce Zheng 363 Dec 28, 2022
This is the repository for the NeurIPS-21 paper [Contrastive Graph Poisson Networks: Semi-Supervised Learning with Extremely Limited Labels].

CGPN This is the repository for the NeurIPS-21 paper [Contrastive Graph Poisson Networks: Semi-Supervised Learning with Extremely Limited Labels]. Req

10 Sep 12, 2022
Reinforcement Learning for Automated Trading

Reinforcement Learning for Automated Trading This thesis has been realized for the obtention of the Master's in Mathematical Engineering at the Polite

Pierpaolo Necchi 80 Jun 19, 2022
Simple tutorials using Google's TensorFlow Framework

TensorFlow-Tutorials Introduction to deep learning based on Google's TensorFlow framework. These tutorials are direct ports of Newmu's Theano Tutorial

Nathan Lintz 6k Jan 06, 2023
An algorithm study of the 6th iOS 10 set of Boost Camp Web Mobile

์•Œ๊ณ ๋ฆฌ์ฆ˜ ์Šคํ„ฐ๋”” ๐Ÿ”ฅ ๋ถ€์ŠคํŠธ์บ ํ”„ ์›น๋ชจ๋ฐ”์ผ 6๊ธฐ iOS 10์กฐ์˜ ์•Œ๊ณ ๋ฆฌ์ฆ˜ ์Šคํ„ฐ๋”” ์ž…๋‹ˆ๋‹ค. ๊ฐœ์ธ์ ์ธ ์‚ฌ์ • ๋“ฑ์œผ๋กœ S034, S055๋งŒ ์ฐธ๊ฐ€ํ•˜์˜€์Šต๋‹ˆ๋‹ค. ์Šคํ„ฐ๋”” ๋ชฉ์  ์ƒ์ง„: ์ฝ”ํ…Œ ํ•ฉ๊ฒฉ + ๋ถ€์บ ๋๋‚˜๊ณ  ์•„์นจ์— ์ผ์–ด๋‚˜๊ธฐ ์œ„ํ•ด ํ•„์š”ํ•œ ์‚ฌ์ดํด ๊ธฐ์™„: ๊พธ์ค€ํ•˜๊ฒŒ ์ž๋ฆฌ์— ์•‰์•„ ๊ณต๋ถ€ํ•˜๊ธฐ +

2 Jan 11, 2022
Implementation of "StrengthNet: Deep Learning-based Emotion Strength Assessment for Emotional Speech Synthesis"

StrengthNet Implementation of "StrengthNet: Deep Learning-based Emotion Strength Assessment for Emotional Speech Synthesis" https://arxiv.org/abs/2110

RuiLiu 65 Dec 20, 2022
Permute Me Softly: Learning Soft Permutations for Graph Representations

Permute Me Softly: Learning Soft Permutations for Graph Representations

Giannis Nikolentzos 7 Jul 10, 2022
Strongly local p-norm-cut algorithms for semi-supervised learning and local graph clustering

Strongly local p-norm-cut algorithms for semi-supervised learning and local graph clustering

Meng Liu 2 Jul 19, 2022
Implementations for the ICLR-2021 paper: SEED: Self-supervised Distillation For Visual Representation.

Implementations for the ICLR-2021 paper: SEED: Self-supervised Distillation For Visual Representation.

Jacob 27 Oct 23, 2022
Node-level Graph Regression with Deep Gaussian Process Models

Node-level Graph Regression with Deep Gaussian Process Models Prerequests our implementation is mainly based on tensorflow 1.x and gpflow 1.x: python

1 Jan 16, 2022
This program creates a formatted excel file which highlights the undervalued stock according to Graham's number.

Over-and-Undervalued-Stocks Of Nepse Using Graham's Number Scrap the latest data using different websites and creates a formatted excel file that high

6 May 03, 2022
Cross-view Transformers for real-time Map-view Semantic Segmentation (CVPR 2022 Oral)

Cross View Transformers This repository contains the source code and data for our paper: Cross-view Transformers for real-time Map-view Semantic Segme

Brady Zhou 363 Dec 25, 2022