(AAAI2020)Grapy-ML: Graph Pyramid Mutual Learning for Cross-dataset Human Parsing

Overview

Grapy-ML: Graph Pyramid Mutual Learning for Cross-dataset Human Parsing

This repository contains pytorch source code for AAAI2020 oral paper: Grapy-ML: Graph Pyramid Mutual Learning for Cross-dataset Human Parsing by Haoyu He, Jing Zhang, Qiming Zhang and Dacheng Tao.


Grapy-ML:

GPM


Getting Started:

Environment:

  • Pytorch = 1.1.0

  • torchvision

  • scipy

  • tensorboardX

  • numpy

  • opencv-python

  • matplotlib

Data Preparation:

You need to download the three datasets. The CIHP dataset and ATR dataset can be found in this repository and our code is heavily borrowed from it as well.

Then, the datasets should be arranged in the following folder, and images should be rearranged with the provided file structure.

/data/dataset/

Testing:

The pretrain models and some trained models are provided here for testing and training.

Model Name Description Derived from
deeplab_v3plus_v3.pth The Deeplab v3+'s pretrain weights
CIHP_pretrain.pth The reproduced Deeplab v3+ model trained on CIHP dataset deeplab_v3plus_v3.pth
CIHP_trained.pth GPM model trained on CIHP dataset CIHP_pretrain.pth
deeplab_multi-dataset.pth The reproduced multi-task learning Deeplab v3+ model trained on CIHP, PASCAL-Person-Part and ATR dataset deeplab_v3plus_v3.pth
GPM-ML_multi-dataset.pth Grapy-ML model trained on CIHP, PASCAL-Person-Part and ATR dataset deeplab_multi-dataset.pth
GPM-ML_finetune_PASCAL.pth Grapy-ML model finetuned on PASCAL-Person-Part dataset GPM-ML_multi-dataset.pth

To test, run the following two scripts:

bash eval_gpm.sh
bash eval_gpm_ml.sh

Training:

GPM:

During training, you first need to get the Deeplab pretrain model(e.g. CIHP_dlab.pth) on each dataset. Such act aims to provide a trustworthy initial raw result for the GSA operation in GPM.

bash train_dlab.sh

The imageNet pretrain model is provided in the following table, and you should swith the dataset name and target classes to the dataset you want in the script. (CIHP: 20 classes, PASCAL: 7 classes and ATR: 18 classes)

In the next step, you should utilize the Deeplab pretrain model to further train the GPM model.

bash train_gpm.sh 

It is recommended to follow the training settings in our paper to reproduce the results.

GPM-ML:

Firstly, you can conduct the deeplab pretrain process by the following script:

bash train_dlab_ml.sh

The multi-dataset Deeplab V3+ is transformed as a simple multi-task task.

Then, you can train the GPM-ML model with the training set from all three datasets by:

bash train_gpm_ml_all.sh

After this phase, the first two levels of the GPM-ML model would be more robust and generalized.

Finally, you can try to finetune on each dataset by the unified pretrain model.

bash train_gpm_ml_pascal.sh

Citation:

@inproceedings{he2020grapy,
title={Grapy-ML: Graph Pyramid Mutual Learning for Cross-dataset Human Parsing},
author={He, Haoyu and Zhang, Jing and Zhang, Qiming and Tao, Dacheng},
booktitle={Proceedings of the AAAI Conference on Artificial Intelligence},
year={2020}
}

Maintainer:

[email protected]

Exploring Versatile Prior for Human Motion via Motion Frequency Guidance (3DV2021)

Exploring Versatile Prior for Human Motion via Motion Frequency Guidance This is the codebase for video-based human motion reconstruction in human-mot

Jiachen Xu 5 Jul 14, 2022
Language Models for the legal domain in Spanish done @ BSC-TEMU within the "Plan de las Tecnologías del Lenguaje" (Plan-TL).

Spanish legal domain Language Model ⚖️ This repository contains the page for two main resources for the Spanish legal domain: A RoBERTa model: https:/

Plan de Tecnologías del Lenguaje - Gobierno de España 12 Nov 14, 2022
Transport Mode detection - can detect the mode of transport with the help of features such as acceeration,jerk etc

title emoji colorFrom colorTo sdk app_file pinned Transport_Mode_Detector 🚀 purple yellow gradio app.py false Configuration title: string Display tit

Nishant Rajadhyaksha 3 Jan 16, 2022
Official implementation of the paper Visual Parser: Representing Part-whole Hierarchies with Transformers

Visual Parser (ViP) This is the official implementation of the paper Visual Parser: Representing Part-whole Hierarchies with Transformers. Key Feature

Shuyang Sun 117 Dec 11, 2022
This repository contains the code for the paper Neural RGB-D Surface Reconstruction

Neural RGB-D Surface Reconstruction Paper | Project Page | Video Neural RGB-D Surface Reconstruction Dejan Azinović, Ricardo Martin-Brualla, Dan B Gol

Dejan 406 Jan 04, 2023
An interpreter for RASP as described in the ICML 2021 paper "Thinking Like Transformers"

RASP Setup Mac or Linux Run ./setup.sh . It will create a python3 virtual environment and install the dependencies for RASP. It will also try to insta

141 Jan 03, 2023
Source Code for AAAI 2022 paper "Graph Convolutional Networks with Dual Message Passing for Subgraph Isomorphism Counting and Matching"

Graph Convolutional Networks with Dual Message Passing for Subgraph Isomorphism Counting and Matching This repository is an official implementation of

HKUST-KnowComp 13 Sep 08, 2022
PyTorch Implementation of our paper Explain Me the Painting: Multi-Topic Knowledgeable Art Description Generation

PyTorch Implementation of our paper Explain Me the Painting: Multi-Topic Knowledgeable Art Description Generation

Zechen Bai 12 Jul 08, 2022
A concise but complete implementation of CLIP with various experimental improvements from recent papers

x-clip (wip) A concise but complete implementation of CLIP with various experimental improvements from recent papers Install $ pip install x-clip Usag

Phil Wang 515 Dec 26, 2022
Hashformers is a framework for hashtag segmentation with transformers.

Hashtag segmentation is the task of automatically inserting the missing spaces between the words in a hashtag. Hashformers applies Transformer models

Ruan Chaves 41 Nov 09, 2022
Research code for Arxiv paper "Camera Motion Agnostic 3D Human Pose Estimation"

GMR(Camera Motion Agnostic 3D Human Pose Estimation) This repo provides the source code of our arXiv paper: Seong Hyun Kim, Sunwon Jeong, Sungbum Park

Seong Hyun Kim 1 Feb 07, 2022
Mask2Former: Masked-attention Mask Transformer for Universal Image Segmentation in TensorFlow 2

Mask2Former: Masked-attention Mask Transformer for Universal Image Segmentation in TensorFlow 2 Bowen Cheng, Ishan Misra, Alexander G. Schwing, Alexan

Phan Nguyen 1 Dec 16, 2021
Live Hand Tracking Using Python

Live-Hand-Tracking-Using-Python Project Description: In this project, we will be

Hassan Shahzad 2 Jan 06, 2022
pytorch implementation of openpose including Hand and Body Pose Estimation.

pytorch-openpose pytorch implementation of openpose including Body and Hand Pose Estimation, and the pytorch model is directly converted from openpose

Hzzone 1.4k Jan 07, 2023
Code for the paper "Spatio-temporal Self-Supervised Representation Learning for 3D Point Clouds" (ICCV 2021)

Spatio-temporal Self-Supervised Representation Learning for 3D Point Clouds This is the official code implementation for the paper "Spatio-temporal Se

Hesper 63 Jan 05, 2023
Code for "Contextual Non-Local Alignment over Full-Scale Representation for Text-Based Person Search"

Contextual Non-Local Alignment over Full-Scale Representation for Text-Based Person Search This is an implementation for our paper Contextual Non-Loca

Tencent YouTu Research 50 Dec 03, 2022
A system used to detect whether a person is wearing a medical mask or not.

Mask_Detection_System A system used to detect whether a person is wearing a medical mask or not. To open the program, please follow these steps: Make

Mohamed Emad 0 Nov 17, 2022
Image-retrieval-baseline - MUGE Multimodal Retrieval Baseline

MUGE Multimodal Retrieval Baseline This repo is implemented based on the open_cl

47 Dec 16, 2022
FTIR-Deep Learning - FTIR Deep Learning With Python

CANDIY-spectrum Human analyis of chemical spectra such as Mass Spectra (MS), Inf

Wei Mei 1 Jan 03, 2022
Space Invaders For Python

Space-Invaders Just download or clone the git repository. To run the Space Invader game you need to have pyhton installed in you system. If you dont h

Fei 5 Jul 27, 2022