CTR-GCN
This repo is the official implementation for Channel-wise Topology Refinement Graph Convolution for Skeleton-Based Action Recognition. The paper is accepted to ICCV2021.
Note: We also provide a simple and strong baseline model, which achieves 83.7% on NTU120 CSub with joint modality only, to facilitate the development of skeleton-based action recognition.
Architecture of CTR-GC
Prerequisites
-
Python >= 3.6
-
PyTorch >= 1.1.0
-
PyYAML, tqdm, tensorboardX
-
We provide the dependency file of our experimental environment, you can install all dependencies by creating a new anaconda virtual environment and running
pip install -r requirements.txt -
Run
pip install -e torchlight
Data Preparation
Download datasets.
There are 3 datasets to download:
- NTU RGB+D 60 Skeleton
- NTU RGB+D 120 Skeleton
- NW-UCLA
NTU RGB+D 60 and 120
- Request dataset here: http://rose1.ntu.edu.sg/Datasets/actionRecognition.asp
- Download the skeleton-only datasets:
nturgbd_skeletons_s001_to_s017.zip(NTU RGB+D 60)nturgbd_skeletons_s018_to_s032.zip(NTU RGB+D 120)- Extract above files to
./data/nturgbd_raw
NW-UCLA
- Download dataset from here
- Move
all_sqeto./data/NW-UCLA
Data Processing
Directory Structure
Put downloaded data into the following directory structure:
- data/
- NW-UCLA/
- all_sqe
... # raw data of NW-UCLA
- ntu/
- ntu120/
- nturgbd_raw/
- nturgb+d_skeletons/ # from `nturgbd_skeletons_s001_to_s017.zip`
...
- nturgb+d_skeletons120/ # from `nturgbd_skeletons_s018_to_s032.zip`
...
Generating Data
- Generate NTU RGB+D 60 or NTU RGB+D 120 dataset:
cd ./data/ntu # or cd ./data/ntu120
# Get skeleton of each performer
python get_raw_skes_data.py
# Remove the bad skeleton
python get_raw_denoised_data.py
# Transform the skeleton to the center of the first frame
python seq_transformation.py
Training & Testing
Training
- Change the config file depending on what you want.
# Example: training CTRGCN on NTU RGB+D 120 cross subject with GPU 0
python main.py --config config/nturgbd120-cross-subject/default.yaml --work-dir work_dir/ntu120/csub/ctrgcn --device 0
# Example: training provided baseline on NTU RGB+D 120 cross subject
python main.py --config config/nturgbd120-cross-subject/default.yaml --model model.baseline.Model--work-dir work_dir/ntu120/csub/baseline --device 0
- To train model on NTU RGB+D 60/120 with bone or motion modalities, setting
boneorvelarguments in the config filedefault.yamlor in the command line.
# Example: training CTRGCN on NTU RGB+D 120 cross subject under bone modality
python main.py --config config/nturgbd120-cross-subject/default.yaml --train_feeder_args bone=True --test_feeder_args bone=True --work-dir work_dir/ntu120/csub/ctrgcn_bone --device 0
- To train model on NW-UCLA with bone or motion modalities, you need to modify
data_pathintrain_feeder_argsandtest_feeder_argsto "bone" or "motion" or "bone motion", and run
python main.py --config config/ucla/default.yaml --work-dir work_dir/ucla/ctrgcn_xxx --device 0
- To train your own model, put model file
your_model.pyunder./modeland run:
# Example: training your own model on NTU RGB+D 120 cross subject
python main.py --config config/nturgbd120-cross-subject/default.yaml --model model.your_model.Model --work-dir work_dir/ntu120/csub/your_model --device 0
Testing
- To test the trained models saved in <work_dir>, run the following command:
python main.py --config <work_dir>/config.yaml --work-dir <work_dir> --phase test --save-score True --weights <work_dir>/xxx.pt --device 0
- To ensemble the results of different modalities, run
# Example: ensemble four modalities of CTRGCN on NTU RGB+D 120 cross subject
python ensemble.py --datasets ntu120/xsub --joint-dir work_dir/ntu120/csub/ctrgcn --bone-dir work_dir/ntu120/csub/ctrgcn_bone --joint-motion-dir work_dir/ntu120/csub/ctrgcn_motion --bone-motion-dir work_dir/ntu120/csub/ctrgcn_bone_motion
Pretrained Models
- Download pretrained models for producing the final results on NTU RGB+D 60&120 cross subject [Google Drive].
- Put files to <work_dir> and run Testing command to produce the final result.
Acknowledgements
This repo is based on 2s-AGCN. The data processing is borrowed from SGN and HCN.
Thanks to the original authors for their work!
Citation
Please cite this work if you find it useful:.
@article{chen2021channel,
title={Channel-wise Topology Refinement Graph Convolution for Skeleton-Based Action Recognition},
author={Chen, Yuxin and Zhang, Ziqi and Yuan, Chunfeng and Li, Bing and Deng, Ying and Hu, Weiming},
journal={arXiv preprint arXiv:2107.12213},
year={2021}
}
Contact
For any questions, feel free to contact: [email protected]
32 Nov 17, 2022
1.5k Dec 31, 2022
789 Dec 29, 2022
46 Sep 13, 2022
1.7k Jan 04, 2023
41 Oct 03, 2022
859 Dec 22, 2022
1 Dec 19, 2022
16 Dec 25, 2022
1.7k Dec 28, 2022
265 Dec 22, 2022
1 Apr 30, 2021
343 Jan 03, 2023
2 Jun 18, 2022
2 Jan 20, 2022
3 Jan 04, 2022
72 Dec 24, 2022
343 Jan 06, 2023
107 Dec 29, 2022
1.2k Jan 04, 2023