ASFormer: Transformer for Action Segmentation

This repo provides training & inference code for BMVC 2021 paper: ASFormer: Transformer for Action Segmentation.

Enviroment

Pytorch == 1.1.0, torchvision == 0.3.0, python == 3.6, CUDA=10.1

Reproduce our results

1. Download the dataset data.zip at (https://mega.nz/#!O6wXlSTS!wcEoDT4Ctq5HRq_hV-aWeVF1_JB3cacQBQqOLjCIbc8) or (https://zenodo.org/record/3625992#.Xiv9jGhKhPY). 
2. Unzip the data.zip file to the current folder. There are three datasets in the ./data folder, i.e. ./data/breakfast, ./data/50salads, ./data/gtea
3. Download the pre-trained models at (https://pan.baidu.com/s/1zf-d-7eYqK-IxroBKTxDfg). There are pretrained models for three datasets, i.e. ./models/50salads, ./models/breakfast, ./models/gtea
4. Run python main.py --action=predict --dataset=50salads/gtea/breakfast --split=1/2/3/4/5 to generate predicted results for each split.
5. Run python eval.py --dataset=50salads/gtea/breakfast --split=0/1/2/3/4/5 to evaluate the performance. **NOTE**: split=0 will evaulate the average results for all splits, It needs to be done after you complete all split predictions.

Train your own model

Also, you can retrain the model by yourself with following command.

python main.py --action=train --dataset=50salads/gtea/breakfast --split=1/2/3/4/5

The training process is very stable in our experiments. It convergences very fast and is not sensitive to the number of training epochs.

Demo for using ASFormer as your backbone

In our paper, we replace the original TCN-based backbone model MS-TCN in ASRF with our ASFormer. The new model achieves even higher results on the 50salads dataset than the original ASRF. Code is Here.

If you find our repo useful, please give us a star and cite

@inproceedings{chinayi_ASformer,  
	author={Fangqiu Yi and Hongyu Wen and Tingting Jiang}, 
	booktitle={The British Machine Vision Conference (BMVC)},   
	title={ASFormer: Transformer for Action Segmentation},
	year={2021},  
}

Feel free to raise a issue if you got trouble with our code.

Official repo for BMVC2021 paper ASFormer: Transformer for Action Segmentation

Related tags

Overview

ASFormer: Transformer for Action Segmentation

Enviroment

Reproduce our results

Train your own model

Demo for using ASFormer as your backbone

Owner

Fast Axiomatic Attribution for Neural Networks (NeurIPS*2021)

Some pvbatch (paraview) scripts for postprocessing OpenFOAM data

A flexible tool for creating, organizing, and sharing visualizations of live, rich data. Supports Torch and Numpy.

MNE: Magnetoencephalography (MEG) and Electroencephalography (EEG) in Python

A custom-designed Spider Robot trained to walk using Deep RL in a PyBullet Simulation

A simple baseline for 3d human pose estimation in tensorflow. Presented at ICCV 17.

Vehicle speed detection with python

Self-Guided Contrastive Learning for BERT Sentence Representations

PyTorch implementation of "Conformer: Convolution-augmented Transformer for Speech Recognition" (INTERSPEECH 2020)

This is an official implementation for "Swin Transformer: Hierarchical Vision Transformer using Shifted Windows" on Object Detection and Instance Segmentation.

Effective Use of Transformer Networks for Entity Tracking

EZ graph is an easy to use AI solution that allows you to make and train your neural networks without a single line of code.

CVPR2020 Counterfactual Samples Synthesizing for Robust VQA

GANSketchingJittor - Implementation of Sketch Your Own GAN in Jittor

Implement Decoupled Neural Interfaces using Synthetic Gradients in Pytorch

[NeurIPS 2020] Official Implementation: "SMYRF: Efficient Attention using Asymmetric Clustering".

Gauge equivariant mesh cnn

This is the code of using DQN to play Sekiro .

Solving Zero-Shot Learning in Named Entity Recognition with Common Sense Knowledge

A scanpy extension to analyse single-cell TCR and BCR data.