Code for paper: Towards Tokenized Human Dynamics Representation

Last update: May 31, 2022

Overview

Video Tokneization

Codebase for video tokenization, based on our paper Towards Tokenized Human Dynamics Representation.

Prerequisites (tested under Python 3.8 and CUDA 11.1)

apt-get install ffmpeg  
pip install torch==1.8  
pip install torchvision  
pip install pytorch-lightning  
pip install pytorch-lightning-bolts  
pip install aniposelib wandb gym test-tube ffmpeg-python matplotlib easydict scikit-learn

Data Preparation

Make a directory besides this repo and name it aistplusplus
Download from AIST++ website until it looks like

├── annotations
│   ├── cameras
│   ├── ignore_list.txt
│   ├── keypoints2d
│   ├── keypoints3d
│   ├── motions
│   └── splits
└── video_list.txt

How to run

Write one configuration file, e.g., configs/tan.yaml.
Run python pretrain.py --cfg configs/tan.yaml with GPU, which will create a folder under logs for this run. Folder name specified by the NAME in configuration file. Then run python cluster.py --cfg configs/tan.yaml (CPU-only) and check results in demo.ipynb.
Or you can download and unzip my training result into logs folder from here.

Code for paper: Towards Tokenized Human Dynamics Representation

Related tags

Overview

Video Tokneization

Prerequisites (tested under Python 3.8 and CUDA 11.1)

Data Preparation

How to run

Owner

Kenneth Li

MVSDF - Learning Signed Distance Field for Multi-view Surface Reconstruction

Detection of drones using their thermal signatures from thermal camera through YOLO-V3 based CNN with modifications to encapsulate drone motion

Audio Domain Adaptation for Acoustic Scene Classification using Disentanglement Learning

Position detection system of mobile robot in the warehouse enviroment

Multi-Object Tracking in Satellite Videos with Graph-Based Multi-Task Modeling

This repository contains tutorials for the py4DSTEM Python package

Python parser for DTED data.

Tensorflow Implementation for "Pre-trained Deep Convolution Neural Network Model With Attention for Speech Emotion Recognition"

NP DRAW paper released code

Implementation of Vision Transformer, a simple way to achieve SOTA in vision classification with only a single transformer encoder, in Pytorch

Code for the paper “The Peril of Popular Deep Learning Uncertainty Estimation Methods”

PyTorch implementation of popular datasets and models in remote sensing

This project hosts the code for implementing the ISAL algorithm for object detection and image classification

Implementation of "Glancing Transformer for Non-Autoregressive Neural Machine Translation"

[AAAI 2022] Sparse Structure Learning via Graph Neural Networks for Inductive Document Classification

4st place solution for the PBVS 2022 Multi-modal Aerial View Object Classification Challenge - Track 1 (SAR) at PBVS2022

Pytorch implementation of One-Shot Affordance Detection

Preprossing-loan-data-with-NumPy - In this project, I have cleaned and pre-processed the loan data that belongs to an affiliate bank based in the United States.

The official implementation of A Unified Game-Theoretic Interpretation of Adversarial Robustness.

Improving Factual Consistency of Abstractive Text Summarization