PyTorch implementation of "Optimization Planning for 3D ConvNets"

Overview

Optimization-Planning-for-3D-ConvNets

Code for the ICML 2021 paper: Optimization Planning for 3D ConvNets.

Authors: Zhaofan Qiu, Ting Yao, Chong-Wah Ngo, Tao Mei

Framework

1. Requirement

The provided codes have been tested with Python-3.9.5 & Pytorch-1.9.0 on four Tesla-V100s.

2. Project structure

├─ base_config             # Pre-set config file for each dataset
├─ dataset                 # Video lists (NOT provided) and code to load video data
├─ jpgs                    # Images for README
├─ layers                  # Custom network layers
├─ model                   # Network architectures
├─ record                  # Config file for each run
├─ utils                   # Basic functions
├─ extract_score_3d.py     # Main script to extract predicted score
├─ helpers.py              # Helper functions for main scripts
├─ merge_score.py          # Main script to merge scores from different clips
├─ train_3d.py             # Main script to launch a training using given strategy
├─ train_3d_op.py          # Main script to launch a searching of best strategy
└─ run.sh                  # Shell script for training-extracting-merging pipeline

3. Run the code

  1. Pre-process the target dataset and put the lists in to the dataset folder. Codes in dataset/video_dataset.py can load three video formats (raw video, jpeg frames and video LMDB) and can be simply modified to support the custom format.
  2. Make config file in the record folder. The config examples include op-*.yml for pre-searched strategy, kinetics-*.yml for simple strategy on Kinetics-400,
  3. Run run.sh for the training-extracting-merging pipeline or replace train_3d.py with train_3d_op.py for searching the optimal strategy.

4. TO DO

Add more explainations and examples.

5. Contact

Please feel free to email to Zhaofan Qiu if you have any question regarding the paper or any suggestions for further improvements.

6. Citation

If you find this code helpful, thanks for citing our work as

@inproceedings{qiu2021optimization,
title={Optimization Planning for 3D ConvNets},
author={Qiu, Zhaofan and Yao, Ting and Ngo, Chong-Wah and Mei, Tao},
booktitle={Proceedings of the 38th International Conference on Machine Learning (ICML)},
publisher={PMLR},
year={2021}
}

Please also pay attention to the citations of the included networks/algorithms.

Owner
Zhaofan Qiu
Ph.D. student in USTC&MSRA
Zhaofan Qiu
StyleGAN2 with adaptive discriminator augmentation (ADA) - Official TensorFlow implementation

StyleGAN2 with adaptive discriminator augmentation (ADA) — Official TensorFlow implementation Training Generative Adversarial Networks with Limited Da

NVIDIA Research Projects 1.7k Dec 29, 2022
Source code for paper: Knowledge Inheritance for Pre-trained Language Models

Knowledge-Inheritance Source code paper: Knowledge Inheritance for Pre-trained Language Models (preprint). The trained model parameters (in Fairseq fo

THUNLP 31 Nov 19, 2022
A pytorch &keras implementation and demo of Fastformer.

Fastformer Notes from the authors Pytorch/Keras implementation of Fastformer. The keras version only includes the core fastformer attention part. The

153 Dec 28, 2022
Pre-Training 3D Point Cloud Transformers with Masked Point Modeling

Point-BERT: Pre-Training 3D Point Cloud Transformers with Masked Point Modeling Created by Xumin Yu*, Lulu Tang*, Yongming Rao*, Tiejun Huang, Jie Zho

Lulu Tang 306 Jan 06, 2023
Minimal diffusion models - Minimal code and simple experiments to play with Denoising Diffusion Probabilistic Models (DDPMs)

Minimal code and simple experiments to play with Denoising Diffusion Probabilist

Rithesh Kumar 16 Oct 06, 2022
Campsite Reservation Finder

yellowstone-camping UPDATE: yellowstone-camping is being expanded and renamed to camply. The updated tool now interfaces with the Recreation.gov API a

Justin Flannery 233 Jan 08, 2023
Code for Deterministic Neural Networks with Appropriate Inductive Biases Capture Epistemic and Aleatoric Uncertainty

Deep Deterministic Uncertainty This repository contains the code for Deterministic Neural Networks with Appropriate Inductive Biases Capture Epistemic

Jishnu Mukhoti 69 Nov 28, 2022
SOLOv2 on onnx & tensorRT

SOLOv2.tensorRT: NOTE: code based on WXinlong/SOLO add support to TensorRT inference onnxruntime tensorRT full_dims and dynamic shape postprocess with

47 Nov 26, 2022
Large-scale open domain KNOwledge grounded conVERsation system based on PaddlePaddle

Knover Knover is a toolkit for knowledge grounded dialogue generation based on PaddlePaddle. Knover allows researchers and developers to carry out eff

607 Dec 31, 2022
PyTorch implementation of the paper Dynamic Token Normalization Improves Vision Transfromers.

Dynamic Token Normalization Improves Vision Transformers This is the PyTorch implementation of the paper Dynamic Token Normalization Improves Vision T

Wenqi Shao 20 Oct 09, 2022
Towards Boosting the Accuracy of Non-Latin Scene Text Recognition

Convolutional Recurrent Neural Network + CTCLoss | STAR-Net Code for paper "Towards Boosting the Accuracy of Non-Latin Scene Text Recognition" Depende

Sanjana Gunna 7 Aug 07, 2022
StyleGAN2 Webtoon / Anime Style Toonify

StyleGAN2 Webtoon / Anime Style Toonify Korea Webtoon or Japanese Anime Character Stylegan2 base high Quality 1024x1024 / 512x512 Generate and Transfe

121 Dec 21, 2022
Official implementation of Neural Bellman-Ford Networks (NeurIPS 2021)

NBFNet: Neural Bellman-Ford Networks This is the official codebase of the paper Neural Bellman-Ford Networks: A General Graph Neural Network Framework

MilaGraph 136 Dec 21, 2022
This is the code used in the paper "Entity Embeddings of Categorical Variables".

This is the code used in the paper "Entity Embeddings of Categorical Variables". If you want to get the original version of the code used for the Kagg

Cheng Guo 845 Nov 29, 2022
Create images and texts with the First Order Generative Adversarial Networks

First Order Divergence for training GANs This repository contains code accompanying the paper First Order Generative Advesarial Netoworks The majority

Zalando Research 35 Dec 11, 2021
FS2KToolbox FS2K Dataset Towards the translation between Face

FS2KToolbox FS2K Dataset Towards the translation between Face -- Sketch. Download (photo+sketch+annotation): Google-drive, Baidu-disk, pw: FS2K. For

Deng-Ping Fan 5 Jan 03, 2023
A collection of implementations of deep domain adaptation algorithms

Deep Transfer Learning on PyTorch This is a PyTorch library for deep transfer learning. We divide the code into two aspects: Single-source Unsupervise

Yongchun Zhu 647 Jan 03, 2023
Implementation of Nalbach et al. 2017 paper.

Deep Shading Convolutional Neural Networks for Screen-Space Shading Our project is based on Nalbach et al. 2017 paper. In this project, a set of buffe

Marcel Santana 17 Sep 08, 2022
This is a Machine Learning Based Hand Detector Project, It Uses Machine Learning Models and Modules Like Mediapipe, Developed By Google!

Machine Learning Hand Detector This is a Machine Learning Based Hand Detector Project, It Uses Machine Learning Models and Modules Like Mediapipe, Dev

Popstar Idhant 3 Feb 25, 2022
SplineConv implementation for Paddle.

SplineConv implementation for Paddle This module implements the SplineConv operators from Matthias Fey, Jan Eric Lenssen, Frank Weichert, Heinrich Mül

北海若 3 Dec 29, 2021