Neural Residual Flow Fields for Efficient Video Representations

1. Download MPI sintel dataset

Download MPI sintel dataset from here

2. GMA optical flow estimator

To obtain optical flow estimations for pretraining, we are using GMA from here. Note that it dose not have to do with our identity.

3. Training

Training neural residual flow fields (NRFF)

# frame 0 - 6
python train_video_flow_midkey.py --use-estimator --lr 0.0005 --training-step 30000 --data-dir {sintel dataset training directory} --video-name alley_1 --start-frame 0 --num-frames 7 --jpeg-quality 98 --hidden-features 96 --use-estimator --tag start0_jq98_hf96
# frame 7 - 13
python train_video_flow_midkey.py --use-estimator --lr 0.0005 --training-step 30000 --data-dir {sintel dataset training directory} --video-name alley_1 --start-frame 7 --num-frames 7 --jpeg-quality 98 --hidden-features 96 --use-estimator --tag start7_jq98_hf96
# frame 14 - 20
python train_video_flow_midkey.py --use-estimator --lr 0.0005 --training-step 30000 --data-dir {sintel dataset training directory} --video-name alley_1 --start-frame 14 --num-frames 7 --jpeg-quality 98 --hidden-features 96 --use-estimator --tag start14_jq98_hf96
# frame 21 - 27
python train_video_flow_midkey.py --use-estimator --lr 0.0005 --training-step 30000 --data-dir {sintel dataset training directory} --video-name alley_1 --start-frame 21 --num-frames 7 --jpeg-quality 98 --hidden-features 96 --use-estimator --tag start21_jq98_hf96

Training baseline (SIREN)

python train_video.py --data-dir {sintel dataset training directory} --video-name alley_1 --hidden-features 256 --num-frames 28 --lr 0.001 --training-step 30000 --tag baseline_siren_hf256

4. Examples

alley_2.mp4

HoneyBee.mp4

Eff video representation - Efficient video representation through neural fields

Related tags

Overview

Neural Residual Flow Fields for Efficient Video Representations

1. Download MPI sintel dataset

2. GMA optical flow estimator

3. Training

4. Examples

Owner

Unsupervised Representation Learning by Invariance Propagation

SPRING is a seq2seq model for Text-to-AMR and AMR-to-Text (AAAI2021).

Real-world Anomaly Detection in Surveillance Videos- pytorch Re-implementation

Code for all the Advent of Code'21 challenges mostly written in python

Simple-Image-Classification - Simple Image Classification Code (PyTorch)

People Interaction Graph

Simple torch.nn.module implementation of Alias-Free-GAN style filter and resample

Meli Data Challenge 2021 - First Place Solution

这是一个mobilenet-yolov4-lite的库，把yolov4主干网络修改成了mobilenet，修改了Panet的卷积组成，使参数量大幅度缩小。

Computer Vision is an elective course of MSAI, SCSE, NTU, Singapore

Breast-Cancer-Prediction

Attentional Focus Modulates Automatic Finger‑tapping Movements

a reimplementation of LiteFlowNet in PyTorch that matches the official Caffe version

CVPR 2022 "Online Convolutional Re-parameterization"

Code for the paper: Adversarial Machine Learning: Bayesian Perspectives

Only a Matter of Style: Age Transformation Using a Style-Based Regression Model

Attention-guided gan for synthesizing IR images

Learning Pixel-level Semantic Affinity with Image-level Supervision for Weakly Supervised Semantic Segmentation, CVPR 2018

ICNet for Real-Time Semantic Segmentation on High-Resolution Images, ECCV2018

ISTR: End-to-End Instance Segmentation with Transformers (https://arxiv.org/abs/2105.00637)