Neural Residual Flow Fields for Efficient Video Representations

1. Download MPI sintel dataset

Download MPI sintel dataset from here

2. GMA optical flow estimator

To obtain optical flow estimations for pretraining, we are using GMA from here. Note that it dose not have to do with our identity.

3. Training

Training neural residual flow fields (NRFF)

# frame 0 - 6
python train_video_flow_midkey.py --use-estimator --lr 0.0005 --training-step 30000 --data-dir {sintel dataset training directory} --video-name alley_1 --start-frame 0 --num-frames 7 --jpeg-quality 98 --hidden-features 96 --use-estimator --tag start0_jq98_hf96
# frame 7 - 13
python train_video_flow_midkey.py --use-estimator --lr 0.0005 --training-step 30000 --data-dir {sintel dataset training directory} --video-name alley_1 --start-frame 7 --num-frames 7 --jpeg-quality 98 --hidden-features 96 --use-estimator --tag start7_jq98_hf96
# frame 14 - 20
python train_video_flow_midkey.py --use-estimator --lr 0.0005 --training-step 30000 --data-dir {sintel dataset training directory} --video-name alley_1 --start-frame 14 --num-frames 7 --jpeg-quality 98 --hidden-features 96 --use-estimator --tag start14_jq98_hf96
# frame 21 - 27
python train_video_flow_midkey.py --use-estimator --lr 0.0005 --training-step 30000 --data-dir {sintel dataset training directory} --video-name alley_1 --start-frame 21 --num-frames 7 --jpeg-quality 98 --hidden-features 96 --use-estimator --tag start21_jq98_hf96

Training baseline (SIREN)

python train_video.py --data-dir {sintel dataset training directory} --video-name alley_1 --hidden-features 256 --num-frames 28 --lr 0.001 --training-step 30000 --tag baseline_siren_hf256

4. Examples

alley_2.mp4

HoneyBee.mp4

Eff video representation - Efficient video representation through neural fields

Related tags

Overview

Neural Residual Flow Fields for Efficient Video Representations

1. Download MPI sintel dataset

2. GMA optical flow estimator

3. Training

4. Examples

Owner

Task-based end-to-end model learning in stochastic optimization

Efficient Multi Collection Style Transfer Using GAN

Lightweight, Portable, Flexible Distributed/Mobile Deep Learning with Dynamic, Mutation-aware Dataflow Dep Scheduler; for Python, R, Julia, Scala, Go, Javascript and more

Rethinking the Importance of Implementation Tricks in Multi-Agent Reinforcement Learning

Official implementation of "Articulation Aware Canonical Surface Mapping"

Space Ship Simulator using python

SuMa++: Efficient LiDAR-based Semantic SLAM (Chen et al IROS 2019)

EMNLP'2021: SimCSE: Simple Contrastive Learning of Sentence Embeddings

SingleVC performs any-to-one VC, which is an important component of MediumVC project.

Diffusion Probabilistic Models for 3D Point Cloud Generation (CVPR 2021)

Uncertain natural language inference

Official PyTorch implementation of the paper Image-Based CLIP-Guided Essence Transfer.

RM Operation can equivalently convert ResNet to VGG, which is better for pruning; and can help RepVGG perform better when the depth is large.

MMRazor: a model compression toolkit for model slimming and AutoML

Tensorflow2.0 🍎🍊 is delicious, just eat it! 😋😋

Official Implementation of "DialogLM: Pre-trained Model for Long Dialogue Understanding and Summarization."

Fake videos detection by tracing the source using video hashing retrieval.

Cours d'Algorithmique Appliquée avec Python pour BTS SIO SISR

E2EC: An End-to-End Contour-based Method for High-Quality High-Speed Instance Segmentation

Official git for "CTAB-GAN: Effective Table Data Synthesizing"