Code for Transformers Solve Limited Receptive Field for Monocular Depth Prediction

Last update: Dec 16, 2022

Related tags

Overview

Official PyTorch code for Transformers Solve Limited Receptive Field for Monocular Depth Prediction.
Guanglei Yang, Hao Tang, Mingli Ding, Nicu Sebe, Elisa Ricci.
Apply Transformer into depth predciton and surface normal estimation.

Prepare pretrain model

we choose R50-ViT-B_16 as our encoder.

wget https://storage.googleapis.com/vit_models/imagenet21k/R50+ViT-B_16.npz 
mkdir ./model/vit_checkpoint/imagenet21k 
mv R50+ViT-B_16.npz ./model/vit_checkpoint/imagenet21k/R50+ViT-B_16.npz

Prepare Dateset

prepare nyu

mkdir -p pytorch/dataset/nyu_depth_v2
python utils/download_from_gdrive.py 1AysroWpfISmm-yRFGBgFTrLy6FjQwvwP pytorch/dataset/nyu_depth_v2/sync.zip
cd pytorch/dataset/nyu_depth_v2
unzip sync.zip

prepare kitti

cd dataset
mkdir kitti_dataset
cd kitti_dataset
### image move kitti_archives_to_download.txt into kitti_dataset
wget -i kitti_archives_to_download.txt

### label
wget https://s3.eu-central-1.amazonaws.com/avg-kitti/data_depth_annotated.zip
unzip data_depth_annotated.zip
cd train
mv * ../
cd ..  
rm -r train
cd val
mv * ../
cd ..
rm -r val
rm data_depth_annotated.zip

Environment

pip install -r requirement.txt

Run

Train

CUDA_VISIBLE_DEVICES=0,1,2,3 python bts_main.py arguments_train_nyu.txt
CUDA_VISIBLE_DEVICES=0,1,2,3 python bts_main.py arguments_train_eigen.txt

Test: Pick up nice result

CUDA_VISIBLE_DEVICES=1 python bts_test.py arguments_test_nyu.txt
python ../utils/eval_with_pngs.py --pred_path vis_att_bts_nyu_v2_pytorch_att/raw/ --gt_path ../../dataset/nyu_depth_v2/official_splits/test/ --dataset nyu --min_depth_eval 1e-3 --max_depth_eval 10 --eigen_crop
CUDA_VISIBLE_DEVICES=1 python bts_test.py arguments_test_eigen.txt
python ../utils/eval_with_pngs.py --pred_path vis_att_bts_eigen_v2_pytorch_att/raw/ --gt_path ./dataset/kitti_dataset/ --dataset kitti --min_depth_eval 1e-3 --max_depth_eval 80 --do_kb_crop --garg_crop

Debug

CUDA_VISIBLE_DEVICES=1 python bts_main.py arguments_train_nyu_debug.txt

Download Pretrained Model

sh scripts/download_TransDepth_model.sh kitti_depth

sh scripts/download_TransDepth_model.sh nyu_depth

sh scripts/download_TransDepth_model.sh nyu_surfacenormal

Reference

BTS

ViT

Do‘s code

Visualization result share

We provide all vis result of all tasks. link

Code for Transformers Solve Limited Receptive Field for Monocular Depth Prediction

Related tags

Overview

Prepare pretrain model

Prepare Dateset

prepare nyu

prepare kitti

Environment

Run

Download Pretrained Model

Reference

Visualization result share

Owner

stanley

Implementation of algorithms for continuous control (DDPG and NAF).

This repository accompanies the ACM TOIS paper "What can I cook with these ingredients?" - Understanding cooking-related information needs in conversational search

App customer segmentation cohort rfm clustering

The dataset of tweets pulling from Twitters with keyword: Hydroxychloroquine, location: US, Time: 2020

SOFT: Softmax-free Transformer with Linear Complexity, NeurIPS 2021 Spotlight

DanceTrack: Multiple Object Tracking in Uniform Appearance and Diverse Motion

A lightweight deep network for fast and accurate optical flow estimation.

A DNN inference latency prediction toolkit for accurately modeling and predicting the latency on diverse edge devices.

FindFunc is an IDA PRO plugin to find code functions that contain a certain assembly or byte pattern, reference a certain name or string, or conform to various other constraints.

The repository forked from NVlabs uses our data. (Differentiable rasterization applied to 3D model simplification tasks)

Official pytorch implementation of paper "Image-to-image Translation via Hierarchical Style Disentanglement".

Implementation of Neural Style Transfer in Pytorch

Implementation of "Fast and Flexible Temporal Point Processes with Triangular Maps" (Oral @ NeurIPS 2020)

PECOS - Prediction for Enormous and Correlated Spaces

Pytorch and Keras Implementations of Hyperspectral Image Classification -- Traditional to Deep Models: A Survey for Future Prospects.

Training, generation, and analysis code for Learning Particle Physics by Example: Location-Aware Generative Adversarial Networks for Physics

Combining Reinforcement Learning and Constraint Programming for Combinatorial Optimization

Official Pytorch implementation of "CLIPstyler:Image Style Transfer with a Single Text Condition"

[ICCV 2021] Focal Frequency Loss for Image Reconstruction and Synthesis

Official codebase for Legged Robots that Keep on Learning: Fine-Tuning Locomotion Policies in the Real World