TransMVSNet: Global Context-aware Multi-view Stereo Network with Transformers.

Overview
Comments
  • abs_depth_error

    abs_depth_error

    I find ABS_DEPTH_ERROR is close to 6 or even 7 during training, is this normal? Here are the training results for Epoch 5. Is it because of the slow convergence?

    avg_test_scalars: {'loss': 4.360309665948113, 'depth_loss': 6.535046514014081, 'entropy_loss': 4.360309665948113, 'abs_depth_error': 6.899323051878795, 'thres2mm_error': 0.16829867261163733, 'thres4mm_error': 0.10954744909229193, 'thres8mm_error': 0.07844322964626443, 'thres14mm_error': 0.06323695212957076, 'thres20mm_error': 0.055751020700780536, 'thres2mm_abserror': 0.597563438798779, 'thres4mm_abserror': 2.7356186663791666, 'thres8mm_abserror': 5.608324628466483, 'thres14mm_abserror': 10.510002394554125, 'thres20mm_abserror': 16.67409769420184, 'thres>20mm_abserror': 78.15814284054947}

    opened by zhang-snowy 7
  • About the fusion setting in DTU

    About the fusion setting in DTU

    Thank you for your great contribution. The script use the gipuma as the fusion method with num_consistent=5prob_threshold=0.05disp_threshold=0.25. However, it produces point cloud results with only 1/2 points compared with the point cloud results you provide in DTU, leading to a much poorer result in DTU. Is there any setting wrong in the script? Or because it does not use the dynamic fusion method described in the paper. Could you provide the dynamic fusion process in DTU?

    opened by DIVE128 5
  • Testing on TnT advanced dataset

    Testing on TnT advanced dataset

    Hi, thank you for sharing this great work!

    I'm try to test transmvsnet on tnt advanced dataset, but meet some problem. My test environment is ubuntu16.04 with cuda11.3 and pytorch 1.10.

    The first thing is that there is no cams_1 folder under tnt dataset, is it a revised version of original cams folder or you just changed the folder name?

    I just changed the folder name, then run scripts/test_tnt.sh, but I find the speed is rather slow, about 10 seconds on 1080ti for a image (1056 x 1920), is it normal?

    Finally I get the fused point cloud, but the cloud is meaningless, I checked the depth map and confidence map, all of the data are very strange, apperantly not right.

    Can you help me with these problems?

    opened by CanCanZeng 4
  • Some implement details about the paper

    Some implement details about the paper

    Firstly thanks for your paper and I'm looking forward to your open-sourced code.

    And I have some questions about your paper: (Hopefully you can reply, thanks in advance!) (1) In section 4.2, "The model is trained with Adam for 10 epochs with an initial learning rate of 0.001, which decays by a factor of 0.5 respectively after 6, 8, and 12 epochs." I'm confused about the epochs. And I also noticed that this training strategy is different from CasMVSNet. Did you try the training strategy in CasMVSNet? What's the difference? (2) In Table4(b), focal loss(what is the value of \gamma?) suppresses CE loss by 0.06. However, In Table4(e) and Table 6, we infer that the best model use CE loss(FL with \gamma=0). My question is: did you keep Focal loss \gamma unchanged in the Ablation study in Table4? If not, how \gamma changes? Could you elaborate?

    Really appreciate it!

    opened by JeffWang987 4
  • source code

    source code

    Hi, @Lxiangyue Thank you for the nice paper.

    It's been over a month since authors announced that the code will be available. May I know when the code will be released? (or whether it will not be released)

    opened by Ys-Jung77 3
  • Testing on my own dataset

    Testing on my own dataset

    Hi thanks for your interesting work. I tested your code on one of the DTU dataset (Moda). as you can see from the following image, the results are quite well. image

    but I got a very bad result, when i tried to tested on one of my dataset (see the following pic) using your pretrained model (model_dtu). Now, my question is that do you thing that the object is too complicated and different compared to DTU dataset and it is all we can get from the pretrain model without retraining it? is it possible to improve by changing the input parameters? In general, would you please share your opinion about this result? image

    opened by AliKaramiFBK 1
  • generate dense 3D point cloud

    generate dense 3D point cloud

    thanks for your greate work I just tried to do a test on DTU testing dataset I got the depth map for each view but I got a bit confised on how to generate 3D point cloud using your code would you please let me know Best

    opened by AliKaramiFBK 1
  • GPU memory consumption

    GPU memory consumption

    Hi! Thanks for your excellent work! When I tested on the DTU dataset with pretrained model, the gpu memory consumption is 4439MB, but the paper gives 3778MB.

    I do not know where the problem is.

    opened by JianfeiJ 0
  • Using my own data

    Using my own data

    If I have the intrinsic matrics and extrinsic matrics of cameras, which means I don't need to run SFM in COLMAP, how should I struct my data to train the model?

    opened by PaperDollssss 2
  • TnT dataset results

    TnT dataset results

    Thanks for the great job. I follow the instruction and upload the reconstruction result of tnt but find the F-score=60.29, and I find the point cloud sizes are a larger than the upload ones. Whether the reconstructed point cloud use the param settting of test_tnt.sh or it should be tuned manually? :smile:

    opened by CC9310 1
  • TankAndTemple Test

    TankAndTemple Test

    Hi, 我测试了TAT数据集中的Family,使用的是默认脚本test_tnt.sh,采用normal融合,最近仅得到13MB点云文件。经检查发现生成的mask文件夹中的_geo.png都是大部分区域黑色图片,从而最后得到的 final.png的大部分区域都是无效的。geometric consistency阈值分别是默认的0.01和1。不知道您这边是否有一样的问题?

    opened by lt-xiang 13
  • Why is there a big gap between the reproducing results and the paper results?

    Why is there a big gap between the reproducing results and the paper results?

    I have tried the pre-trained model you offered on DTU dataset. But the results I got are mean_acc=0.299, mean_comp=0.385, overall=0.342, and the results you presented in the paper are mean_acc=0.321, mean_comp=0.289, overall=0.305.

    I do not know where the problem is.

    opened by cainsmile 14
Releases(T&T_ply)
Owner
旷视研究院 3D 组
旷视科技(Face++)研究院 3D 组(原 SLAM 组)
旷视研究院 3D 组
Official implementation for: Blended Diffusion for Text-driven Editing of Natural Images.

Blended Diffusion for Text-driven Editing of Natural Images Blended Diffusion for Text-driven Editing of Natural Images Omri Avrahami, Dani Lischinski

328 Dec 30, 2022
Revitalizing CNN Attention via Transformers in Self-Supervised Visual Representation Learning

Revitalizing CNN Attention via Transformers in Self-Supervised Visual Representation Learning This repository is the official implementation of CARE.

ChongjianGE 89 Dec 02, 2022
SwinIR: Image Restoration Using Swin Transformer

SwinIR: Image Restoration Using Swin Transformer This repository is the official PyTorch implementation of SwinIR: Image Restoration Using Shifted Win

Jingyun Liang 2.4k Jan 08, 2023
UmlsBERT: Clinical Domain Knowledge Augmentation of Contextual Embeddings Using the Unified Medical Language System Metathesaurus

UmlsBERT: Clinical Domain Knowledge Augmentation of Contextual Embeddings Using the Unified Medical Language System Metathesaurus General info This is

71 Oct 25, 2022
Libraries, tools and tasks created and used at DeepMind Robotics.

dm_robotics: Libraries, tools, and tasks created and used for Robotics research at DeepMind. Package overview Package Summary Transformations Rigid bo

DeepMind 273 Jan 06, 2023
Task-based end-to-end model learning in stochastic optimization

Task-based End-to-end Model Learning in Stochastic Optimization This repository is by Priya L. Donti, Brandon Amos, and J. Zico Kolter and contains th

CMU Locus Lab 164 Dec 29, 2022
Implementation of the state of the art beat-detection, downbeat-detection and tempo-estimation model

The ISMIR 2020 Beat Detection, Downbeat Detection and Tempo Estimation Model Implementation. This is an implementation in TensorFlow to implement the

Koen van den Brink 1 Nov 12, 2021
Certified Patch Robustness via Smoothed Vision Transformers

Certified Patch Robustness via Smoothed Vision Transformers This repository contains the code for replicating the results of our paper: Certified Patc

Madry Lab 35 Dec 14, 2022
FSL-Mate: A collection of resources for few-shot learning (FSL).

FSL-Mate is a collection of resources for few-shot learning (FSL). In particular, FSL-Mate currently contains FewShotPapers: a paper list which tracks

Yaqing Wang 1.5k Jan 08, 2023
ObsPy: A Python Toolbox for seismology/seismological observatories.

ObsPy is an open-source project dedicated to provide a Python framework for processing seismological data. It provides parsers for common file formats

ObsPy 979 Jan 07, 2023
JAX + dataclasses

jax_dataclasses jax_dataclasses provides a wrapper around dataclasses.dataclass for use in JAX, which enables automatic support for: Pytree registrati

Brent Yi 35 Dec 21, 2022
(ICCV 2021 Oral) Re-distributing Biased Pseudo Labels for Semi-supervised Semantic Segmentation: A Baseline Investigation.

DARS Code release for the paper "Re-distributing Biased Pseudo Labels for Semi-supervised Semantic Segmentation: A Baseline Investigation", ICCV 2021

CVMI Lab 58 Jan 01, 2023
Powerful and efficient Computer Vision Annotation Tool (CVAT)

Computer Vision Annotation Tool (CVAT) CVAT is free, online, interactive video and image annotation tool for computer vision. It is being used by our

OpenVINO Toolkit 8.6k Jan 01, 2023
Using VapourSynth with super resolution models and speeding them up with TensorRT.

VSGAN-tensorrt-docker Using image super resolution models with vapoursynth and speeding them up with TensorRT. Using NVIDIA/Torch-TensorRT combined wi

111 Jan 05, 2023
My solutions for Stanford University course CS224W: Machine Learning with Graphs Fall 2021 colabs (GNN, GAT, GraphSAGE, GCN)

machine-learning-with-graphs My solutions for Stanford University course CS224W: Machine Learning with Graphs Fall 2021 colabs Course materials can be

Marko Njegomir 7 Dec 14, 2022
Rotation-Only Bundle Adjustment

ROBA: Rotation-Only Bundle Adjustment Paper, Video, Poster, Presentation, Supplementary Material In this repository, we provide the implementation of

Seong 51 Nov 29, 2022
A static analysis library for computing graph representations of Python programs suitable for use with graph neural networks.

python_graphs This package is for computing graph representations of Python programs for machine learning applications. It includes the following modu

Google Research 258 Dec 29, 2022
VisionKG: Vision Knowledge Graph

VisionKG: Vision Knowledge Graph Official Repository of VisionKG by Anh Le-Tuan, Trung-Kien Tran, Manh Nguyen-Duc, Jicheng Yuan, Manfred Hauswirth and

Continuous Query Evaluation over Linked Stream (CQELS) 9 Jun 23, 2022
TensorFlow2 Classification Model Zoo playing with TensorFlow2 on the CIFAR-10 dataset.

Training CIFAR-10 with TensorFlow2(TF2) TensorFlow2 Classification Model Zoo. I'm playing with TensorFlow2 on the CIFAR-10 dataset. Architectures LeNe

Chia-Hung Yuan 16 Sep 27, 2022
Datasets and source code for our paper Webly Supervised Fine-Grained Recognition: Benchmark Datasets and An Approach

Introduction Datasets and source code for our paper Webly Supervised Fine-Grained Recognition: Benchmark Datasets and An Approach Datasets: WebFG-496

21 Sep 30, 2022