TransMVSNet: Global Context-aware Multi-view Stereo Network with Transformers.

Overview
Comments
  • abs_depth_error

    abs_depth_error

    I find ABS_DEPTH_ERROR is close to 6 or even 7 during training, is this normal? Here are the training results for Epoch 5. Is it because of the slow convergence?

    avg_test_scalars: {'loss': 4.360309665948113, 'depth_loss': 6.535046514014081, 'entropy_loss': 4.360309665948113, 'abs_depth_error': 6.899323051878795, 'thres2mm_error': 0.16829867261163733, 'thres4mm_error': 0.10954744909229193, 'thres8mm_error': 0.07844322964626443, 'thres14mm_error': 0.06323695212957076, 'thres20mm_error': 0.055751020700780536, 'thres2mm_abserror': 0.597563438798779, 'thres4mm_abserror': 2.7356186663791666, 'thres8mm_abserror': 5.608324628466483, 'thres14mm_abserror': 10.510002394554125, 'thres20mm_abserror': 16.67409769420184, 'thres>20mm_abserror': 78.15814284054947}

    opened by zhang-snowy 7
  • About the fusion setting in DTU

    About the fusion setting in DTU

    Thank you for your great contribution. The script use the gipuma as the fusion method with num_consistent=5prob_threshold=0.05disp_threshold=0.25. However, it produces point cloud results with only 1/2 points compared with the point cloud results you provide in DTU, leading to a much poorer result in DTU. Is there any setting wrong in the script? Or because it does not use the dynamic fusion method described in the paper. Could you provide the dynamic fusion process in DTU?

    opened by DIVE128 5
  • Testing on TnT advanced dataset

    Testing on TnT advanced dataset

    Hi, thank you for sharing this great work!

    I'm try to test transmvsnet on tnt advanced dataset, but meet some problem. My test environment is ubuntu16.04 with cuda11.3 and pytorch 1.10.

    The first thing is that there is no cams_1 folder under tnt dataset, is it a revised version of original cams folder or you just changed the folder name?

    I just changed the folder name, then run scripts/test_tnt.sh, but I find the speed is rather slow, about 10 seconds on 1080ti for a image (1056 x 1920), is it normal?

    Finally I get the fused point cloud, but the cloud is meaningless, I checked the depth map and confidence map, all of the data are very strange, apperantly not right.

    Can you help me with these problems?

    opened by CanCanZeng 4
  • Some implement details about the paper

    Some implement details about the paper

    Firstly thanks for your paper and I'm looking forward to your open-sourced code.

    And I have some questions about your paper: (Hopefully you can reply, thanks in advance!) (1) In section 4.2, "The model is trained with Adam for 10 epochs with an initial learning rate of 0.001, which decays by a factor of 0.5 respectively after 6, 8, and 12 epochs." I'm confused about the epochs. And I also noticed that this training strategy is different from CasMVSNet. Did you try the training strategy in CasMVSNet? What's the difference? (2) In Table4(b), focal loss(what is the value of \gamma?) suppresses CE loss by 0.06. However, In Table4(e) and Table 6, we infer that the best model use CE loss(FL with \gamma=0). My question is: did you keep Focal loss \gamma unchanged in the Ablation study in Table4? If not, how \gamma changes? Could you elaborate?

    Really appreciate it!

    opened by JeffWang987 4
  • source code

    source code

    Hi, @Lxiangyue Thank you for the nice paper.

    It's been over a month since authors announced that the code will be available. May I know when the code will be released? (or whether it will not be released)

    opened by Ys-Jung77 3
  • Testing on my own dataset

    Testing on my own dataset

    Hi thanks for your interesting work. I tested your code on one of the DTU dataset (Moda). as you can see from the following image, the results are quite well. image

    but I got a very bad result, when i tried to tested on one of my dataset (see the following pic) using your pretrained model (model_dtu). Now, my question is that do you thing that the object is too complicated and different compared to DTU dataset and it is all we can get from the pretrain model without retraining it? is it possible to improve by changing the input parameters? In general, would you please share your opinion about this result? image

    opened by AliKaramiFBK 1
  • generate dense 3D point cloud

    generate dense 3D point cloud

    thanks for your greate work I just tried to do a test on DTU testing dataset I got the depth map for each view but I got a bit confised on how to generate 3D point cloud using your code would you please let me know Best

    opened by AliKaramiFBK 1
  • GPU memory consumption

    GPU memory consumption

    Hi! Thanks for your excellent work! When I tested on the DTU dataset with pretrained model, the gpu memory consumption is 4439MB, but the paper gives 3778MB.

    I do not know where the problem is.

    opened by JianfeiJ 0
  • Using my own data

    Using my own data

    If I have the intrinsic matrics and extrinsic matrics of cameras, which means I don't need to run SFM in COLMAP, how should I struct my data to train the model?

    opened by PaperDollssss 2
  • TnT dataset results

    TnT dataset results

    Thanks for the great job. I follow the instruction and upload the reconstruction result of tnt but find the F-score=60.29, and I find the point cloud sizes are a larger than the upload ones. Whether the reconstructed point cloud use the param settting of test_tnt.sh or it should be tuned manually? :smile:

    opened by CC9310 1
  • TankAndTemple Test

    TankAndTemple Test

    Hi, 我测试了TAT数据集中的Family,使用的是默认脚本test_tnt.sh,采用normal融合,最近仅得到13MB点云文件。经检查发现生成的mask文件夹中的_geo.png都是大部分区域黑色图片,从而最后得到的 final.png的大部分区域都是无效的。geometric consistency阈值分别是默认的0.01和1。不知道您这边是否有一样的问题?

    opened by lt-xiang 13
  • Why is there a big gap between the reproducing results and the paper results?

    Why is there a big gap between the reproducing results and the paper results?

    I have tried the pre-trained model you offered on DTU dataset. But the results I got are mean_acc=0.299, mean_comp=0.385, overall=0.342, and the results you presented in the paper are mean_acc=0.321, mean_comp=0.289, overall=0.305.

    I do not know where the problem is.

    opened by cainsmile 14
Releases(T&T_ply)
Owner
旷视研究院 3D 组
旷视科技(Face++)研究院 3D 组(原 SLAM 组)
旷视研究院 3D 组
JUSTICE: A Benchmark Dataset for Supreme Court’s Judgment Prediction

JUSTICE: A Benchmark Dataset for Supreme Court’s Judgment Prediction CSCI 544 Final Project done by: Mohammed Alsayed, Shaayan Syed, Mohammad Alali, S

Smit Patel 3 Dec 28, 2022
CvT2DistilGPT2 is an encoder-to-decoder model that was developed for chest X-ray report generation.

CvT2DistilGPT2 Improving Chest X-Ray Report Generation by Leveraging Warm-Starting This repository houses the implementation of CvT2DistilGPT2 from [1

The Australian e-Health Research Centre 21 Dec 28, 2022
[ArXiv 2021] Data-Efficient Instance Generation from Instance Discrimination

InsGen - Data-Efficient Instance Generation from Instance Discrimination Data-Efficient Instance Generation from Instance Discrimination Ceyuan Yang,

GenForce: May Generative Force Be with You 93 Dec 25, 2022
SPRING is a seq2seq model for Text-to-AMR and AMR-to-Text (AAAI2021).

SPRING This is the repo for SPRING (Symmetric ParsIng aNd Generation), a novel approach to semantic parsing and generation, presented at AAAI 2021. Wi

Sapienza NLP group 98 Dec 21, 2022
An Api for Emotion recognition.

PLAYEMO Playemo was built from the ground-up with Flask, a python tool that makes it easy for developers to build APIs. Use Cases Is Python your langu

greek geek 2 Jul 16, 2022
[CVPR'21] MonoRUn: Monocular 3D Object Detection by Reconstruction and Uncertainty Propagation

MonoRUn MonoRUn: Monocular 3D Object Detection by Reconstruction and Uncertainty Propagation. CVPR 2021. [paper] Hansheng Chen, Yuyao Huang, Wei Tian*

同济大学智能汽车研究所综合感知研究组 ( Comprehensive Perception Research Group under Institute of Intelligent Vehicles, School of Automotive Studies, Tongji University) 96 Dec 10, 2022
Multi-Task Temporal Shift Attention Networks for On-Device Contactless Vitals Measurement (NeurIPS 2020)

MTTS-CAN: Multi-Task Temporal Shift Attention Networks for On-Device Contactless Vitals Measurement Paper Xin Liu, Josh Fromm, Shwetak Patel, Daniel M

Xin Liu 106 Dec 30, 2022
Approaches to modeling terrain and maps in python

topography 🌎 Contains different approaches to modeling terrain and topographic-style maps in python Features Inverse Distance Weighting (IDW) A given

John Gutierrez 1 Aug 10, 2022
Personal thermal comfort models using digital twins: Preference prediction with BIM-extracted spatial-temporal proximity data from Build2Vec

Personal thermal comfort models using digital twins: Preference prediction with BIM-extracted spatial-temporal proximity data from Build2Vec This repo

Building and Urban Data Science (BUDS) Group 5 Dec 02, 2022
Semantic Segmentation Architectures Implemented in PyTorch

pytorch-semseg Semantic Segmentation Algorithms Implemented in PyTorch This repository aims at mirroring popular semantic segmentation architectures i

Meet Shah 3.3k Dec 29, 2022
A testcase generation tool for Persistent Memory Programs.

PMFuzz PMFuzz is a testcase generation tool to generate high-value tests cases for PM testing tools (XFDetector, PMDebugger, PMTest and Pmemcheck) If

Systems Research at ShiftLab 14 Jul 24, 2022
This code provides a PyTorch implementation for OTTER (Optimal Transport distillation for Efficient zero-shot Recognition), as described in the paper.

Data Efficient Language-Supervised Zero-Shot Recognition with Optimal Transport Distillation This repository contains PyTorch evaluation code, trainin

Meta Research 45 Dec 20, 2022
Official PyTorch implemention of our paper "Learning to Rectify for Robust Learning with Noisy Labels".

WarPI The official PyTorch implemention of our paper "Learning to Rectify for Robust Learning with Noisy Labels". Run python main.py --corruption_type

Haoliang Sun 3 Sep 03, 2022
This repository is an open-source implementation of the ICRA 2021 paper: Locus: LiDAR-based Place Recognition using Spatiotemporal Higher-Order Pooling.

Locus This repository is an open-source implementation of the ICRA 2021 paper: Locus: LiDAR-based Place Recognition using Spatiotemporal Higher-Order

Robotics and Autonomous Systems Group 96 Dec 15, 2022
Pytorch implementation of "A simple neural network module for relational reasoning" (Relational Networks)

Pytorch implementation of Relational Networks - A simple neural network module for relational reasoning Implemented & tested on Sort-of-CLEVR task. So

Kim Heecheol 800 Dec 05, 2022
Avalanche RL: an End-to-End Library for Continual Reinforcement Learning

Avalanche RL: an End-to-End Library for Continual Reinforcement Learning Avalanche Website | Getting Started | Examples | Tutorial | API Doc | Paper |

ContinualAI 43 Dec 24, 2022
Code for "Graph-Evolving Meta-Learning for Low-Resource Medical Dialogue Generation". [AAAI 2021]

Graph Evolving Meta-Learning for Low-resource Medical Dialogue Generation Code to be further cleaned... This repo contains the code of the following p

Shuai Lin 29 Nov 01, 2022
Implementation of "A Deep Learning Loss Function based on Auditory Power Compression for Speech Enhancement" by pytorch

This repository is used to suspend the results of our paper "A Deep Learning Loss Function based on Auditory Power Compression for Speech Enhancement"

ScorpioMiku 19 Sep 30, 2022
Clustering is a popular approach to detect patterns in unlabeled data

Visual Clustering Clustering is a popular approach to detect patterns in unlabeled data. Existing clustering methods typically treat samples in a data

Tarek Naous 24 Nov 11, 2022