This repository contains the implementation of the following paper: Cross-Descriptor Visual Localization and Mapping

Overview

Cross-Descriptor Visual Localization and Mapping

This repository contains the implementation of the following paper:

"Cross-Descriptor Visual Localization and Mapping".
M. Dusmanu, O. Miksik, J.L. Schönberger, and M. Pollefeys. ICCV 2021.

[Paper on arXiv]

Requirements

COLMAP

We use COLMAP for DoG keypoint extraction as well as localization and mapping. Please follow the installation instructions available on the official webpage. Before proceeding, we recommend setting an environmental variable to the COLMAP executable folder by running export COLMAP_PATH=path_to_colmap_executable_folder.

Python

The environment can be set up directly using conda:

conda env create -f env.yml
conda activate cross-descriptor-vis-loc-map

Training data

We provide a script for downloading the raw training data:

bash scripts/download_training_data.sh

Evaluation data

We provide a script for downloading the LFE dataset along with the GT used for evaluation as well as the Aachen Day-Night dataset:

bash scripts/download_evaluation_data.sh

Training

Data preprocessing

First step is extracting keypoints and descriptors on the training data downloaded above.

bash scripts/process_training_data.sh

Alternatively, you can directly download the processed training data by running:

bash scripts/download_processed_training_data.sh

Training

To run training with the default architecture and hyper-parameters, execute the following:

python train.py \
    --dataset_path data/train/colmap \
    --features brief sift-kornia hardnet sosnet

Pretrained models

We provide two pretrained models trained on descriptors extracted from COLMAP SIFT and OpenCV SIFT keypoints, respectively. These models can be downloaded by running:

bash scripts/download_checkpoints.sh

Evaluation

Demo Notebook

Click for details...

Local Feature Evaluation Benchmark

Click for details...

First step is extracting descriptors on all datasets:

bash scripts/process_LFE_data.sh

We provide examples below for running reconstruction on Madrid Metrpolis in each different evaluation scenario.

Reconstruction using a single descriptor (standard)

python local-feature-evaluation/reconstruction_pipeline_progressive.py \
    --dataset_path data/eval/LFE-release/Madrid_Metropolis \
    --colmap_path $COLMAP_PATH \
    --features sift-kornia \
    --exp_name sift-kornia-single

Reconstruction using the progressive approach (ours)

python local-feature-evaluation/reconstruction_pipeline_progressive.py \
    --dataset_path data/eval/LFE-release/Madrid_Metropolis \
    --colmap_path $COLMAP_PATH \
    --features brief sift-kornia hardnet sosnet \
    --exp_name progressive

Reconstruction using the joint embedding approach (ours)

python local-feature-evaluation/reconstruction_pipeline_embed.py \
    --dataset_path data/eval/LFE-release/Madrid_Metropolis \
    --colmap_path $COLMAP_PATH \
    --features brief sift-kornia hardnet sosnet \
    --exp_name embed

Reconstruction using a single descriptor on the associated split (real-world)

python local-feature-evaluation/reconstruction_pipeline_subset.py \
    --dataset_path data/eval/LFE-release/Madrid_Metropolis/ \
    --colmap_path $COLMAP_PATH \
    --features brief sift-kornia hardnet sosnet \
    --feature sift-kornia \
    --exp_name sift-kornia-subset

Evaluation of a reconstruction w.r.t. metric pseudo-ground-truth

python local-feature-evaluation/align_and_compare.py \
    --colmap_path $COLMAP_PATH \
    --reference_model_path data/eval/LFE-release/Madrid_Metropolis/sparse-reference/filtered-metric/ \
    --model_path data/eval/LFE-release/Madrid_Metropolis/sparse-sift-kornia-single/0/

Aachen Day-Night

Click for details...

BibTeX

If you use this code in your project, please cite the following paper:

@InProceedings{Dusmanu2021Cross,
    author = {Dusmanu, Mihai and Miksik, Ondrej and Sch\"onberger, Johannes L. and Pollefeys, Marc},
    title = {{Cross Descriptor Visual Localization and Mapping}},
    booktitle = {Proceedings of the International Conference on Computer Vision},
    year = {2021}
}
Owner
Mihai Dusmanu
PhD Student at ETH Zurich. Computer Vision + Deep Learning. Feature detection / description / matching, 3D reconstruction.
Mihai Dusmanu
PFFDTD is an open-source FDTD simulator for 3D room acoustics

PFFDTD is an open-source FDTD simulator for 3D room acoustics

Brian Hamilton 34 Nov 24, 2022
2021 CCF BDCI 全国信息检索挑战杯(CCIR-Cup)智能人机交互自然语言理解赛道第二名参赛解决方案

2021 CCF BDCI 全国信息检索挑战杯(CCIR-Cup) 智能人机交互自然语言理解赛道第二名解决方案 比赛网址: CCIR-Cup-智能人机交互自然语言理解 1.依赖环境: python==3.8 torch==1.7.1+cu110 numpy==1.19.2 transformers=

JinXiang 22 Oct 29, 2022
Faune proche - Retrieval of Faune-France data near a google maps location

faune_proche Récupération des données de Faune-France près d'un lieu google maps

4 Feb 15, 2022
You can draw the corresponding bounding box into the image and save it according to the result file (txt format) run by the tracker.

You can draw the corresponding bounding box into the image and save it according to the result file (txt format) run by the tracker.

Huiyiqianli 42 Dec 06, 2022
WaveFake: A Data Set to Facilitate Audio DeepFake Detection

WaveFake: A Data Set to Facilitate Audio DeepFake Detection This is the code repository for our NeurIPS 2021 (Track on Datasets and Benchmarks) paper

Chair for Sys­tems Se­cu­ri­ty 27 Dec 22, 2022
LightningFSL: Pytorch-Lightning implementations of Few-Shot Learning models.

LightningFSL: Few-Shot Learning with Pytorch-Lightning In this repo, a number of pytorch-lightning implementations of FSL algorithms are provided, inc

Xu Luo 76 Dec 11, 2022
More Photos are All You Need: Semi-Supervised Learning for Fine-Grained Sketch Based Image Retrieval

More Photos are All You Need: Semi-Supervised Learning for Fine-Grained Sketch Based Image Retrieval, CVPR 2021. Ayan Kumar Bhunia, Pinaki nath Chowdh

Ayan Kumar Bhunia 22 Aug 27, 2022
Implementation for Curriculum DeepSDF

Curriculum-DeepSDF This repository is an implementation for Curriculum DeepSDF. Full paper is available here. Preparation Please follow original setti

Haidong Zhu 69 Dec 29, 2022
Multistream CNN for Robust Acoustic Modeling

Multistream Convolutional Neural Network (CNN) A multistream CNN is a novel neural network architecture for robust acoustic modeling in speech recogni

ASAPP Research 37 Sep 21, 2022
Syed Waqas Zamir 906 Dec 30, 2022
Tools for the Cleveland State Human Motion and Control Lab

Introduction This is a collection of tools that are helpful for gait analysis. Some are specific to the needs of the Human Motion and Control Lab at C

CSU Human Motion and Control Lab 88 Dec 16, 2022
Python package provinding tools for artistic interactive applications using AI

Documentation redrawing Python package provinding tools for artistic interactive applications using AI Created by ReDrawing Campinas team for the Open

ReDrawing Campinas 1 Sep 30, 2021
Code for ACM MM 2020 paper "NOH-NMS: Improving Pedestrian Detection by Nearby Objects Hallucination"

NOH-NMS: Improving Pedestrian Detection by Nearby Objects Hallucination The offical implementation for the "NOH-NMS: Improving Pedestrian Detection by

Tencent YouTu Research 64 Nov 11, 2022
This is the repo for the paper "Improving the Accuracy-Memory Trade-Off of Random Forests Via Leaf-Refinement".

Improving the Accuracy-Memory Trade-Off of Random Forests Via Leaf-Refinement This is the repository for the paper "Improving the Accuracy-Memory Trad

3 Dec 29, 2022
Exposure Time Calculator (ETC) and radial velocity precision estimator for the Near InfraRed Planet Searcher (NIRPS) spectrograph

NIRPS-ETC Exposure Time Calculator (ETC) and radial velocity precision estimator for the Near InfraRed Planet Searcher (NIRPS) spectrograph February 2

Nolan Grieves 2 Sep 15, 2022
code and data for paper "GIANT: Scalable Creation of a Web-scale Ontology"

GIANT Code and data for paper "GIANT: Scalable Creation of a Web-scale Ontology" https://arxiv.org/pdf/2004.02118.pdf Please cite our paper if this pr

Excalibur 39 Dec 29, 2022
Implementation of C-RNN-GAN.

Implementation of C-RNN-GAN. Publication: Title: C-RNN-GAN: Continuous recurrent neural networks with adversarial training Information: http://mogren.

Olof Mogren 427 Dec 25, 2022
[CVPR 2021] Exemplar-Based Open-Set Panoptic Segmentation Network (EOPSN)

EOPSN: Exemplar-Based Open-Set Panoptic Segmentation Network (CVPR 2021) PyTorch implementation for EOPSN. We propose open-set panoptic segmentation t

Jaedong Hwang 49 Dec 30, 2022
DenseNet Implementation in Keras with ImageNet Pretrained Models

DenseNet-Keras with ImageNet Pretrained Models This is an Keras implementation of DenseNet with ImageNet pretrained weights. The weights are converted

Felix Yu 568 Oct 31, 2022
A PyTorch implementation of Radio Transformer Networks from the paper "An Introduction to Deep Learning for the Physical Layer".

An Introduction to Deep Learning for the Physical Layer An usable PyTorch implementation of the noisy autoencoder infrastructure in the paper "An Intr

Gram.AI 120 Nov 21, 2022