This repository contains the implementation of the following paper: Cross-Descriptor Visual Localization and Mapping

Overview

Cross-Descriptor Visual Localization and Mapping

This repository contains the implementation of the following paper:

"Cross-Descriptor Visual Localization and Mapping".
M. Dusmanu, O. Miksik, J.L. Schönberger, and M. Pollefeys. ICCV 2021.

[Paper on arXiv]

Requirements

COLMAP

We use COLMAP for DoG keypoint extraction as well as localization and mapping. Please follow the installation instructions available on the official webpage. Before proceeding, we recommend setting an environmental variable to the COLMAP executable folder by running export COLMAP_PATH=path_to_colmap_executable_folder.

Python

The environment can be set up directly using conda:

conda env create -f env.yml
conda activate cross-descriptor-vis-loc-map

Training data

We provide a script for downloading the raw training data:

bash scripts/download_training_data.sh

Evaluation data

We provide a script for downloading the LFE dataset along with the GT used for evaluation as well as the Aachen Day-Night dataset:

bash scripts/download_evaluation_data.sh

Training

Data preprocessing

First step is extracting keypoints and descriptors on the training data downloaded above.

bash scripts/process_training_data.sh

Alternatively, you can directly download the processed training data by running:

bash scripts/download_processed_training_data.sh

Training

To run training with the default architecture and hyper-parameters, execute the following:

python train.py \
    --dataset_path data/train/colmap \
    --features brief sift-kornia hardnet sosnet

Pretrained models

We provide two pretrained models trained on descriptors extracted from COLMAP SIFT and OpenCV SIFT keypoints, respectively. These models can be downloaded by running:

bash scripts/download_checkpoints.sh

Evaluation

Demo Notebook

Click for details...

Local Feature Evaluation Benchmark

Click for details...

First step is extracting descriptors on all datasets:

bash scripts/process_LFE_data.sh

We provide examples below for running reconstruction on Madrid Metrpolis in each different evaluation scenario.

Reconstruction using a single descriptor (standard)

python local-feature-evaluation/reconstruction_pipeline_progressive.py \
    --dataset_path data/eval/LFE-release/Madrid_Metropolis \
    --colmap_path $COLMAP_PATH \
    --features sift-kornia \
    --exp_name sift-kornia-single

Reconstruction using the progressive approach (ours)

python local-feature-evaluation/reconstruction_pipeline_progressive.py \
    --dataset_path data/eval/LFE-release/Madrid_Metropolis \
    --colmap_path $COLMAP_PATH \
    --features brief sift-kornia hardnet sosnet \
    --exp_name progressive

Reconstruction using the joint embedding approach (ours)

python local-feature-evaluation/reconstruction_pipeline_embed.py \
    --dataset_path data/eval/LFE-release/Madrid_Metropolis \
    --colmap_path $COLMAP_PATH \
    --features brief sift-kornia hardnet sosnet \
    --exp_name embed

Reconstruction using a single descriptor on the associated split (real-world)

python local-feature-evaluation/reconstruction_pipeline_subset.py \
    --dataset_path data/eval/LFE-release/Madrid_Metropolis/ \
    --colmap_path $COLMAP_PATH \
    --features brief sift-kornia hardnet sosnet \
    --feature sift-kornia \
    --exp_name sift-kornia-subset

Evaluation of a reconstruction w.r.t. metric pseudo-ground-truth

python local-feature-evaluation/align_and_compare.py \
    --colmap_path $COLMAP_PATH \
    --reference_model_path data/eval/LFE-release/Madrid_Metropolis/sparse-reference/filtered-metric/ \
    --model_path data/eval/LFE-release/Madrid_Metropolis/sparse-sift-kornia-single/0/

Aachen Day-Night

Click for details...

BibTeX

If you use this code in your project, please cite the following paper:

@InProceedings{Dusmanu2021Cross,
    author = {Dusmanu, Mihai and Miksik, Ondrej and Sch\"onberger, Johannes L. and Pollefeys, Marc},
    title = {{Cross Descriptor Visual Localization and Mapping}},
    booktitle = {Proceedings of the International Conference on Computer Vision},
    year = {2021}
}
Owner
Mihai Dusmanu
PhD Student at ETH Zurich. Computer Vision + Deep Learning. Feature detection / description / matching, 3D reconstruction.
Mihai Dusmanu
A PaddlePaddle implementation of STGCN with a few modifications in the model architecture in order to forecast traffic jam.

About This repository contains the code of a PaddlePaddle implementation of STGCN based on the paper Spatio-Temporal Graph Convolutional Networks: A D

Tianjian Li 1 Jan 11, 2022
The source code for Adaptive Kernel Graph Neural Network at AAAI2022

AKGNN The source code for Adaptive Kernel Graph Neural Network at AAAI2022. Please cite our paper if you think our work is helpful to you: @inproceedi

11 Nov 25, 2022
【steal piano】GitHub偷情分析工具!

【steal piano】GitHub偷情分析工具! 你是否有这样的困扰,有一天你的仓库被很多人加了star,但是你却不知道这些人都是从哪来的? 别担心,GitHub偷情分析工具帮你轻松解决问题! 原理 GitHub偷情分析工具透过分析star的时间以及他们之间的follow关系,可以推测出每个st

黄巍 442 Dec 21, 2022
Few-Shot Object Detection via Association and DIscrimination

Few-Shot Object Detection via Association and DIscrimination Code release of our NeurIPS 2021 paper: Few-Shot Object Detection via Association and DIs

Cao Yuhang 49 Dec 18, 2022
AAAI 2022: Stationary diffusion state neural estimation

Stationary Diffusion State Neural Estimation Although many graph-based clustering methods attempt to model the stationary diffusion state in their obj

绽琨 33 Nov 24, 2022
The Python ensemble sampling toolkit for affine-invariant MCMC

emcee The Python ensemble sampling toolkit for affine-invariant MCMC emcee is a stable, well tested Python implementation of the affine-invariant ense

Dan Foreman-Mackey 1.3k Dec 31, 2022
Pyramid Pooling Transformer for Scene Understanding

Pyramid Pooling Transformer for Scene Understanding Requirements: torch 1.6+ torchvision 0.7.0 timm==0.3.2 Validated on torch 1.6.0, torchvision 0.7.0

Yu-Huan Wu 119 Dec 29, 2022
[CVPRW 2022] Attentions Help CNNs See Better: Attention-based Hybrid Image Quality Assessment Network

Attention Helps CNN See Better: Hybrid Image Quality Assessment Network [CVPRW 2022] Code for Hybrid Image Quality Assessment Network [paper] [code] T

IIGROUP 49 Dec 11, 2022
PiRapGenerator - Make anyone rap the digits of pi

PiRapGenerator Make anyone rap the digits of pi (sample files are of Ted Nivison

7 Oct 02, 2022
Contains code for the paper "Vision Transformers are Robust Learners".

Vision Transformers are Robust Learners This repository contains the code for the paper Vision Transformers are Robust Learners by Sayak Paul* and Pin

Sayak Paul 103 Jan 05, 2023
Combining Reinforcement Learning and Constraint Programming for Combinatorial Optimization

Hybrid solving process for combinatorial optimization problems Combinatorial optimization has found applications in numerous fields, from aerospace to

117 Dec 13, 2022
Robust, modular and efficient implementation of advanced Hamiltonian Monte Carlo algorithms

AdvancedHMC.jl AdvancedHMC.jl provides a robust, modular and efficient implementation of advanced HMC algorithms. An illustrative example for Advanced

The Turing Language 167 Jan 01, 2023
A scikit-learn compatible neural network library that wraps PyTorch

A scikit-learn compatible neural network library that wraps PyTorch. Resources Documentation Source Code Examples To see more elaborate examples, look

4.9k Jan 03, 2023
Unsupervised Representation Learning by Invariance Propagation

Unsupervised Learning by Invariance Propagation This repository is the official implementation of Unsupervised Learning by Invariance Propagation. Pre

FengWang 15 Jul 06, 2022
Official implementation of "An Image is Worth 16x16 Words, What is a Video Worth?" (2021 paper)

An Image is Worth 16x16 Words, What is a Video Worth? paper Official PyTorch Implementation Gilad Sharir, Asaf Noy, Lihi Zelnik-Manor DAMO Academy, Al

213 Nov 12, 2022
git《Beta R-CNN: Looking into Pedestrian Detection from Another Perspective》(NeurIPS 2020) GitHub:[fig3]

Beta R-CNN: Looking into Pedestrian Detection from Another Perspective This is the pytorch implementation of our paper "[Beta R-CNN: Looking into Pede

35 Sep 08, 2021
Bottom-up Human Pose Estimation

Introduction This is the official code of Rethinking the Heatmap Regression for Bottom-up Human Pose Estimation. This paper has been accepted to CVPR2

108 Dec 01, 2022
ViViT: Curvature access through the generalized Gauss-Newton's low-rank structure

ViViT is a collection of numerical tricks to efficiently access curvature from the generalized Gauss-Newton (GGN) matrix based on its low-rank structure. Provided functionality includes computing

Felix Dangel 12 Dec 08, 2022
Code image classification of MNIST dataset using different architectures: simple linear NN, autoencoder, and highway network

Deep Learning for image classification pip install -r http://webia.lip6.fr/~baskiotisn/requirements-amal.txt Train an autoencoder python3 train_auto

Hector Kohler 0 Mar 30, 2022
JORLDY an open-source Reinforcement Learning (RL) framework provided by KakaoEnterprise

Repository for Open Source Reinforcement Learning Framework JORLDY

Kakao Enterprise Corp. 330 Dec 30, 2022