Implementation of Monocular Direct Sparse Localization in a Prior 3D Surfel Map (DSL)

Related tags

Deep Learningdsl
Overview

DSL

Project page: https://sites.google.com/view/dsl-ram-lab/

Monocular Direct Sparse Localization in a Prior 3D Surfel Map

Authors: Haoyang Ye, Huaiyang Huang, and Ming Liu from RAM-LAB.

Paper and Video

Related publications:

@inproceedings{ye2020monocular,
  title={Monocular direct sparse localization in a prior 3d surfel map},
  author={Ye, Haoyang and Huang, Huaiyang and Liu, Ming},
  booktitle={2020 IEEE International Conference on Robotics and Automation (ICRA)},
  pages={8892--8898},
  year={2020},
  organization={IEEE}
}
@inproceedings{ye20213d,
  title={3D Surfel Map-Aided Visual Relocalization with Learned Descriptors},
  author={Ye, Haoyang and Huang, Huaiyang and Hutter, Marco and Sandy, Timothy and Liu, Ming},
  booktitle={2021 International Conference on Robotics and Automation (ICRA)},
  pages={5574-5581},
  year={2021},
  organization={IEEE}
}

Video: https://www.youtube.com/watch?v=LTihCBGcURo

Dependency

  1. Pangolin.
  2. CUDA.
  3. Ceres-solver.
  4. PCL, the default version accompanying by ROS.
  5. OpenCV, the default version accompanying by ROS.

Build

  1. git submodule update --init --recursive
  2. mkdir build && cd build
  3. cmake .. -DCMAKE_BUILD_TYPE=RelWithDebInfo
  4. make -j8

Example

The sample config file can be downloaded from this link.

To run the example:

[path_to_build]/src/dsl_main --path "[path_to_dataset]/left_pinhole"

Preparing Your Own Data

  1. Collect LiDAR and camera data.
  2. Build LiDAR map and obtain LiDAR poses (the poses are not necessary).
  3. Pre-process LiDAR map to make the [path_to_dataset]/*.pcd map file contains normal_x, normal_y, normal_z fields (downsample & normal estimation).
  4. Extract and undistort images into [path_to_dataset]/images.
  5. Set the first camera pose to initial_pose and other camera parameters in [path_to_dataset]/config.yaml.

Note

This implementation of DSL takes Ceres Solver as backend, which is different from the the implementation of the original paper with DSO-backend. This leads to different performance, i.e., speed and accuracy, compared to the reported results.

Credits

This work is inspired from several open-source projects, such as DSO, DSM, Elastic-Fusion, SuperPoint, DBoW2, NetVlad, LIO-mapping and etc.

Licence

The source code is released under GPL-3.0.

A Self-Supervised Contrastive Learning Framework for Aspect Detection

AspDecSSCL A Self-Supervised Contrastive Learning Framework for Aspect Detection This repository is a pytorch implementation for the following AAAI'21

Tian Shi 30 Dec 28, 2022
Sparse R-CNN: End-to-End Object Detection with Learnable Proposals, CVPR2021

End-to-End Object Detection with Learnable Proposal, CVPR2021

Peize Sun 1.2k Dec 27, 2022
(under submission) Bayesian Integration of a Generative Prior for Image Restoration

BIGPrior: Towards Decoupling Learned Prior Hallucination and Data Fidelity in Image Restoration Authors: Majed El Helou, and Sabine Süsstrunk {Note: p

Majed El Helou 22 Dec 17, 2022
This repository provides the code for MedViLL(Medical Vision Language Learner).

MedViLL This repository provides the code for MedViLL(Medical Vision Language Learner). Our proposed architecture MedViLL is a single BERT-based model

SuperSuperMoon 39 Jan 05, 2023
Pyramid Pooling Transformer for Scene Understanding

Pyramid Pooling Transformer for Scene Understanding Requirements: torch 1.6+ torchvision 0.7.0 timm==0.3.2 Validated on torch 1.6.0, torchvision 0.7.0

Yu-Huan Wu 119 Dec 29, 2022
Real-time multi-object tracker using YOLO v5 and deep sort

This repository contains a two-stage-tracker. The detections generated by YOLOv5, a family of object detection architectures and models pretrained on the COCO dataset, are passed to a Deep Sort algor

Mike 3.6k Jan 05, 2023
This is the implementation of our work Deep Extreme Cut (DEXTR), for object segmentation from extreme points.

This is the implementation of our work Deep Extreme Cut (DEXTR), for object segmentation from extreme points.

Sergi Caelles 828 Jan 05, 2023
OOD Dataset Curator and Benchmark for AI-aided Drug Discovery

🔥 DrugOOD 🔥 : OOD Dataset Curator and Benchmark for AI Aided Drug Discovery This is the official implementation of the DrugOOD project, this is the

108 Dec 17, 2022
Official implementation of VaxNeRF (Voxel-Accelearated NeRF).

VaxNeRF Paper | Google Colab This is the official implementation of VaxNeRF (Voxel-Accelearated NeRF). VaxNeRF provides very fast training and slightl

naruya 132 Nov 21, 2022
Weakly Supervised Dense Event Captioning in Videos, i.e. generating multiple sentence descriptions for a video in a weakly-supervised manner.

WSDEC This is the official repo for our NeurIPS paper Weakly Supervised Dense Event Captioning in Videos. Description Repo directories ./: global conf

Melon(Xuguang Duan) 96 Nov 01, 2022
Code for the ICCV 2021 Workshop paper: A Unified Efficient Pyramid Transformer for Semantic Segmentation.

Unified-EPT Code for the ICCV 2021 Workshop paper: A Unified Efficient Pyramid Transformer for Semantic Segmentation. Installation Linux, CUDA=10.0,

29 Aug 23, 2022
Collection of tasks for fast prototyping, baselining, finetuning and solving problems with deep learning.

Collection of tasks for fast prototyping, baselining, finetuning and solving problems with deep learning Installation

Pytorch Lightning 1.6k Jan 08, 2023
PyTorch implementation of "Image-to-Image Translation Using Conditional Adversarial Networks".

pix2pix-pytorch PyTorch implementation of Image-to-Image Translation Using Conditional Adversarial Networks. Based on pix2pix by Phillip Isola et al.

mrzhu 383 Dec 17, 2022
Group project for MFIN7036. Our goal is to predict firm profitability with text-based competition measures.

NLP_0-project Group project for MFIN7036. Our goal is to predict firm profitability with text-based competition measures1. We are a "democratic" and c

3 Mar 16, 2022
Local-Global Stratified Transformer for Efficient Video Recognition

DualFormer This repo is the implementation of our manuscript entitled "Local-Global Stratified Transformer for Efficient Video Recognition". Our model

Sea AI Lab 19 Dec 07, 2022
Simple PyTorch implementations of Badnets on MNIST and CIFAR10.

Simple PyTorch implementations of Badnets on MNIST and CIFAR10.

Vera 75 Dec 13, 2022
[ICRA 2022] CaTGrasp: Learning Category-Level Task-Relevant Grasping in Clutter from Simulation

This is the official implementation of our paper: Bowen Wen, Wenzhao Lian, Kostas Bekris, and Stefan Schaal. "CaTGrasp: Learning Category-Level Task-R

Bowen Wen 199 Jan 04, 2023
A bare-bones TensorFlow framework for Bayesian deep learning and Gaussian process approximation

Aboleth A bare-bones TensorFlow framework for Bayesian deep learning and Gaussian process approximation [1] with stochastic gradient variational Bayes

Gradient Institute 127 Dec 12, 2022
DrQ-v2: Improved Data-Augmented Reinforcement Learning

DrQ-v2: Improved Data-Augmented RL Agent Method DrQ-v2 is a model-free off-policy algorithm for image-based continuous control. DrQ-v2 builds on DrQ,

Facebook Research 234 Jan 01, 2023
⚓ Eurybia monitor model drift over time and securize model deployment with data validation

View Demo · Documentation · Medium article 🔍 Overview Eurybia is a Python library which aims to help in : Detecting data drift and model drift Valida

MAIF 172 Dec 27, 2022