SuMa++: Efficient LiDAR-based Semantic SLAM (Chen et al IROS 2019)

Overview

SuMa++: Efficient LiDAR-based Semantic SLAM

This repository contains the implementation of SuMa++, which generates semantic maps only using three-dimensional laser range scans.

Developed by Xieyuanli Chen and Jens Behley.

SuMa++ is built upon SuMa and RangeNet++. For more details, we refer to the original project websites SuMa and RangeNet++.

An example of using SuMa++: ptcl

Table of Contents

  1. Introduction
  2. Publication
  3. Dependencies
  4. Build
  5. How to run
  6. More Related Work
  7. License

Publication

If you use our implementation in your academic work, please cite the corresponding paper:

@inproceedings{chen2019iros, 
		author = {X. Chen and A. Milioto and E. Palazzolo and P. Giguère and J. Behley and C. Stachniss},
		title  = {{SuMa++: Efficient LiDAR-based Semantic SLAM}},
		booktitle = {Proceedings of the IEEE/RSJ Int. Conf. on Intelligent Robots and Systems (IROS)},
		year = {2019},
		codeurl = {https://github.com/PRBonn/semantic_suma/},
		videourl = {https://youtu.be/uo3ZuLuFAzk},
}

Dependencies

  • catkin
  • Qt5 >= 5.2.1
  • OpenGL >= 4.0
  • libEigen >= 3.2
  • gtsam >= 4.0 (tested with 4.0.0-alpha2)

In Ubuntu 16.04: Installing all dependencies should be accomplished by

sudo apt-get install build-essential cmake libgtest-dev libeigen3-dev libboost-all-dev qtbase5-dev libglew-dev libqt5libqgtk2 catkin

Additionally, make sure you have catkin-tools and the fetch verb installed:

sudo apt install python-pip
sudo pip install catkin_tools catkin_tools_fetch empy

Build

rangenet_lib

To use SuMa++, you need to first build the rangenet_lib with the TensorRT and C++ interface. For more details about building and using rangenet_lib you could find in rangenet_lib.

SuMa++

Clone the repository in the src directory of the same catkin workspace where you built the rangenet_lib:

git clone https://github.com/PRBonn/semantic_suma.git

Download the additional dependencies (or clone glow into your catkin workspace src yourself):

catkin deps fetch

For the first setup of your workspace containing this project, you need:

catkin build --save-config -i --cmake-args -DCMAKE_BUILD_TYPE=Release -DOPENGL_VERSION=430 -DENABLE_NVIDIA_EXT=YES

Where you have to set OPENGL_VERSION to the supported OpenGL core profile version of your system, which you can query as follows:

$ glxinfo | grep "version"
server glx version string: 1.4
client glx version string: 1.4
GLX version: 1.4
OpenGL core profile version string: 4.3.0 NVIDIA 367.44
OpenGL core profile shading language version string: 4.30 NVIDIA [...]
OpenGL version string: 4.5.0 NVIDIA 367.44
OpenGL shading language version string: 4.50 NVIDIA

Here the line OpenGL core profile version string: 4.3.0 NVIDIA 367.44 is important and therefore you should use -DOPENGL_VERSION = 430. If you are unsure you can also leave it on the default version 330, which should be supported by all OpenGL-capable devices.

If you have a NVIDIA device, like a Geforce or Quadro graphics card, you should also activate the NVIDIA extensions using -DENABLE_NVIDIA_EXT=YES for info about the current GPU memory usage of the program.

After this setup steps, you can build with catkin build, since the configuration has been saved to your current Catkin profile (therefore, --save-config was needed).

Now the project root directory (e.g. ~/catkin_ws/src/semantic_suma) should contain a bin directory containing the visualizer.

How to run

Important Notice

  • Before running SuMa++, you need to first build the rangenet_lib and download the pretrained model.
  • You need to specify the model path in the configuration file in the config/ folder.
  • For the first time using, rangenet_lib will take several minutes to build a .trt model for SuMa++.
  • SuMa++ now can only work with KITTI dataset, since the semantic segmentation may not generalize well in other environments.
  • To use SuMa++ with your own dataset, you may finetune or retrain the semantic segmentation network.

All binaries are copied to the bin directory of the source folder of the project. Thus,

  1. run visualizer in the bin directory by ./visualizer,
  2. open a Velodyne directory from the KITTI Visual Odometry Benchmark and select a ".bin" file,
  3. start the processing of the scans via the "play button" in the GUI.

More Related Work

OverlapNet - Loop Closing for 3D LiDAR-based SLAM

This repo contains the code for our RSS2020 paper: OverlapNet - Loop Closing for 3D LiDAR-based SLAM.

OverlapNet is a modified Siamese Network that predicts the overlap and relative yaw angle of a pair of range images generated by 3D LiDAR scans, which can be used for place recognition and loop closing.

Overlap-based LiDAR Global Localization

This repo contains the code for our IROS2020 paper: Learning an Overlap-based Observation Model for 3D LiDAR Localization.

It uses the OverlapNet to train an observation model for Monte Carlo Localization and achieves global localization with 3D LiDAR scans.

License

Copyright 2019, Xieyuanli Chen, Jens Behley, Cyrill Stachniss, Photogrammetry and Robotics Lab, University of Bonn.

This project is free software made available under the MIT License. For details see the LICENSE file.

Owner
Photogrammetry & Robotics Bonn
Photogrammetry & Robotics Lab at the University of Bonn
Photogrammetry & Robotics Bonn
This porject is intented to build the most accurate model for predicting the porbability of loan default

Estimating-Loan-Default-Probability IBA ML2 Mid-project / Kaggle Competition This porject is intented to build the most accurate model for predicting

Adil Gahramanov 1 Jan 24, 2022
Measure WWjj polarization fraction

WlWl Polarization Measure WWjj polarization fraction Paper: arXiv:2109.09924 Notice: This code can only be used for the inference process, if you want

4 Apr 10, 2022
Real-Time Social Distance Monitoring tool using Computer Vision

Social Distance Detector A Real-Time Social Distance Monitoring Tool Table of Contents Motivation YOLO Theory Detection Output Tech Stack Functionalit

Pranav B 13 Oct 14, 2022
An official repository for Paper "Uformer: A General U-Shaped Transformer for Image Restoration".

Uformer: A General U-Shaped Transformer for Image Restoration Zhendong Wang, Xiaodong Cun, Jianmin Bao and Jianzhuang Liu Paper: https://arxiv.org/abs

Zhendong Wang 497 Dec 22, 2022
RetinaFace: Deep Face Detection Library in TensorFlow for Python

RetinaFace is a deep learning based cutting-edge facial detector for Python coming with facial landmarks.

Sefik Ilkin Serengil 512 Dec 29, 2022
Code for our paper "Graph Pre-training for AMR Parsing and Generation" in ACL2022

AMRBART An implementation for ACL2022 paper "Graph Pre-training for AMR Parsing and Generation". You may find our paper here (Arxiv). Requirements pyt

xfbai 60 Jan 03, 2023
Space Invaders For Python

Space-Invaders Just download or clone the git repository. To run the Space Invader game you need to have pyhton installed in you system. If you dont h

Fei 5 Jul 27, 2022
Implementation of CSRL from the AAAI2022 paper: Constraint Sampling Reinforcement Learning: Incorporating Expertise For Faster Learning

CSRL Implementation of CSRL from the AAAI2022 paper: Constraint Sampling Reinforcement Learning: Incorporating Expertise For Faster Learning Python: 3

4 Apr 14, 2022
Price-Prediction-For-a-Dream-Home - A machine learning based linear regression trained model for house price prediction.

Price-Prediction-For-a-Dream-Home ROADMAP TO THIS LINEAR REGRESSION BASED HOUSE PRICE PREDICTION PREDICTION MODEL Import all the dependencies of the p

DIKSHA DESWAL 1 Dec 29, 2021
Multilingual Image Captioning

Multilingual Image Captioning Authors: Bhavitvya Malik, Gunjan Chhablani Demo Link: https://huggingface.co/spaces/flax-community/multilingual-image-ca

Gunjan Chhablani 32 Nov 25, 2022
Python Implementation of the CoronaWarnApp (CWA) Event Registration

Python implementation of the Corona-Warn-App (CWA) Event Registration This is an implementation of the Protocol used to generate event and location QR

MaZderMind 17 Oct 05, 2022
Equivariant GNN for the prediction of atomic multipoles up to quadrupoles.

Equivariant Graph Neural Network for Atomic Multipoles Description Repository for the Model used in the publication 'Learning Atomic Multipoles: Predi

16 Nov 22, 2022
A curated list of awesome Deep Learning tutorials, projects and communities.

Awesome Deep Learning Table of Contents Books Courses Videos and Lectures Papers Tutorials Researchers Websites Datasets Conferences Frameworks Tools

Christos 20k Jan 05, 2023
gym-anm is a framework for designing reinforcement learning (RL) environments that model Active Network Management (ANM) tasks in electricity distribution networks.

gym-anm is a framework for designing reinforcement learning (RL) environments that model Active Network Management (ANM) tasks in electricity distribution networks. It is built on top of the OpenAI G

Robin Henry 99 Dec 12, 2022
PyTorch code accompanying the paper "Landmark-Guided Subgoal Generation in Hierarchical Reinforcement Learning" (NeurIPS 2021).

HIGL This is a PyTorch implementation for our paper: Landmark-Guided Subgoal Generation in Hierarchical Reinforcement Learning (NeurIPS 2021). Our cod

Junsu Kim 20 Dec 14, 2022
YOLOX-RMPOLY

本算法为适应robomaster比赛,而改动自矩形识别的yolox算法。 基于旷视科技YOLOX,实现对不规则四边形的目标检测 TODO 修改onnx推理模型 更改/添加标注: 1.yolox/models/yolox_polyhead.py: 1.1继承yolox/models/yolo_

3 Feb 25, 2022
Article Reranking by Memory-enhanced Key Sentence Matching for Detecting Previously Fact-checked Claims.

MTM This is the official repository of the paper: Article Reranking by Memory-enhanced Key Sentence Matching for Detecting Previously Fact-checked Cla

ICTMCG 13 Sep 17, 2022
Large Scale Multi-Illuminant (LSMI) Dataset for Developing White Balance Algorithm under Mixed Illumination

Large Scale Multi-Illuminant (LSMI) Dataset for Developing White Balance Algorithm under Mixed Illumination (ICCV 2021) Dataset License This work is l

DongYoung Kim 33 Jan 04, 2023
Pytorch Implementation of the paper "Cross-domain Correspondence Learning for Exemplar-based Image Translation"

CoCosNet Pytorch Implementation of the paper "Cross-domain Correspondence Learning for Exemplar-based Image Translation" (CVPR 2020 oral). Update: 202

Lingbo Yang 38 Sep 22, 2021
[ ICCV 2021 Oral ] Our method can estimate camera poses and neural radiance fields jointly when the cameras are initialized at random poses in complex scenarios (outside-in scenes, even with less texture or intense noise )

GNeRF This repository contains official code for the ICCV 2021 paper: GNeRF: GAN-based Neural Radiance Field without Posed Camera. This implementation

Quan Meng 191 Dec 26, 2022