SSL_SLAM2: Lightweight 3-D Localization and Mapping for Solid-State LiDAR (mapping and localization separated) ICRA 2021

Overview

SSL_SLAM2

Lightweight 3-D Localization and Mapping for Solid-State LiDAR (Intel Realsense L515 as an example)

This repo is an extension work of SSL_SLAM. Similar to RTABMAP, SSL_SLAM2 separates the mapping module and localization module. Map saving and map optimization is enabled in the mapping unit. Map loading and localization is enabled in the localziation unit.

This code is an implementation of paper "Lightweight 3-D Localization and Mapping for Solid-State LiDAR", published in IEEE Robotics and Automation Letters, 2021 paper

A summary video demo can be found at Video

Modifier: Wang Han, Nanyang Technological University, Singapore

Running speed: 20 Hz on Intel NUC, 30 Hz on PC

1. Solid-State Lidar Sensor Example

1.1 Scene reconstruction example

1.2 Localization with built map

1.3 Comparison

2. Prerequisites

2.1 Ubuntu and ROS

Ubuntu 64-bit 18.04.

ROS Melodic. ROS Installation

2.2. Ceres Solver

Follow Ceres Installation.

2.3. PCL

Follow PCL Installation.

Tested with 1.8.1

2.4. GTSAM

Follow GTSAM Installation.

2.5. Trajectory visualization

For visualization purpose, this package uses hector trajectory sever, you may install the package by

sudo apt-get install ros-melodic-hector-trajectory-server

Alternatively, you may remove the hector trajectory server node if trajectory visualization is not needed

3. Sensor Setup

If you have new Realsense L515 sensor, you may follow the below setup instructions

3.1 L515

3.2 Librealsense

Follow Librealsense Installation

3.3 Realsense_ros

Copy realsense_ros package to your catkin folder

    cd ~/catkin_ws/src
    git clone https://github.com/IntelRealSense/realsense-ros.git
    cd ..
    catkin_make

4. Build SSL_SLAM2

4.1 Clone repository:

    cd ~/catkin_ws/src
    git clone https://github.com/wh200720041/ssl_slam2.git
    cd ..
    catkin_make
    source ~/catkin_ws/devel/setup.bash

4.2 Download test rosbag

You may download our recorded data: MappingTest.bag (3G) and LocalizationTest.bag (6G)if you dont have realsense L515, and by defult the file should be under home/user/Downloads

unzip the file (it may take a while to unzip)

cd ~/Downloads
unzip LocalizationTest.zip
unzip MappingTest.zip

4.3 Map Building

map optimization and building

    roslaunch ssl_slam2 ssl_slam2_mapping.launch

The map optimization is performed based on loop closure, you have to specify the loop clousre manually in order to trigger global optimization. To save map, open a new terminal and

  rosservice call /save_map

Upon calling the serviece, the map will be automatically saved. It is recommended to have a loop closure to reduce the drifts. Once the service is called, loop closure will be checked. For example, in the rosbag provided, the loop closure appears at frame 1060-1120, thus, when you see "total_frame 1070" or "total_frame 1110" you may immediately type

  rosservice call /save_map

Since the current frame is between 1060 and 1120, the loop closure will be triggered automatically and the global map will be optimized and saved

4.4 Localization

Type

    roslaunch ssl_slam2 ssl_slam2_localization.launch

If your map is large, it may takes a while to load

4.5 Parameters Explanation

The map size depends on number of keyframes used. The more keyframes used for map buildin, the larger map will be.

min_map_update_distance: distance threshold to add a keyframe. higher means lower update rate. min_map_update_angle: angle threshold to add a keyframe. higher means lower update rate. min_map_update_frame: time threshold to add a keyframe. higher means lower update rate.

4.6 Relocalization

The relocalization module under tracking loss is still under development. You must specify the robot init pose w.r.t. the map coordinate if the starting position is not the origin of map. You can set this by

    <param name="offset_x" type="double" value="0.0" />
    <param name="offset_y" type="double" value="0.0" />
    <param name="offset_yaw" type="double" value="0.0" />

4.7 Running speed

The realsense is running at 30Hz and some computer may not be able to support such high processing rate. You may reduce the processing rate by skipping frames. You can do thid by setting the

<param name="skip_frames" type="int" value="1" />

1 implies no skip frames, i.e., 30Hz; implies skip 1 frames, i.e., 15Hz. For small map building, you can do it online. however, it is recommended to record a rosbag and build map offline for large mapping since the dense map cannot be generated in real-time.

5 Map Building with multiple loop closure places

5.1 Dataset

You may download a larger dataset LargeMappingTest.bag (10G), and by defult the file should be under home/user/Downloads

unzip the file (it may take a while to unzip)

cd ~/Downloads
unzip LargeMappingTest.zip

5.2 Map Building

Two loop closure places appear at frame 0-1260 and 1270-3630, i.e., frame 0 and frame 1260 are the same place, frame 1270 adn 3630 are the same place. Run

    roslaunch ssl_slam2 ssl_slam2_large_mapping.launch

open a new terminal, when you see "total_frame 1260", immediately type

  rosservice call /save_map

when you see "total_frame 3630", immediately type again

  rosservice call /save_map

6. Citation

If you use this work for your research, you may want to cite the paper below, your citation will be appreciated

@article{wang2021lightweight,
  author={H. {Wang} and C. {Wang} and L. {Xie}},
  journal={IEEE Robotics and Automation Letters}, 
  title={Lightweight 3-D Localization and Mapping for Solid-State LiDAR}, 
  year={2021},
  volume={6},
  number={2},
  pages={1801-1807},
  doi={10.1109/LRA.2021.3060392}}
Owner
Wang Han 王晗
I am currently a Phd Candidate at Nanyang Technological University, Singapore, specialize in computer vision and robotics
Wang Han 王晗
Repo for the ACMMM20 submission: "Personalized breath based biometric authentication with wearable multimodality".

personalized-breath Repo for the ACMMM20 submission: "Personalized breath based biometric authentication with wearable multimodality". Guideline To ex

Manh-Ha Bui 2 Nov 15, 2021
Complete* list of autonomous driving related datasets

AD Datasets Complete* and curated list of autonomous driving related datasets Contributing Contributions are very welcome! To add or update a dataset:

Daniel Bogdoll 13 Dec 19, 2022
A python bot to move your mouse every few seconds to appear active on Skype, Teams or Zoom as you go AFK. 🐭 🤖

PyMouseBot If you're from GT and annoyed with SGVPN idle timeouts while working on development laptop, You might find this useful. A python cli bot to

Oaker Min 6 Oct 24, 2022
Social Distancing Detector

Computer vision has opened up a lot of opportunities to explore into AI domain that were earlier highly limited. Here is an application of haarcascade classifier and OpenCV to develop a social distan

Ashish Pandey 2 Jul 18, 2022
CoRe: Contrastive Recurrent State-Space Models

CoRe: Contrastive Recurrent State-Space Models This code implements the CoRe model and reproduces experimental results found in Robust Robotic Control

Apple 21 Aug 11, 2022
The Deep Learning with Julia book, using Flux.jl.

Deep Learning with Julia DL with Julia is a book about how to do various deep learning tasks using the Julia programming language and specifically the

Logan Kilpatrick 67 Dec 25, 2022
deep learning model that learns to code with drawing in the Processing language

sketchnet sketchnet - processing code generator can we teach a computer to draw pictures with code. We use Processing and java/jruby code paired with

41 Dec 12, 2022
An executor that loads ONNX models and embeds documents using the ONNX runtime.

ONNXEncoder An executor that loads ONNX models and embeds documents using the ONNX runtime. Usage via Docker image (recommended) from jina import Flow

Jina AI 2 Mar 15, 2022
Generating Videos with Scene Dynamics

Generating Videos with Scene Dynamics This repository contains an implementation of Generating Videos with Scene Dynamics by Carl Vondrick, Hamed Pirs

Carl Vondrick 706 Jan 04, 2023
Implementation of FSGNN

FSGNN Implementation of FSGNN. For more details, please refer to our paper Experiments were conducted with following setup: Pytorch: 1.6.0 Python: 3.8

19 Dec 05, 2022
Blind Video Temporal Consistency via Deep Video Prior

deep-video-prior (DVP) Code for NeurIPS 2020 paper: Blind Video Temporal Consistency via Deep Video Prior PyTorch implementation | paper | project web

Chenyang LEI 272 Dec 21, 2022
Video Instance Segmentation using Inter-Frame Communication Transformers (NeurIPS 2021)

Video Instance Segmentation using Inter-Frame Communication Transformers (NeurIPS 2021) Paper Video Instance Segmentation using Inter-Frame Communicat

Sukjun Hwang 81 Dec 29, 2022
EgGateWayGetShell py脚本

EgGateWayGetShell_py 免责声明 由于传播、利用此文所提供的信息而造成的任何直接或者间接的后果及损失,均由使用者本人负责,作者不为此承担任何责任。 使用 python3 eg.py urls.txt 目标 title:锐捷网络-EWEB网管系统 port:4430 漏洞成因 ?p

榆木 61 Nov 09, 2022
Individual Tree Crown classification on WorldView-2 Images using Autoencoder -- Group 9 Weak learners - Final Project (Machine Learning 2020 Course)

Created by Olga Sutyrina, Sarah Elemili, Abduragim Shtanchaev and Artur Bille Individual Tree Crown classification on WorldView-2 Images using Autoenc

2 Dec 08, 2022
Learning Features with Parameter-Free Layers (ICLR 2022)

Learning Features with Parameter-Free Layers (ICLR 2022) Dongyoon Han, YoungJoon Yoo, Beomyoung Kim, Byeongho Heo | Paper NAVER AI Lab, NAVER CLOVA Up

NAVER AI 65 Dec 07, 2022
Pytorch Lightning Implementation of SC-Depth Methods.

SC_Depth_pl: This is a pytorch lightning implementation of SC-Depth (V1, V2) for self-supervised learning of monocular depth from video. In the V1 (IJ

JiaWang Bian 216 Dec 30, 2022
SMD-Nets: Stereo Mixture Density Networks

SMD-Nets: Stereo Mixture Density Networks This repository contains a Pytorch implementation of "SMD-Nets: Stereo Mixture Density Networks" (CVPR 2021)

Fabio Tosi 115 Dec 26, 2022
MPRNet-Cloud-removal: Progressive cloud removal

MPRNet-Cloud-removal Progressive cloud removal Requirements 1.Pytorch = 1.0 2.Python 3 3.NVIDIA GPU + CUDA 9.0 4.Tensorboard Installation 1.Clone the

Semi 95 Dec 18, 2022
Like a cowsay but without cows!

Foxsay This is a simple program that generates pictures of a cute fox with a message. It is like a cowsay but without cows! Fox girls are better! Usag

Anastasia Kim 28 Feb 20, 2022
Pytorch re-implementation of Paper: SwinTextSpotter: Scene Text Spotting via Better Synergy between Text Detection and Text Recognition (CVPR 2022)

SwinTextSpotter This is the pytorch implementation of Paper: SwinTextSpotter: Scene Text Spotting via Better Synergy between Text Detection and Text R

mxin262 183 Jan 03, 2023