For visualizing the dair-v2x-i dataset

Overview

3D Detection & Tracking Viewer

The project is based on hailanyi/3D-Detection-Tracking-Viewer and is modified, you can find the original version of the code below: https://github.com/hailanyi/3D-Detection-Tracking-Viewer

This project was developed for viewing 3D object detection results from the Dair-V2X-I datasets.

It supports rendering 3D bounding boxes and rendering boxes on images.

Features

  • Captioning box ids(infos) in 3D scene
  • Projecting 3D box or points on 2D image

Design pattern

This code includes two parts, one for convert tools, other one for visualization of 3D detection results.

Change log

  • (2022.02.01) Adapted to the Dair-V2X-I dataset

Prepare data

  • Dair-V2X-I detection dataset
  • Convert the Dair-V2X-I dataset to kitti format using the conversion tool

Requirements (Updated 2021.11.2)

python==3.7.11
numpy==1.21.4
vedo==2022.0.1
vtk==8.1.2
opencv-python==4.1.1.26
matplotlib==3.4.3
open3d==0.14.1

It is recommended to use anaconda to create the visualization environment

conda create -n dair_vis python=3.8

To activate this environment, use

conda activate dair_vis

Install the requirements

pip install -r requirements.txt

To deactivate an active environment, use

conda deactivate

Convert tools

  • Prepare a dataset of the following structure:
  • "kitti_format" must be an empty folder to store the conversion result
  • "source_format" to store the source Dair-V2X-I datasets.
# For Dair-V2X-I Dataset  
dair_v2x_i
├── kitti_format
├── source_format
│   ├── single-infrastructure-side
│   │   ├── calib
│   │   │   ├── camera_intrinsic
│   │   │   └── virtuallidar_to_camera
│   │   └── label
│   │       ├── camera
│   │       └── virtuallidar
│   ├── single-infrastructure-side-example
│   │   ├── calib
│   │   │   ├── camera_intrinsic
│   │   │   └── virtuallidar_to_camera
│   │   ├── image
│   │   ├── label
│   │   │   ├── camera
│   │   │   └── virtuallidar
│   │   └── velodyne
│   ├── single-infrastructure-side-image
│   └── single-infrastructure-side-velodyne

  • If you have the same folder structure, you only need change the "root path" to your local path from config/config.yaml
  • Running the jupyter notebook server and open the "convert.ipynb"
  • The code is very simple , so there are no input parameters for advanced customization, you need to comment or copy the code to implemented separately following functions : -Convert calib files to KITTI format -Convert camera-based label files to KITTI format -Convert lidar-based label files to KITTI format -Convert image folders to KITTI format -Convert velodyne folders to KITTI format

After the convet you will get the following result. the

dair_v2x_i
├── kitti_format
│   ├── calib
│   ├── image_2
│   ├── label_2
│   ├── label_velodyne
│   └── velodyne
 
  • The label_2 base the camera label, and use the lidar label information replace the size information(w,h,l). In the camera view looks like better.
  • The label_velodyne base the velodyne label.
  • P2 represents the camera internal reference, which is a 3×3 matrix, not the same as KITTI. It convert frome the "cam_K" of the json file.
  • Tr_velo_to_cam: represents the camera to lidar transformation matrix, as a 3×4 matrix.

Usage

1. Set the path to the dataset folder used for input to the visualizer

If you have completed the conversion operation, the path should have been set correctly. Otherwise you need to set "root_path" in the config/config.yaml to the correct path

2. Choose whether camera or lidar based tagging for visualization

You need to set the "label_select" parameter in config.yaml to "cam" or "vel", to specify the label frome label_2 or velodyne_label.

2. Run and Terminate

  • You can start the program with the following command
python dair_3D_detection_viewer.py
  • Pressing space in the lidar window will display the next frame
  • Terminating the program is more complicated, you cannot terminate the program at static image status. You need to press the space quickly to make the frames play continuously, and when it becomes obvious that the system is overloaded with resources and the program can't respond, press Ctrl-C in the terminal window to terminate it. Try a few more times and you will eventually get the hang of it.

Notes on the Dair-V2X-I dataset

  • In the calib file of this dataset, "cam_K" is the real intrinsic matrix parameter of the camera, not "P". Although they are very close in value and structure.
  • There are multiple camera images with different focal and perspectives in this dataset, and the camera intrinsic matrix reference will change with each image file. Therefore, when using this dataset, please make sure that the calib file you are using corresponds to the image file (e.g. do not use only the 000000.txt parameter for all image files)
  • The sequence of files in this dataset is non-contiguous (e.g. missing the 000023), do not only use 00000 to lens(dataset) to get the sequence of file names directly.
  • The dataset provides optimized labels for both lidar and camera, and after testing, there are errors in the projection of the lidar label on camera (but the projection matrix is correct, only the label itself has issues). Likewise, there is a disadvantage of using the camera's label in lidar. Therefore it is recommended to use the corresponding label for lidar, and use the fused label for the camera.
  • There are some other objects in the label, for example you can see some trafficcone.
Expressive Body Capture: 3D Hands, Face, and Body from a Single Image

Expressive Body Capture: 3D Hands, Face, and Body from a Single Image [Project Page] [Paper] [Supp. Mat.] Table of Contents License Description Fittin

Vassilis Choutas 1.3k Jan 07, 2023
Python project to take sound as input and output as RGB + Brightness values suitable for DMX

sound-to-light Python project to take sound as input and output as RGB + Brightness values suitable for DMX Current goals: Get one pixel working: Vary

Bobby Cox 1 Nov 17, 2021
Graph-total-spanning-trees - A Python script to get total number of Spanning Trees in a Graph

Total number of Spanning Trees in a Graph This is a python script just written f

Mehdi I. 0 Jul 18, 2022
Step by Step on how to create an vision recognition model using LOBE.ai, export the model and run the model in an Azure Function

Step by Step on how to create an vision recognition model using LOBE.ai, export the model and run the model in an Azure Function

El Bruno 3 Mar 30, 2022
CLIP-GEN: Language-Free Training of a Text-to-Image Generator with CLIP

CLIP-GEN [简体中文][English] 本项目在萤火二号集群上用 PyTorch 实现了论文 《CLIP-GEN: Language-Free Training of a Text-to-Image Generator with CLIP》。 CLIP-GEN 是一个 Language-F

75 Dec 29, 2022
Image Processing, Image Smoothing, Edge Detection and Transforms

opevcvdl-hw1 This project uses openCV and Qt to achieve the requirements. Version Python 3.7 opencv-contrib-python 3.4.2.17 Matplotlib 3.1.1 pyqt5 5.1

Kenny Cheng 3 Aug 17, 2022
This repo in the implementation of EMNLP'21 paper "SPARQLing Database Queries from Intermediate Question Decompositions" by Irina Saparina, Anton Osokin

SPARQLing Database Queries from Intermediate Question Decompositions This repo is the implementation of the following paper: SPARQLing Database Querie

Yandex Research 20 Dec 19, 2022
A short code in python, Enchpyter, is able to encrypt and decrypt words as you determine, of course

Enchpyter Enchpyter is a program do encrypt and decrypt any word you want (just letters). You enter how many letters jumps and write the word, so, the

João Assalim 2 Oct 10, 2022
Pytorch implementation of various High Dynamic Range (HDR) Imaging algorithms

Deep High Dynamic Range Imaging Benchmark This repository is the pytorch impleme

Tianhong Dai 5 Nov 16, 2022
The official repo for CVPR2021——ViPNAS: Efficient Video Pose Estimation via Neural Architecture Search.

ViPNAS: Efficient Video Pose Estimation via Neural Architecture Search [paper] Introduction This is the official implementation of ViPNAS: Efficient V

Lumin 42 Sep 26, 2022
Gapmm2: gapped alignment using minimap2 (align transcripts to genome)

gapmm2: gapped alignment using minimap2 This tool is a wrapper for minimap2 to r

Jon Palmer 2 Jan 27, 2022
Pytorch library for seismic data augmentation

Pytorch library for seismic data augmentation

Artemii Novoselov 27 Nov 22, 2022
Code for our paper "Sematic Representation for Dialogue Modeling" in ACL2021

AMR-Dialogue An implementation for paper "Semantic Representation for Dialogue Modeling". You may find our paper here. Requirements python 3.6 pytorch

xfbai 45 Dec 26, 2022
Exploring the Dual-task Correlation for Pose Guided Person Image Generation

Dual-task Pose Transformer Network The source code for our paper "Exploring Dual-task Correlation for Pose Guided Person Image Generation“ (CVPR2022)

63 Dec 15, 2022
The authors' implementation of Unsupervised Adversarial Learning of 3D Human Pose from 2D Joint Locations

Unsupervised Adversarial Learning of 3D Human Pose from 2D Joint Locations This is the authors' implementation of Unsupervised Adversarial Learning of

Dwango Media Village 140 Dec 07, 2022
Implementation of Barlow Twins paper

barlowtwins PyTorch Implementation of Barlow Twins paper: Barlow Twins: Self-Supervised Learning via Redundancy Reduction This is currently a work in

IgorSusmelj 86 Dec 20, 2022
This is the source code of the solver used to compete in the International Timetabling Competition 2019.

ITC2019 Solver This is the source code of the solver used to compete in the International Timetabling Competition 2019. Building .NET Core (2.1 or hig

Edon Gashi 8 Jan 22, 2022
MlTr: Multi-label Classification with Transformer

MlTr: Multi-label Classification with Transformer This is official implement of "MlTr: Multi-label Classification with Transformer". Abstract The task

程星 38 Nov 08, 2022
Spherical CNNs

Spherical CNNs Equivariant CNNs for the sphere and SO(3) implemented in PyTorch Overview This library contains a PyTorch implementation of the rotatio

Jonas Köhler 893 Dec 28, 2022
A vision library for performing sliced inference on large images/small objects

SAHI: Slicing Aided Hyper Inference A vision library for performing sliced inference on large images/small objects Overview Object detection and insta

Open Business Software Solutions 2.3k Jan 04, 2023