(CVPR 2021) Back-tracing Representative Points for Voting-based 3D Object Detection in Point Clouds

Related tags

Computer VisionBRNet
Overview

BRNet

fig_overview-c2

Introduction

This is a release of the code of our paper Back-tracing Representative Points for Voting-based 3D Object Detection in Point Clouds, CVPR 2021.

Authors: Bowen Cheng, Lu Sheng*, Shaoshuai Shi, Ming Yang, Dong Xu (*corresponding author)

[arxiv]

In this repository, we reimplement BRNet based on mmdetection3d for easier usage.

Citation

If you find our work useful in your research, please consider citing:

@inproceedings{cheng2021brnet,
  title={Back-tracing Representative Points for Voting-based 3D Object Detection in Point Clouds},
  author={Cheng, Bowen and Sheng, Lu and Shi, Shaoshuai and Yang, Ming and Xu, Dong},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year={2021}
}

Installation

This repo is built based on mmdetection3d (V0.11.0), please follow the getting_started.md for installation.

The code is tested under the following environment:

  • Ubuntu 16.04 LTS
  • Python 3.7.10
  • Pytorch 1.5.0
  • CUDA 10.1
  • GCC 7.3

Datasets

ScanNet

Please follow the instruction here to prepare ScanNet Data.

SUN RGB-D

Please follow the instruction here to prepare SUN RGB-D Data.

Download Trained Models

We provide the trained models of ScanNet and SUN RGB-D with per-class performances.

ScanNet V2 AP_0.25 AR_0.25 AP_0.50 AR_0.50
cabinet 0.4898 0.7634 0.2800 0.5349
bed 0.8849 0.9506 0.7915 0.8642
chair 0.9149 0.9357 0.8354 0.8604
sofa 0.9049 0.9794 0.8027 0.9278
table 0.6802 0.8486 0.6146 0.7600
door 0.5955 0.7430 0.3721 0.5418
window 0.4814 0.7092 0.2405 0.4078
bookshelf 0.5876 0.8701 0.5032 0.7532
picture 0.1716 0.3243 0.0687 0.1396
counter 0.6085 0.8846 0.3545 0.5385
desk 0.7538 0.9528 0.5481 0.7874
curtain 0.6275 0.7910 0.4126 0.5224
refrigerator 0.5467 0.9474 0.4882 0.8070
showercurtrain 0.7349 0.9643 0.5189 0.6786
toilet 0.9896 1.0000 0.9227 0.9310
sink 0.5901 0.6735 0.3521 0.4490
bathtub 0.8605 0.9355 0.8565 0.9032
garbagebin 0.4726 0.7151 0.3169 0.5170
Overall 0.6608 0.8327 0.5155 0.6624
SUN RGB-D AP_0.25 AR_0.25 AP_0.50 AR_0.50
bed 0.8633 0.9553 0.6544 0.7592
table 0.5136 0.8552 0.2981 0.5268
sofa 0.6754 0.8931 0.5830 0.7193
chair 0.7864 0.8723 0.6301 0.7137
toilet 0.8699 0.9793 0.7125 0.8345
desk 0.2929 0.8082 0.1134 0.4017
dresser 0.3237 0.7615 0.2058 0.4954
night_stand 0.5933 0.8627 0.4490 0.6588
bookshelf 0.3394 0.7199 0.1574 0.3652
bathtub 0.7505 0.8776 0.5383 0.6531
Overall 0.6008 0.8585 0.4342 0.6128

Note: Due to the detection results are unstable and fluctuate within 1~2 mAP points, the results here are slightly different from those in the paper.

Training

For ScanNet V2, please run:

CUDA_VISIBLE_DEVICES=0 python tools/train.py configs/brnet/brnet_8x1_scannet-3d-18class.py --seed 42

For SUN RGB-D, please run:

CUDA_VISIBLE_DEVICES=0 python tools/train.py configs/brnet/brnet_8x1_sunrgbd-3d-10class.py --seed 42

Demo

To test a 3D detector on point cloud data, please refer to Single modality demo and Point cloud demo in MMDetection3D docs.

Here, we provide a demo on SUN RGB-D dataset.

CUDA_VISIBLE_DEVICES=0 python demo/pcd_demo.py sunrgbd_000094.bin demo/brnet_8x1_sunrgbd-3d-10class.py checkpoints/brnet_8x1_sunrgbd-3d-10class_trained.pth

Visualization results

ScanNet

SUN RGB-D

Acknowledgments

Our code is heavily based on mmdetection3d. Thanks mmdetection3d Development Team for their awesome codebase.

The Open Source Framework for Machine Vision

SimpleCV Quick Links: About Installation [Docker] (#docker) Ubuntu Virtual Environment Arch Linux Fedora MacOS Windows Raspberry Pi SimpleCV Shell Vid

Sight Machine 2.6k Dec 31, 2022
A facial recognition program that plays a alarm (mp3 file) when a person i seen in the room. A basic theif using Python and OpenCV

Home-Security-Demo A facial recognition program that plays a alarm (mp3 file) when a person is seen in the room. A basic theif using Python and OpenCV

SysKey 4 Nov 02, 2021
Balabobapy - Using artificial intelligence algorithms to continue the text

Balabobapy - Using artificial intelligence algorithms to continue the text

qxtony 1 Feb 04, 2022
Distilling Knowledge via Knowledge Review, CVPR 2021

ReviewKD Distilling Knowledge via Knowledge Review Pengguang Chen, Shu Liu, Hengshuang Zhao, Jiaya Jia This project provides an implementation for the

DV Lab 194 Dec 28, 2022
Maze generator and solver with python

Procedural-Maze-Generator-Algorithms Check out my youtube channel : Auctux Ressources Thanks to Jamis Buck Book : Mazes for programmers Requirements P

Joseph 19 Dec 07, 2022
Repository relating to the CVPR21 paper TimeLens: Event-based Video Frame Interpolation

TimeLens: Event-based Video Frame Interpolation This repository is about the High Speed Event and RGB (HS-ERGB) dataset, used in the 2021 CVPR paper T

Robotics and Perception Group 544 Dec 19, 2022
An expandable and scalable OCR pipeline

Overview Nidaba is the central controller for the entire OGL OCR pipeline. It oversees and automates the process of converting raw images into citable

81 Jan 04, 2023
[BMVC'21] Official PyTorch Implementation of Grounded Situation Recognition with Transformers

Grounded Situation Recognition with Transformers Paper | Model Checkpoint This is the official PyTorch implementation of Grounded Situation Recognitio

Junhyeong Cho 18 Jul 19, 2022
Primary QPDF source code and documentation

QPDF QPDF is a command-line tool and C++ library that performs content-preserving transformations on PDF files. It supports linearization, encryption,

QPDF 2.2k Jan 04, 2023
Code for the paper STN-OCR: A single Neural Network for Text Detection and Text Recognition

STN-OCR: A single Neural Network for Text Detection and Text Recognition This repository contains the code for the paper: STN-OCR: A single Neural Net

Christian Bartz 496 Jan 05, 2023
deployment of a hybrid model for automatic weapon detection/ anomaly detection for surveillance applications

Automatic Weapon Detection Deployment of a hybrid model for automatic weapon detection/ anomaly detection for surveillance applications. Loved the pro

Janhavi 4 Mar 04, 2022
Virtualdragdrop - Virtual Drag and Drop Using OpenCV and Arduino

Virtualdragdrop - Virtual Drag and Drop Using OpenCV and Arduino

Rizky Dermawan 4 Mar 10, 2022
The papers published in top-tier AI conferences in recent years.

AI-conference-papers The papers published in top-tier AI conferences in recent years. Paper table AAAI ICLR CVPR ICML ICCV ECCV NIPS 2019 ✔️ ✔️ ✔️ ✔️

Jinbae Park 6 Dec 09, 2022
OpenGait is a flexible and extensible gait recognition project

A flexible and extensible framework for gait recognition. You can focus on designing your own models and comparing with state-of-the-arts easily with the help of OpenGait.

Shiqi Yu 335 Dec 22, 2022
"Very simple but works well" Computer Vision based ID verification solution provided by LibraX.

ID Verification by LibraX.ai This is the first free Identity verification in the market. LibraX.ai is an identity verification platform for developers

LibraX.ai 46 Dec 06, 2022
Layout Analysis Evaluator for the ICDAR 2017 competition on Layout Analysis for Challenging Medieval Manuscripts

LayoutAnalysisEvaluator Layout Analysis Evaluator for: ICDAR 2019 Historical Document Reading Challenge on Large Structured Chinese Family Records ICD

17 Dec 08, 2022
An advanced 2D image manipulation with features such as edge detection and image segmentation built using OpenCV

OpenCV-ToothPaint3-Advanced-Digital-Image-Editor This application named ‘Tooth Paint’ version TP_2020.3 (64-bit) or version 3 was developed within a w

JunHong 1 Nov 05, 2021
Repository for playing the computer vision apps: People analytics on Raspberry Pi.

play-with-torch Repository for playing the computer vision apps: People analytics on Raspberry Pi. Tools Tested Hardware RasberryPi 4 Model B here, RA

eMHa 1 Sep 23, 2021
ISI's Optical Character Recognition (OCR) software for machine-print and handwriting data

VistaOCR ISI's Optical Character Recognition (OCR) software for machine-print and handwriting data Publications "How to Efficiently Increase Resolutio

ISI Center for Vision, Image, Speech, and Text Analytics 21 Dec 08, 2021