Progressive Domain Adaptation for Object Detection

Overview

Progressive Domain Adaptation for Object Detection

Implementation of our paper Progressive Domain Adaptation for Object Detection, based on pytorch-faster-rcnn and PyTorch-CycleGAN.

Paper

Progressive Domain Adaptation for Object Detection Han-Kai Hsu, Chun-Han Yao, Yi-Hsuan Tsai, Wei-Chih Hung, Hung-Yu Tseng, Maneesh Singh and Ming-Hsuan Yang IEEE Winter Conference on Applications of Computer Vision (WACV), 2020.

Please cite our paper if you find it useful for your research.

@inproceedings{hsu2020progressivedet,
  author = {Han-Kai Hsu and Chun-Han Yao and Yi-Hsuan Tsai and Wei-Chih Hung and Hung-Yu Tseng and Maneesh Singh and Ming-Hsuan Yang},
  booktitle = {IEEE Winter Conference on Applications of Computer Vision (WACV)},
  title = {Progressive Domain Adaptation for Object Detection},
  year = {2020}
}

Dependencies

This code is tested with Pytorch 0.4.1 and CUDA 9.0

# Pytorch via pip: Download and install Pytorch 0.4.1 wheel for CUDA 9.0
#                  from https://download.pytorch.org/whl/cu90/torch_stable.html
# Pytorch via conda: 
conda install pytorch=0.4.1 cuda90 -c pytorch
# Other dependencies:
pip install -r requirements.txt
sh ./lib/make.sh

Data Preparation

KITTI

  • Download the data from here.
  • Extract the files under data/KITTI/

Cityscapes

  • Download the data from here.
  • Extract the files under data/CityScapes/

Foggy Cityscapes

  • Follow the instructions here to request for the dataset download.
  • Locate the data under data/CityScapes/leftImg8bit/ as foggytrain and foggyval.

BDD100k

  • Download the data from here.
  • Extract the files under data/bdd100k/

Generate synthetic data with CycleGAN

Generate the synthetic data with the PyTorch-CycleGAN implementation.

git clone https://github.com/aitorzip/PyTorch-CycleGAN

Dataset loader code

Import the dataset loader code in ./cycleGAN_dataset_loader/ to train/test the CycleGAN on corresponding image translation task.

Generate from pre-trained weight:

Follow the testing instructions on PyTorch-CycleGAN and download the weight below to generate synthetic images. (Remember to change to the corresponding output image size)

  • KITTI with Cityscapes style (KITTI->Cityscapes): size=(376,1244) Locate the generated data under data/KITTI/training/synthCity_image_2/ with same naming and folder structure as original KITTI data.
  • Cityscapes with FoggyCityscapes style (Cityscapes->FoggyCityscapes): size=(1024,2048) Locate the generated data under data/CityScapes/leftImg8bit/synthFoggytrain with same naming and folder structure as original Cityscapes data.
  • Cityscapes with BDD style (Cityscpaes->BDD100k): size=(1024,1280) Locate the generated data under data/CityScapes/leftImg8bit/synthBDDdaytrain and data/CityScapes/leftImg8bit/synthBDDdayval with same naming and folder structure as original Cityscapes data.

Train your own CycleGAN:

Please follow the training instructions on PyTorch-CycleGAN.

Test the adaptation model

Download the following adapted weights to ./trained_weights/adapt_weight/

./experiments/scripts/test_adapt_faster_rcnn_stage1.sh [GPU_ID] [Adapt_mode] vgg16
# Specify the GPU_ID you want to use
# Adapt_mode selection:
#   'K2C': KITTI->Cityscapes
#   'C2F': Cityscapes->Foggy Cityscapes
#   'C2BDD': Cityscapes->BDD100k_day
# Example:
./experiments/scripts/test_adapt_faster_rcnn_stage2.sh 0 K2C vgg16

Train your own model

Stage one

./experiments/scripts/train_adapt_faster_rcnn_stage1.sh [GPU_ID] [Adapt_mode] vgg16
# Specify the GPU_ID you want to use
# Adapt_mode selection:
#   'K2C': KITTI->Cityscapes
#   'C2F': Cityscapes->Foggy Cityscapes
#   'C2BDD': Cityscapes->BDD100k_day
# Example:
./experiments/scripts/train_adapt_faster_rcnn_stage1.sh 0 K2C vgg16

Download the following pretrained detector weights to ./trained_weights/pretrained_detector/

Stage two

./experiments/scripts/train_adapt_faster_rcnn_stage2.sh 0 K2C vgg16

Discriminator score files:

  • netD_synthC_score.json
  • netD_CsynthFoggyC_score.json
  • netD_CsynthBDDday_score.json

Extract the pretrained CycleGAN discriminator scores to ./trained_weights/
or
Save a dictionary of CycleGAN discriminator scores with image name as key and score as value
Ex: {'jena_000074_000019_leftImg8bit.png': 0.64}

Detection results

Adaptation results

Acknowledgement

Thanks to the awesome implementations from pytorch-faster-rcnn and PyTorch-CycleGAN.

House-GAN++: Generative Adversarial Layout Refinement Network towards Intelligent Computational Agent for Professional Architects

House-GAN++ Code and instructions for our paper: House-GAN++: Generative Adversarial Layout Refinement Network towards Intelligent Computational Agent

122 Dec 28, 2022
Official Repository for the paper "Improving Baselines in the Wild".

iWildCam and FMoW baselines (WILDS) This repository was originally forked from the official repository of WILDS datasets (commit 7e103ed) For general

Kazuki Irie 3 Nov 24, 2022
Generating Videos with Scene Dynamics

Generating Videos with Scene Dynamics This repository contains an implementation of Generating Videos with Scene Dynamics by Carl Vondrick, Hamed Pirs

Carl Vondrick 706 Jan 04, 2023
MMDetection3D is an open source object detection toolbox based on PyTorch

MMDetection3D is an open source object detection toolbox based on PyTorch, towards the next-generation platform for general 3D detection. It is a part of the OpenMMLab project developed by MMLab.

OpenMMLab 3.2k Jan 05, 2023
An open-source online reverse dictionary.

An open-source online reverse dictionary.

THUNLP 6.3k Jan 09, 2023
SARS-Cov-2 Recombinant Finder for fasta sequences

Sc2rf - SARS-Cov-2 Recombinant Finder Pronounced: Scarf What's this? Sc2rf can search genome sequences of SARS-CoV-2 for potential recombinants - new

Lena Schimmel 41 Oct 03, 2022
PyTorch implementation for Partially View-aligned Representation Learning with Noise-robust Contrastive Loss (CVPR 2021)

2021-CVPR-MvCLN This repo contains the code and data of the following paper accepted by CVPR 2021 Partially View-aligned Representation Learning with

XLearning Group 33 Nov 01, 2022
Surrogate-Assisted Genetic Algorithm for Wrapper Feature Selection

SAGA Surrogate-Assisted Genetic Algorithm for Wrapper Feature Selection Please refer to the Jupyter notebook (Example.ipynb) for an example of using t

9 Dec 28, 2022
[ICCV'21] NEAT: Neural Attention Fields for End-to-End Autonomous Driving

NEAT: Neural Attention Fields for End-to-End Autonomous Driving Paper | Supplementary | Video | Poster | Blog This repository is for the ICCV 2021 pap

254 Jan 02, 2023
Code for KDD'20 "Generative Pre-Training of Graph Neural Networks"

GPT-GNN: Generative Pre-Training of Graph Neural Networks GPT-GNN is a pre-training framework to initialize GNNs by generative pre-training. It can be

Ziniu Hu 346 Dec 19, 2022
Machine Learning Time-Series Platform

cesium: Open-Source Platform for Time Series Inference Summary cesium is an open source library that allows users to: extract features from raw time s

632 Dec 26, 2022
Attention-driven Robot Manipulation (ARM) which includes Q-attention

Attention-driven Robotic Manipulation (ARM) This codebase is home to: Q-attention: Enabling Efficient Learning for Vision-based Robotic Manipulation I

Stephen James 84 Dec 29, 2022
TensorLight - A high-level framework for TensorFlow

TensorLight is a high-level framework for TensorFlow-based machine intelligence applications. It reduces boilerplate code and enables advanced feature

Benjamin Kan 10 Jul 31, 2022
L-Verse: Bidirectional Generation Between Image and Text

Far beyond learning long-range interactions of natural language, transformers are becoming the de-facto standard for many vision tasks with their power and scalabilty

Kim, Taehoon 102 Dec 21, 2022
A list of Machine Learning Art Colabs

ML Visual Art Colabs A list of cool Colabs on Machine Learning Imagemaking or other artistic purposes 3D Ken Burns Effect Ken Burns Effect by Manuel R

Derrick Schultz (he/him) 789 Dec 12, 2022
The official code for paper "R2D2: Recursive Transformer based on Differentiable Tree for Interpretable Hierarchical Language Modeling".

R2D2 This is the official code for paper titled "R2D2: Recursive Transformer based on Differentiable Tree for Interpretable Hierarchical Language Mode

Alipay 49 Dec 17, 2022
The code for our CVPR paper PISE: Person Image Synthesis and Editing with Decoupled GAN, Project Page, supp.

PISE The code for our CVPR paper PISE: Person Image Synthesis and Editing with Decoupled GAN, Project Page, supp. Requirement conda create -n pise pyt

jinszhang 110 Nov 21, 2022
Only a Matter of Style: Age Transformation Using a Style-Based Regression Model

Only a Matter of Style: Age Transformation Using a Style-Based Regression Model The task of age transformation illustrates the change of an individual

444 Dec 30, 2022
Mosaic of Object-centric Images as Scene-centric Images (MosaicOS) for long-tailed object detection and instance segmentation.

MosaicOS Mosaic of Object-centric Images as Scene-centric Images (MosaicOS) for long-tailed object detection and instance segmentation. Introduction M

Cheng Zhang 27 Oct 12, 2022
A collection of differentiable SVD methods and also the official implementation of the ICCV21 paper "Why Approximate Matrix Square Root Outperforms Accurate SVD in Global Covariance Pooling?"

Differentiable SVD Introduction This repository contains: The official Pytorch implementation of ICCV21 paper Why Approximate Matrix Square Root Outpe

YueSong 32 Dec 25, 2022