Single Image Deraining Using Bilateral Recurrent Network (TIP 2020)

Overview

Single Image Deraining Using Bilateral Recurrent Network

Introduction

Single image deraining has received considerable progress based on deep convolutional neural network. Most existing deep deraining methods follow residual learning in image denoising to learn rain streak layer, and perform limited in restoring background image layer. In this work, we propose bilateral recurrent network (BRN) to allow the interplay between rain streak and background image layers. In particular, two recurrent networks are coupled to simultaneously exploit these two layers. Instead of naive combination, we propose bilateral LSTMs, which not only can respectively propagate deep features across stages, but also bring the interplay between these two SRNs, which is essential in separating two layers from rainy observation. The experimental results demonstrate that our BRN notably outperforms state-of-the-art deep deraining networks on synthetic datasets quantitatively and qualitatively. The proposed method also performs more favorably in terms of generalization performance on real-world rainy dataset.

Prerequisites

  • Python 3.6, PyTorch >= 0.4.0
  • Requirements: opencv-python, tensorboardX
  • Platforms: Ubuntu 16.04, cuda-10.0 & cuDNN v-7.5
  • MATLAB for computing evaluation metrics

Datasets

SRN and BRN are evaluated on seven datasets*: Rain100H [1], Rain100L [1], RainHeavy*[5], RainLight*[5], Rain12 [2], Rain1400 [3] and SPA-data [4]. Please download the testing datasets from BaiduYun or OneDrive, download the RainHeavy*[5] and RainLight*[5] from here, and download the testing generalization dataset SPA-data [4] from GoogleDrive. And then place the unzipped folders into './datasets/'. Make sure that the path of the extracted file is consistent with '--data_path'.

*We note that:

(i) The datasets of Rain100H and Rain100L have been updated by the authors. We notate them as RainHeavy* and RainLight*, that can be downloaded from here.

(ii) We upload the old datasets of Rain100H and Rain100L to BaiduYun or OneDrive. For Rain100H, we strictly exclude 546 rainy images that have the same background contents with testing images.

Getting Started

1) Testing

We have placed our pre-trained models into ./logs/.

Run shell scripts to test the models:

bash test_RainHeavy.sh   # test models on RainHeavy
bash test_RainLight.sh   # test models on RainLight
bash test_Rain100H.sh   # test models on Rain100H
bash test_Rain100L.sh   # test models on Rain100L
bash test_Rain12.sh     # test models on Rain12
bash test_Rain1400.sh   # test models on Rain1400
bash test_real.sh       # test models on SPA-data

(i) On RainHeavy* [5] and RainLight* [5], we re-train all the competing methods. We have uploaded all the trained models to ./logs/RainHeavy/ and ./logs/RainLight/. You can use their source codes to reproduce the results in the paper.

(ii) All the results in the paper are also available at GoogleDrive. You can place the downloaded results into ./results/, and directly compute all the evaluation metrics in this paper.

2) Evaluation metrics

We also provide the MATLAB scripts to compute the average PSNR and SSIM values reported in the paper.

 cd ./statistic
 run statistic_RainHeavy.m
 run statistic_RainLight.m
 run statistic_Rain100H.m
 run statistic_Rain100L.m
 run statistic_Rain12.m
 run statistic_Rain1400.m
 run statistic_real.m

3) Training

python train.py --save_path path_to_save_trained_models  --data_path path_to_training_dataset

*If you use the new dataset by yourself, please make sure to define new function for preprocessing training patches in DerainDataset.py.

References

[1] Yang W, Tan R, Feng J, Liu J, Guo Z, and Yan S. Deep joint rain detection and removal from a single image. In IEEE CVPR 2017.

[2] Li Y, Tan RT, Guo X, Lu J, and Brown M. Rain streak removal using layer priors. In IEEE CVPR 2016.

[3] Fu X, Huang J, Zeng D, Huang Y, Ding X, and Paisley J. Removing rain from single images via a deep detail network. In IEEE CVPR 2017.

[4] Wang T, Yang X, Xu K, Chen S, Zhang Q, and Lau R. Spatial attentive single-image deraining with a high quality real rain dataset. In IEEE CVPR 2019.

[5] Yang W, Tan R, Feng J, Liu J, Yan S, and Guo Z. Joint rain detection and removal from a single image with contextualized deep networks. IEEE T-PAMI 2019.

MOpt-AFL provided by the paper "MOPT: Optimized Mutation Scheduling for Fuzzers"

MOpt-AFL 1. Description MOpt-AFL is a AFL-based fuzzer that utilizes a customized Particle Swarm Optimization (PSO) algorithm to find the optimal sele

172 Dec 18, 2022
Source code for the paper: Variance-Aware Machine Translation Test Sets (NeurIPS 2021 Datasets and Benchmarks Track)

Variance-Aware-MT-Test-Sets Variance-Aware Machine Translation Test Sets License See LICENSE. We follow the data licensing plan as the same as the WMT

NLP2CT Lab, University of Macau 5 Dec 21, 2021
TransCD: Scene Change Detection via Transformer-based Architecture

TransCD: Scene Change Detection via Transformer-based Architecture

wangzhixue 29 Dec 11, 2022
Code for LIGA-Stereo Detector, ICCV'21

LIGA-Stereo Introduction This is the official implementation of the paper LIGA-Stereo: Learning LiDAR Geometry Aware Representations for Stereo-based

Xiaoyang Guo 75 Dec 09, 2022
Contrastive Learning of Structured World Models

Contrastive Learning of Structured World Models This repository contains the official PyTorch implementation of: Contrastive Learning of Structured Wo

Thomas Kipf 371 Jan 06, 2023
code for Fast Point Cloud Registration with Optimal Transport

robot This is the repository for the paper "Accurate Point Cloud Registration with Robust Optimal Transport". We are in the process of refactoring the

28 Jan 04, 2023
Cross-platform CLI tool to generate your Github profile's stats and summary.

ghs Cross-platform CLI tool to generate your Github profile's stats and summary. Preview Hop on to examples for other usecases. Jump to: Installation

HackerRank 134 Dec 20, 2022
This is the official implementation of our proposed SwinMR

SwinMR This is the official implementation of our proposed SwinMR: Swin Transformer for Fast MRI Please cite: @article{huang2022swin, title={Swi

A Yang Lab (led by Dr Guang Yang) 27 Nov 17, 2022
The implementation of "Shuffle Transformer: Rethinking Spatial Shuffle for Vision Transformer"

Shuffle Transformer The implementation of "Shuffle Transformer: Rethinking Spatial Shuffle for Vision Transformer" Introduction Very recently, window-

87 Nov 29, 2022
PFLD pytorch Implementation

PFLD-pytorch Implementation of PFLD A Practical Facial Landmark Detector by pytorch. 1. install requirements pip3 install -r requirements.txt 2. Datas

zhaozhichao 669 Jan 02, 2023
Turi Create simplifies the development of custom machine learning models.

Quick Links: Installation | Documentation | WWDC 2019 | WWDC 2018 Turi Create Check out our talks at WWDC 2019 and at WWDC 2018! Turi Create simplifie

Apple 10.9k Jan 01, 2023
This repository is for the preprint "A generative nonparametric Bayesian model for whole genomes"

BEAR Overview This repository contains code associated with the preprint A generative nonparametric Bayesian model for whole genomes (2021), which pro

Debora Marks Lab 10 Sep 18, 2022
La source de mon module 'pyfade' disponible sur Pypi.

Version: 1.2 Introduction Pyfade est un module permettant de créer des dégradés colorés. Il vous permettra de changer chaque ligne de votre texte par

Billy 20 Sep 12, 2021
modelvshuman is a Python library to benchmark the gap between human and machine vision

modelvshuman is a Python library to benchmark the gap between human and machine vision. Using this library, both PyTorch and TensorFlow models can be evaluated on 17 out-of-distribution datasets with

Bethge Lab 244 Jan 03, 2023
Learning an Adaptive Meta Model-Generator for Incrementally Updating Recommender Systems

Learning an Adaptive Meta Model-Generator for Incrementally Updating Recommender Systems This is our experimental code for RecSys 2021 paper "Learning

11 Jul 28, 2022
RTSeg: Real-time Semantic Segmentation Comparative Study

Real-time Semantic Segmentation Comparative Study The repository contains the official TensorFlow code used in our papers: RTSEG: REAL-TIME SEMANTIC S

Mennatullah Siam 592 Nov 18, 2022
Fuwa-http - The http client implementation for the fuwa eco-system

Fuwa HTTP The HTTP client implementation for the fuwa eco-system Example import

Fuwa 2 Feb 16, 2022
An open source library for face detection in images. The face detection speed can reach 1000FPS.

libfacedetection This is an open source library for CNN-based face detection in images. The CNN model has been converted to static variables in C sour

Shiqi Yu 11.4k Dec 27, 2022
AoT is a system for automatically generating off-target test harness by using build information.

AoT: Auto off-Target Automatically generating off-target test harness by using build information. Brought to you by the Mobile Security Team at Samsun

Samsung 10 Oct 19, 2022
a curated list of docker-compose files prepared for testing data engineering tools, databases and open source libraries.

data-services A repository for storing various Data Engineering docker-compose files in one place. How to use it ? Set the required settings in .env f

BigData.IR 525 Dec 03, 2022