Pytorch code for our paper "Feedback Network for Image Super-Resolution" (CVPR2019)

Overview

Feedback Network for Image Super-Resolution [arXiv] [CVF] [Poster]

Update: Our proposed Gated Multiple Feedback Network (GMFN) will appear in BMVC2019. [Project Website]

"With two time steps and each contains 7 RDBs, the proposed GMFN achieves better reconstruction performance compared to state-of-the-art image SR methods including RDN which contains 16 RDBs."

This repository is Pytorch code for our proposed SRFBN.

The code is developed by Paper99 and penguin1214 based on BasicSR, and tested on Ubuntu 16.04/18.04 environment (Python 3.6/3/7, PyTorch 0.4.0/1.0.0/1.0.1, CUDA 8.0/9.0/10.0) with 2080Ti/1080Ti GPUs.

The architecture of our proposed SRFBN. Blue arrows represent feedback connections. The details about our proposed SRFBN can be found in our main paper.

If you find our work useful in your research or publications, please consider citing:

@inproceedings{li2019srfbn,
    author = {Li, Zhen and Yang, Jinglei and Liu, Zheng and Yang, Xiaomin and Jeon, Gwanggil and Wu, Wei},
    title = {Feedback Network for Image Super-Resolution},
    booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
    year= {2019}
}

@inproceedings{wang2018esrgan,
    author = {Wang, Xintao and Yu, Ke and Wu, Shixiang and Gu, Jinjin and Liu, Yihao and Dong, Chao and Qiao, Yu and Loy, Chen Change},
    title = {ESRGAN: Enhanced super-resolution generative adversarial networks},
    booktitle = {The European Conference on Computer Vision Workshops (ECCVW)},
    year = {2018}
}

Contents

  1. Requirements
  2. Test
  3. Train
  4. Results
  5. Acknowledgements

Requirements

  • Python 3 (Anaconda is recommended)
  • skimage
  • imageio
  • Pytorch (Pytorch version >=0.4.1 is recommended)
  • tqdm
  • pandas
  • cv2 (pip install opencv-python)
  • Matlab

Test

Quick start

  1. Clone this repository:

    git clone https://github.com/Paper99/SRFBN_CVPR19.git
  2. Download our pre-trained models from the links below, unzip the models and place them to ./models.

    Model Param. Links
    SRFBN 3,631K [GoogleDrive] [BaiduYun](code:6qta)
    SRFBN-S 483K [GoogleDrive] [BaiduYun](code:r4cp)
  3. Then, cd to SRFBN_CVPR19 and run one of following commands for evaluation on Set5:

    # SRFBN
    python test.py -opt options/test/test_SRFBN_x2_BI.json
    python test.py -opt options/test/test_SRFBN_x3_BI.json
    python test.py -opt options/test/test_SRFBN_x4_BI.json
    python test.py -opt options/test/test_SRFBN_x3_BD.json
    python test.py -opt options/test/test_SRFBN_x3_DN.json
    
    # SRFBN-S
    python test.py -opt options/test/test_SRFBN-S_x2_BI.json
    python test.py -opt options/test/test_SRFBN-S_x3_BI.json
    python test.py -opt options/test/test_SRFBN-S_x4_BI.json
  4. Finally, PSNR/SSIM values for Set5 are shown on your screen, you can find the reconstruction images in ./results.

Test on standard SR benchmark

  1. If you have cloned this repository and downloaded our pre-trained models, you can first download SR benchmark (Set5, Set14, B100, Urban100 and Manga109) from GoogleDrive or BaiduYun(code:z6nz).

  2. Run ./results/Prepare_TestData_HR_LR.m in Matlab to generate HR/LR images with different degradation models.

  3. Edit ./options/test/test_SRFBN_example.json for your needs according to ./options/test/README.md.

  4. Then, run command:

    cd SRFBN_CVPR19
    python test.py -opt options/test/test_SRFBN_example.json
  5. Finally, PSNR/SSIM values are shown on your screen, you can find the reconstruction images in ./results. You can further evaluate SR results using ./results/Evaluate_PSNR_SSIM.m.

Test on your own images

  1. If you have cloned this repository and downloaded our pre-trained models, you can first place your own images to ./results/LR/MyImage.

  2. Edit ./options/test/test_SRFBN_example.json for your needs according to ./options/test/README.md.

  3. Then, run command:

    cd SRFBN_CVPR19
    python test.py -opt options/test/test_SRFBN_example.json
  4. Finally, you can find the reconstruction images in ./results.

Train

  1. Download training set DIV2K [Official Link] or DF2K [GoogleDrive] [BaiduYun] (provided by BasicSR).

  2. Run ./scripts/Prepare_TrainData_HR_LR.m in Matlab to generate HR/LR training pairs with corresponding degradation model and scale factor. (Note: Please place generated training data to SSD (Solid-State Drive) for fast training)

  3. Run ./results/Prepare_TestData_HR_LR.m in Matlab to generate HR/LR test images with corresponding degradation model and scale factor, and choose one of SR benchmark for evaluation during training.

  4. Edit ./options/train/train_SRFBN_example.json for your needs according to ./options/train/README.md.

  5. Then, run command:

    cd SRFBN_CVPR19
    python train.py -opt options/train/train_SRFBN_example.json
  6. You can monitor the training process in ./experiments.

  7. Finally, you can follow the test pipeline to evaluate your model.

Results

Quantitative Results

Average PSNR/SSIM for scale factors x2, x3 and x4 with BI degradation model. The best performance is shown in red and the second best performance is shown in blue.

Average PSNR/SSIM values for scale factor x3 with BD and DN degradation models. The best performance is shown in red and the second best performance is shown in blue.

More Qualitative Results

Qualitative results with BI degradation model (x4) on “img 004” from Urban100.

Qualitative results with BD degradation model (x3) on “MisutenaideDaisy” from Manga109.

Qualitative results with DN degradation model (x3) on “head” from Set14.

TODO

  • Curriculum learning for complex degradation models (i.e. BD and DN degradation models).

Acknowledgements

  • Thank penguin1214, who accompanies me to develop this repository.
  • Thank Xintao. Our code structure is derived from his repository BasicSR.
  • Thank authors of BasicSR/RDN/EDSR. They provide many useful codes which facilitate our work.
Owner
Zhen Li
Glad to see you.
Zhen Li
Homepage of paper: Paint Transformer: Feed Forward Neural Painting with Stroke Prediction, ICCV 2021.

Paint Transformer: Feed Forward Neural Painting with Stroke Prediction [Paper] [Official Paddle Implementation] [Huggingface Gradio Demo] [Unofficial

442 Dec 16, 2022
[CVPR 2021 Oral] ForgeryNet: A Versatile Benchmark for Comprehensive Forgery Analysis

ForgeryNet: A Versatile Benchmark for Comprehensive Forgery Analysis ForgeryNet: A Versatile Benchmark for Comprehensive Forgery Analysis [arxiv|pdf|v

Yinan He 78 Dec 22, 2022
Unconstrained Text Detection with Box Supervisionand Dynamic Self-Training

SelfText Beyond Polygon: Unconstrained Text Detection with Box Supervisionand Dynamic Self-Training Introduction This is a PyTorch implementation of "

weijiawu 34 Nov 09, 2022
Train neural network for semantic segmentation (deep lab V3) with pytorch in less then 50 lines of code

Train neural network for semantic segmentation (deep lab V3) with pytorch in 50 lines of code Train net semantic segmentation net using Trans10K datas

17 Dec 19, 2022
A Python Package for Convex Regression and Frontier Estimation

pyStoNED pyStoNED is a Python package that provides functions for estimating multivariate convex regression, convex quantile regression, convex expect

Sheng Dai 17 Jan 08, 2023
unet-family: Ultimate version

unet-family: Ultimate version 基于之前my-unet代码,我整理出来了这一份终极版本unet-family,方便其他人阅读。 相比于之前的my-unet代码,代码分类更加规范,有条理 对于clone下来的代码不需要修改各种复杂繁琐的路径问题,直接就可以运行。 并且代码有

2 Sep 19, 2022
discovering subdomains, hidden paths, extracting unique links

python-website-crawler discovering subdomains, hidden paths, extracting unique links pip install -r requirements.txt discover subdomain: You can give

merve 4 Sep 05, 2022
Image restoration with neural networks but without learning.

Warning! The optimization may not converge on some GPUs. We've personally experienced issues on Tesla V100 and P40 GPUs. When running the code, make s

Dmitry Ulyanov 7.4k Jan 01, 2023
Vision transformers (ViTs) have found only limited practical use in processing images

CXV Convolutional Xformers for Vision Vision transformers (ViTs) have found only limited practical use in processing images, in spite of their state-o

Cloudwalker 23 Sep 10, 2022
The MLOps platform for innovators 🚀

​ DS2.ai is an integrated AI operation solution that supports all stages from custom AI development to deployment. It is an AI-specialized platform service that collects data, builds a training datas

9 Jan 03, 2023
Official implementation for paper: A Latent Transformer for Disentangled Face Editing in Images and Videos.

A Latent Transformer for Disentangled Face Editing in Images and Videos Official implementation for paper: A Latent Transformer for Disentangled Face

InterDigital 108 Dec 09, 2022
A set of simple scripts to process the Imagenet-1K dataset as TFRecords and make index files for NVIDIA DALI.

Overview This is a set of simple scripts to process the Imagenet-1K dataset as TFRecords and make index files for NVIDIA DALI. Make TFRecords To run t

8 Nov 01, 2022
Adversarial Self-Defense for Cycle-Consistent GANs

Adversarial Self-Defense for Cycle-Consistent GANs This is the official implementation of the CycleGAN robust to self-adversarial attacks used in pape

Dina Bashkirova 10 Oct 10, 2022
EfficientMPC - Efficient Model Predictive Control Implementation

efficientMPC Efficient Model Predictive Control Implementation The original algo

Vin 8 Dec 04, 2022
Simulation of moving particles under microscopic imaging

Simulation of moving particles under microscopic imaging Install scipy numpy scikit-image tiffile Run python simulation.py Read result https://imagej

Zehao Wang 2 Dec 14, 2021
Leveraging Social Influence based on Users Activity Centers for Point-of-Interest Recommendation

SUCP Leveraging Social Influence based on Users Activity Centers for Point-of-Interest Recommendation () Direct Friends (i.e., users who follow each o

Kosar 8 Nov 26, 2022
Explanatory Learning: Beyond Empiricism in Neural Networks

Explanatory Learning This is the official repository for "Explanatory Learning: Beyond Empiricism in Neural Networks". Datasets Download the datasets

GLADIA Research Group 10 Dec 06, 2022
An index of recommendation algorithms that are based on Graph Neural Networks.

An index of recommendation algorithms that are based on Graph Neural Networks.

FIB LAB, Tsinghua University 564 Jan 07, 2023
This code uses generative adversarial networks to generate diverse task allocation plans for Multi-agent teams.

Mutli-agent task allocation This code uses generative adversarial networks to generate diverse task allocation plans for Multi-agent teams. To change

Biorobotics Lab 5 Oct 12, 2022
[NeurIPS 2021] Official implementation of paper "Learning to Simulate Self-driven Particles System with Coordinated Policy Optimization".

Code for Coordinated Policy Optimization Webpage | Code | Paper | Talk (English) | Talk (Chinese) Hi there! This is the source code of the paper “Lear

DeciForce: Crossroads of Machine Perception and Autonomy 81 Dec 19, 2022