Pytorch code for our paper "Feedback Network for Image Super-Resolution" (CVPR2019)

Overview

Feedback Network for Image Super-Resolution [arXiv] [CVF] [Poster]

Update: Our proposed Gated Multiple Feedback Network (GMFN) will appear in BMVC2019. [Project Website]

"With two time steps and each contains 7 RDBs, the proposed GMFN achieves better reconstruction performance compared to state-of-the-art image SR methods including RDN which contains 16 RDBs."

This repository is Pytorch code for our proposed SRFBN.

The code is developed by Paper99 and penguin1214 based on BasicSR, and tested on Ubuntu 16.04/18.04 environment (Python 3.6/3/7, PyTorch 0.4.0/1.0.0/1.0.1, CUDA 8.0/9.0/10.0) with 2080Ti/1080Ti GPUs.

The architecture of our proposed SRFBN. Blue arrows represent feedback connections. The details about our proposed SRFBN can be found in our main paper.

If you find our work useful in your research or publications, please consider citing:

@inproceedings{li2019srfbn,
    author = {Li, Zhen and Yang, Jinglei and Liu, Zheng and Yang, Xiaomin and Jeon, Gwanggil and Wu, Wei},
    title = {Feedback Network for Image Super-Resolution},
    booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
    year= {2019}
}

@inproceedings{wang2018esrgan,
    author = {Wang, Xintao and Yu, Ke and Wu, Shixiang and Gu, Jinjin and Liu, Yihao and Dong, Chao and Qiao, Yu and Loy, Chen Change},
    title = {ESRGAN: Enhanced super-resolution generative adversarial networks},
    booktitle = {The European Conference on Computer Vision Workshops (ECCVW)},
    year = {2018}
}

Contents

  1. Requirements
  2. Test
  3. Train
  4. Results
  5. Acknowledgements

Requirements

  • Python 3 (Anaconda is recommended)
  • skimage
  • imageio
  • Pytorch (Pytorch version >=0.4.1 is recommended)
  • tqdm
  • pandas
  • cv2 (pip install opencv-python)
  • Matlab

Test

Quick start

  1. Clone this repository:

    git clone https://github.com/Paper99/SRFBN_CVPR19.git
  2. Download our pre-trained models from the links below, unzip the models and place them to ./models.

    Model Param. Links
    SRFBN 3,631K [GoogleDrive] [BaiduYun](code:6qta)
    SRFBN-S 483K [GoogleDrive] [BaiduYun](code:r4cp)
  3. Then, cd to SRFBN_CVPR19 and run one of following commands for evaluation on Set5:

    # SRFBN
    python test.py -opt options/test/test_SRFBN_x2_BI.json
    python test.py -opt options/test/test_SRFBN_x3_BI.json
    python test.py -opt options/test/test_SRFBN_x4_BI.json
    python test.py -opt options/test/test_SRFBN_x3_BD.json
    python test.py -opt options/test/test_SRFBN_x3_DN.json
    
    # SRFBN-S
    python test.py -opt options/test/test_SRFBN-S_x2_BI.json
    python test.py -opt options/test/test_SRFBN-S_x3_BI.json
    python test.py -opt options/test/test_SRFBN-S_x4_BI.json
  4. Finally, PSNR/SSIM values for Set5 are shown on your screen, you can find the reconstruction images in ./results.

Test on standard SR benchmark

  1. If you have cloned this repository and downloaded our pre-trained models, you can first download SR benchmark (Set5, Set14, B100, Urban100 and Manga109) from GoogleDrive or BaiduYun(code:z6nz).

  2. Run ./results/Prepare_TestData_HR_LR.m in Matlab to generate HR/LR images with different degradation models.

  3. Edit ./options/test/test_SRFBN_example.json for your needs according to ./options/test/README.md.

  4. Then, run command:

    cd SRFBN_CVPR19
    python test.py -opt options/test/test_SRFBN_example.json
  5. Finally, PSNR/SSIM values are shown on your screen, you can find the reconstruction images in ./results. You can further evaluate SR results using ./results/Evaluate_PSNR_SSIM.m.

Test on your own images

  1. If you have cloned this repository and downloaded our pre-trained models, you can first place your own images to ./results/LR/MyImage.

  2. Edit ./options/test/test_SRFBN_example.json for your needs according to ./options/test/README.md.

  3. Then, run command:

    cd SRFBN_CVPR19
    python test.py -opt options/test/test_SRFBN_example.json
  4. Finally, you can find the reconstruction images in ./results.

Train

  1. Download training set DIV2K [Official Link] or DF2K [GoogleDrive] [BaiduYun] (provided by BasicSR).

  2. Run ./scripts/Prepare_TrainData_HR_LR.m in Matlab to generate HR/LR training pairs with corresponding degradation model and scale factor. (Note: Please place generated training data to SSD (Solid-State Drive) for fast training)

  3. Run ./results/Prepare_TestData_HR_LR.m in Matlab to generate HR/LR test images with corresponding degradation model and scale factor, and choose one of SR benchmark for evaluation during training.

  4. Edit ./options/train/train_SRFBN_example.json for your needs according to ./options/train/README.md.

  5. Then, run command:

    cd SRFBN_CVPR19
    python train.py -opt options/train/train_SRFBN_example.json
  6. You can monitor the training process in ./experiments.

  7. Finally, you can follow the test pipeline to evaluate your model.

Results

Quantitative Results

Average PSNR/SSIM for scale factors x2, x3 and x4 with BI degradation model. The best performance is shown in red and the second best performance is shown in blue.

Average PSNR/SSIM values for scale factor x3 with BD and DN degradation models. The best performance is shown in red and the second best performance is shown in blue.

More Qualitative Results

Qualitative results with BI degradation model (x4) on “img 004” from Urban100.

Qualitative results with BD degradation model (x3) on “MisutenaideDaisy” from Manga109.

Qualitative results with DN degradation model (x3) on “head” from Set14.

TODO

  • Curriculum learning for complex degradation models (i.e. BD and DN degradation models).

Acknowledgements

  • Thank penguin1214, who accompanies me to develop this repository.
  • Thank Xintao. Our code structure is derived from his repository BasicSR.
  • Thank authors of BasicSR/RDN/EDSR. They provide many useful codes which facilitate our work.
Owner
Zhen Li
Glad to see you.
Zhen Li
Library of deep learning models and datasets designed to make deep learning more accessible and accelerate ML research.

Tensor2Tensor Tensor2Tensor, or T2T for short, is a library of deep learning models and datasets designed to make deep learning more accessible and ac

12.9k Jan 09, 2023
Bayesian optimization in PyTorch

BoTorch is a library for Bayesian Optimization built on PyTorch. BoTorch is currently in beta and under active development! Why BoTorch ? BoTorch Prov

2.5k Dec 31, 2022
PyTorch implementation of convolutional neural networks-based text-to-speech synthesis models

Deepvoice3_pytorch PyTorch implementation of convolutional networks-based text-to-speech synthesis models: arXiv:1710.07654: Deep Voice 3: Scaling Tex

Ryuichi Yamamoto 1.8k Jan 08, 2023
[CVPR 2021] Modular Interactive Video Object Segmentation: Interaction-to-Mask, Propagation and Difference-Aware Fusion

[CVPR 2021] Modular Interactive Video Object Segmentation: Interaction-to-Mask, Propagation and Difference-Aware Fusion

Rex Cheng 364 Jan 03, 2023
A universal framework for learning timestamp-level representations of time series

TS2Vec This repository contains the official implementation for the paper Learning Timestamp-Level Representations for Time Series with Hierarchical C

Zhihan Yue 284 Dec 30, 2022
Faune proche - Retrieval of Faune-France data near a google maps location

faune_proche Récupération des données de Faune-France près d'un lieu google maps

4 Feb 15, 2022
This toolkit provides codes to download and pre-process the SLUE datasets, train the baseline models, and evaluate SLUE tasks.

slue-toolkit We introduce Spoken Language Understanding Evaluation (SLUE) benchmark. This toolkit provides codes to download and pre-process the SLUE

ASAPP Research 39 Sep 21, 2022
Multimodal Temporal Context Network (MTCN)

Multimodal Temporal Context Network (MTCN) This repository implements the model proposed in the paper: Evangelos Kazakos, Jaesung Huh, Arsha Nagrani,

Evangelos Kazakos 13 Nov 24, 2022
PECOS - Prediction for Enormous and Correlated Spaces

PECOS - Predictions for Enormous and Correlated Output Spaces PECOS is a versatile and modular machine learning (ML) framework for fast learning and i

Amazon 387 Jan 04, 2023
Reviving Iterative Training with Mask Guidance for Interactive Segmentation

This repository provides the source code for training and testing state-of-the-art click-based interactive segmentation models with the official PyTorch implementation

Visual Understanding Lab @ Samsung AI Center Moscow 406 Jan 01, 2023
Ranger - a synergistic optimizer using RAdam (Rectified Adam), Gradient Centralization and LookAhead in one codebase

Ranger-Deep-Learning-Optimizer Ranger - a synergistic optimizer combining RAdam (Rectified Adam) and LookAhead, and now GC (gradient centralization) i

Less Wright 1.1k Dec 21, 2022
Official implementation of EdiTTS: Score-based Editing for Controllable Text-to-Speech

EdiTTS: Score-based Editing for Controllable Text-to-Speech Official implementation of EdiTTS: Score-based Editing for Controllable Text-to-Speech. Au

Neosapience 98 Dec 25, 2022
🧠 A PyTorch implementation of 'Deep CORAL: Correlation Alignment for Deep Domain Adaptation.', ECCV 2016

Deep CORAL A PyTorch implementation of 'Deep CORAL: Correlation Alignment for Deep Domain Adaptation. B Sun, K Saenko, ECCV 2016' Deep CORAL can learn

Andy Hsu 200 Dec 25, 2022
Publication describing 3 ML examples at NSLS-II and interfacing into Bluesky

Machine learning enabling high-throughput and remote operations at large-scale user facilities. Overview This repository contains the source code and

BNL 4 Sep 24, 2022
Deep Reinforcement Learning for Multiplayer Online Battle Arena

MOBA_RL Deep Reinforcement Learning for Multiplayer Online Battle Arena Prerequisite Python 3 gym-derk Tensorflow 2.4.1 Dotaservice of TimZaman Seed R

Dohyeong Kim 32 Dec 18, 2022
An efficient PyTorch library for Global Wheat Detection using YOLOv5. The project is based on this Kaggle competition Global Wheat Detection (2021).

Global-Wheat-Detection An efficient PyTorch library for Global Wheat Detection using YOLOv5. The project is based on this Kaggle competition Global Wh

Chuxin Wang 11 Sep 25, 2022
Wordle-solver - Wordle answer generation program in python

🟨 Wordle Solver 🟩 Wordle answer generation program in python ✔️ Requirements U

Dahyun Kang 4 May 28, 2022
TensorFlow for Raspberry Pi

TensorFlow on Raspberry Pi It's officially supported! As of TensorFlow 1.9, Python wheels for TensorFlow are being officially supported. As such, this

Sam Abrahams 2.2k Dec 16, 2022
Weighted K Nearest Neighbors (kNN) algorithm implemented on python from scratch.

kNN_From_Scratch I implemented the k nearest neighbors (kNN) classification algorithm on python. This algorithm is used to predict the classes of new

1 Dec 14, 2021