Hierarchical Conditional Flow: A Unified Framework for Image Super-Resolution and Image Rescaling (HCFlow, ICCV2021)
This repository is the official PyTorch implementation of Hierarchical Conditional Flow: A Unified Framework for Image Super-Resolution and Image Rescaling (arxiv, supp).
- Aug. 17, 2021: See our recent work for spatially variant kernel estimation: Mutual Affine Network for Spatially Variant Kernel Estimation in Blind Image Super-Resolution (MANet), ICCV2021
- Aug. 17, 2021: See our recent work for real-world image SR: Designing a Practical Degradation Model for Deep Blind Image Super-Resolution (BSRGAN), ICCV2021
- Aug. 17, 2021: See our previous flow-based work: Flow-based Kernel Prior with Application to Blind Super-Resolution (FKP), CVPR2021.
Normalizing flows have recently demonstrated promising results for low-level vision tasks. For image super-resolution (SR), it learns to predict diverse photo-realistic high-resolution (HR) images from the low-resolution (LR) image rather than learning a deterministic mapping. For image rescaling, it achieves high accuracy by jointly modelling the downscaling and upscaling processes. While existing approaches employ specialized techniques for these two tasks, we set out to unify them in a single formulation. In this paper, we propose the hierarchical conditional flow (HCFlow) as a unified framework for image SR and image rescaling. More specifically, HCFlow learns a bijective mapping between HR and LR image pairs by modelling the distribution of the LR image and the rest high-frequency component simultaneously. In particular, the high-frequency component is conditional on the LR image in a hierarchical manner. To further enhance the performance, other losses such as perceptual loss and GAN loss are combined with the commonly used negative log-likelihood loss in training. Extensive experiments on general image SR, face image SR and image rescaling have demonstrated that the proposed HCFlow achieves state-of-the-art performance in terms of both quantitative metrics and visual quality.
        Â
Requirements
- Python 3.7, PyTorch == 1.7.1
- Requirements: opencv-python, lpips, natsort, etc.
- Platforms: Ubuntu 16.04, cuda-11.0
cd HCFlow-master
pip install -r requirements.txt
Quick Run (takes 1 Minute)
To run the code with one command (without preparing data), run this command:
cd codes
# face image SR
python test_HCFLow.py --opt options/test/test_SR_CelebA_8X_HCFlow.yml
# general image SR
python test_HCFLow.py --opt options/test/test_SR_DF2K_4X_HCFlow.yml
# image rescaling
python test_HCFLow.py --opt options/test/test_Rescaling_DF2K_4X_HCFlow.yml
Data Preparation
The framework of this project is based on MMSR and SRFlow. To prepare data, put training and testing sets in ./datasets as ./datasets/DIV2K/HR/0801.png. Commonly used SR datasets can be downloaded here. There are two ways for accerleration in data loading: First, one can use ./scripts/png2npy.py to generate .npy files and use data/GTLQnpy_dataset.py. Second, one can use .pklv4 dataset (recommended) and use data/LRHR_PKL_dataset.py. Please refer to SRFlow for more details. Prepared datasets can be downloaded here.
Training
To train HCFlow for general image SR/ face image SR/ image rescaling, run this command:
cd codes
# face image SR
python train_HCFLow.py --opt options/train/train_SR_CelebA_8X_HCFlow.yml
# general image SR
python train_HCFLow.py --opt options/train/train_SR_DF2K_4X_HCFlow.yml
# image rescaling
python train_HCFLow.py --opt options/train/train_Rescaling_DF2K_4X_HCFlow.yml
All trained models can be downloaded from here.
Testing
Please follow the Quick Run section. Just modify the dataset path in test_HCFlow_*.yml.
Results
We achieved state-of-the-art performance on general image SR, face image SR and image rescaling.
For more results, please refer to the paper and supp for details.
Citation
@inproceedings{liang21hcflow,
title={Hierarchical Conditional Flow: A Unified Framework for Image Super-Resolution and Image Rescaling},
author={Liang, Jingyun and Lugmayr, Andreas and Zhang, Kai and Danelljan, Martin and Van Gool, Luc and Timofte, Radu},
booktitle={IEEE Conference on International Conference on Computer Vision},
year={2021}
}
License & Acknowledgement
This project is released under the Apache 2.0 license. The codes are based on MMSR, SRFlow, IRN and Glow-pytorch. Please also follow their licenses. Thanks for their great works.
3 Feb 26, 2022
1 Feb 08, 2022
246 Dec 11, 2022
34 Dec 08, 2022
2 Jun 29, 2022
32 Dec 15, 2022
49 Nov 10, 2022
75 Dec 09, 2022
10 Mar 16, 2022
590 Dec 22, 2022
30 Nov 12, 2022
27 Jan 05, 2023
9 Nov 07, 2022
185 Dec 21, 2022
251 Jan 08, 2023
62 Nov 30, 2022
41 Nov 30, 2022
1.2k Dec 26, 2022
37 Dec 15, 2022
5 Oct 25, 2022