awesome gan-inversion papers

This repo is a collection of resources on GAN inversion, as a supplement for our survey:

@article{xia2021survey,
  author    = {Xia, Weihao and Zhang, Yulun and Yang, Yujiu and Xue, Jing-Hao and Zhou, Bolei and Yang, Ming-Hsuan},
  title     = {GAN Inversion: A Survey},
  journal={arXiv preprint arXiv: 2101.05278},
  year={2021}
}

Contributing

Feedback and contributions are welcome!

If you think I have missed out on something (or) have any suggestions (papers, implementations and other resources), feel free to pull a request.

I have released the latex files. Please pull a request, open an issue, or send me an email if you find any inappropriate expressions of the survey.

markdown format:

**Here is the Paper Name.**

*[Author 1](homepage), Author 2, and Author 3.*

Conference or Journal Year. [[PDF](link)] [[Project](link)] [[Github](link)] [[Video](link)] [[Data](link)]

Survey

[Papers on Generative Modeling]

GAN Inversion: A Survey.
Weihao Xia, Yulun Zhang, Yujiu Yang, Jing-Hao Xue, Bolei Zhou, Ming-Hsuan Yang.
arxiv 2021. [PDF]

inverted pretrained model

StyleGAN2-Ada: Training Generative Adversarial Networks with Limited Data.
Tero Karras, Miika Aittala, Janne Hellsten, Samuli Laine, Jaakko Lehtinen, Timo Aila.
NeurIPS 2020. [PDF] [Github] [Steam StyleGAN2-ADA]

StyleGAN2: Analyzing and Improving the Image Quality of StyleGAN.
Tero Karras, Samuli Laine, Miika Aittala, Janne Hellsten, Jaakko Lehtinen, Timo Aila.
CVPR 2020. [PDF] [Offical TF] [PyTorch] [Unoffical Tensorflow 2.0]

StyleGAN: A Style-Based Generator Architecture for Generative Adversarial Networks.
Tero Karras, Samuli Laine, Timo Aila.
CVPR 2019. [PDF] [Offical TF]

ProGAN: Progressive Growing of GANs for Improved Quality, Stability, and Variation.
Tero Karras, Timo Aila, Samuli Laine, Jaakko Lehtinen.
ICLR 2018. [PDF] [Offical TF]

inversion method

This part contatins generatal inversion methods, while methods in the next application part are mainly designed for specific tasks.

Using Latent Space Regression to Analyze and Leverage Compositionality in GANs.
Lucy Chai, Jonas Wulff, Phillip Isola.
ICLR 2021. [PDF] [Github] [Project] [Colab]

Hijack-GAN: Unintended-Use of Pretrained, Black-Box GANs.
Hui-Po Wang, Ning Yu, Mario Fritz.
CVPR 2021. [PDF]

e4e: Designing an Encoder for StyleGAN Image Manipulation.
Omer Tov, Yuval Alaluf, Yotam Nitzan, Or Patashnik, Daniel Cohen-Or.
arxiv 2021. [PDF] [Github]

Enjoy Your Editing: Controllable GANs for Image Editing via Latent Space Navigation.
Peiye Zhuang, Oluwasanmi Koyejo, Alexander G. Schwing.
ICLR 2021. [PDF]

Improved StyleGAN Embedding: Where are the Good Latents?
Peihao Zhu, Rameen Abdal, Yipeng Qin, Peter Wonka.
arxiv 2020. [PDF]

Learning a Deep Reinforcement Learning Policy Over the Latent Space of a Pre-trained GAN for Semantic Age Manipulation.
Kumar Shubham, Gopalakrishnan Venkatesh, Reijul Sachdev, Akshi, Dinesh Babu Jayagopi, G. Srinivasaraghavan.
arxiv 2020. [PDF]

Lifting 2D StyleGAN for 3D-Aware Face Generation.
Yichun Shi, Divyansh Aggarwal, Anil K. Jain.
arxiv 2020. [PDF]

Navigating the GAN Parameter Space for Semantic Image Editing.
Anton Cherepkov, Andrey Voynov, Artem Babenko.
arxiv 2020. [PDF] [Github]

Augmentation-Interpolative AutoEncoders for Unsupervised Few-Shot Image Generation.
Davis Wertheimer, Omid Poursaeed, Bharath Hariharan.
arxiv 2020. [PDF]

Mask-Guided Discovery of Semantic Manifolds in Generative Models.
Mengyu Yang, David Rokeby, Xavier Snelgrove.
Workshop on Machine Learning for Creativity and Design (NeurIPS) 2020. [PDF] [Github]

Unsupervised Discovery of Disentangled Manifolds in GANs.
Yu-Ding Lu, Hsin-Ying Lee, Hung-Yu Tseng, Ming-Hsuan Yang.
arxiv 2020. [PDF]]

StyleSpace Analysis: Disentangled Controls for StyleGAN Image Generation.
Zongze Wu, Dani Lischinski, Eli Shechtman.
arxiv 2020. [PDF]

GAN Steerability without optimization.
Nurit Spingarn-Eliezer, Ron Banner, Tomer Michaeli.
ICLR 2021. [OpenReview] [PDF]

On The Inversion Of Deep Generative Models (When and How Can Deep Generative Models be Inverted?).
Aviad Aberdam, Dror Simon, Michael Elad.
arxiv 2020. [PDF] [OpenReview]

PIE: Portrait Image Embedding for Semantic Control.
A. Tewari, M. Elgharib, M. BR, F. Bernard, H-P. Seidel, P. P‌érez, M. Zollhöfer, C.Theobalt.
SIGGRAPH Asia 2020. [PDF] [Project]

Encoding in Style: a StyleGAN Encoder for Image-to-Image Translation.
Elad Richardson, Yuval Alaluf, Or Patashnik, Yotam Nitzan, Yaniv Azar, Stav Shapiro, Daniel Cohen-Or.
CVPR 2021. [PDF] [Github] [Project]

GAN-Control: Explicitly Controllable GANs.
Alon Shoshan, Nadav Bhonker, Igor Kviatkovsky, Gerard Medioni.
arxiv 2021. [PDF]

Understanding the Role of Individual Units in a Deep Neural Network.
David Bau, Jun-Yan Zhu, Hendrik Strobelt, Agata Lapedriza, Bolei Zhou, Antonio Torralba.
National Academy of Sciences 2020. [PDF] [Github] [Project]

GHFeat: Generative Hierarchical Features from Synthesizing Images.
Yinghao Xu, Yujun Shen, Jiapeng Zhu, Ceyuan Yang, Bolei Zhou.
CVPR 2021. [PDF] [Github] [Project]

SeFa: Closed-Form Factorization of Latent Semantics in GANs.
Yujun Shen, Bolei Zhou.
CVPR 2021. [PDF] [Github] [Project]

Collaborative Learning for Faster StyleGAN Embedding.
Shanyan Guan, Ying Tai, Bingbing Ni, Feida Zhu, Feiyue Huang, Xiaokang Yang.
arxiv 2020. [PDF]

Disentangling in Latent Space by Harnessing a Pretrained Generator.
Yotam Nitzan, Amit Bermano, Yangyan Li, Daniel Cohen-Or.
arxiv 2020. [PDF]

Face Identity Disentanglement via Latent Space Mapping.
Yotam Nitzan, Amit Bermano, Yangyan Li, Daniel Cohen-Or.
SIGGRAPH Asia (TOG) 2020. [PDF] [Github]

Transforming and Projecting Images into Class-conditional Generative Networks.
Minyoung Huh, Richard Zhang, Jun-Yan Zhu, Sylvain Paris, Aaron Hertzmann.
ECCV 2020. [PDF] [Github] [Project]

Interpreting the Latent Space of GANs via Correlation Analysis for Controllable Concept Manipulation.
Ziqiang Li, Rentuo Tao, Hongjing Niu, Bin Li.
arxiv 2020. [PDF]

Improving Inversion and Generation Diversity in StyleGAN using a Gaussianized Latent Space.
Jonas Wulff, Antonio Torralba.
arxiv 2020. [PDF]

GANSpace: Discovering Interpretable GAN Controls.
Erik Härkönen, Aaron Hertzmann, Jaakko Lehtinen, Sylvain Paris.
NeurIPS 2020. [PDF] [Github]

MimicGAN: Robust Projection onto Image Manifolds with Corruption Mimicking.
Rushil Anirudh, Jayaraman J. Thiagarajan, Bhavya Kailkhura, Timo Bremer.
IJCV 2020. [PDF]

StyleFlow: Attribute-conditioned Exploration of StyleGAN-Generated Images using Conditional Continuous Normalizing Flows.
Rameen Abdal, Peihao Zhu, Niloy Mitra, Peter Wonka.
Siggraph (TOG) 2021. [PDF] [Github]

Rewriting a Deep Generative Model.
David Bau, Steven Liu, Tongzhou Wang, Jun-Yan Zhu, Antonio Torralba.
ECCV 2020. [PDF] [Github]

StyleGAN2 Distillation for Feed-forward Image Manipulation.
Yuri Viazovetskyi, Vladimir Ivashkin, Evgeny Kashin.
ECCV 2020. [PDF] [Github]

In-Domain GAN Inversion for Real Image Editing.
Jiapeng Zhu, Yujun Shen, Deli Zhao, Bolei Zhou.
ECCV 2020. [PDF] [Project] [Github]

Exploiting Deep Generative Prior for Versatile Image Restoration and Manipulation.
Xingang Pan, Xiaohang Zhan, Bo Dai, Dahua Lin, Chen Change Loy, Ping Luo.
ECCV 2020. [PDF] [Github]

On the "steerability" of generative adversarial networks.
Ali Jahanian, Lucy Chai, Phillip Isola.
ICLR 2020. [PDF] [Project]

Unsupervised Discovery of Interpretable Directions in the GAN Latent Space.
Andrey Voynov, Artem Babenko.
ICML 2020. [PDF] [Github]

Your Local GAN: Designing Two Dimensional Local Attention Mechanisms for Generative Models.
Giannis Daras, Augustus Odena, Han Zhang, Alexandros G. Dimakis.
CVPR 2020. [PDF]

A Disentangling Invertible Interpretation Network for Explaining Latent Representations.
Patrick Esser, Robin Rombach, Björn Ommer.
CVPR 2020. [PDF] [Project] [Github]

Editing in Style: Uncovering the Local Semantics of GANs.
Edo Collins, Raja Bala, Bob Price, Sabine Süsstrunk.
CVPR 2020. [PDF] [Github]

Image Processing Using Multi-Code GAN Prior.
Jinjin Gu, Yujun Shen, Bolei Zhou.
CVPR 2020. [PDF] [Project] [Github]

Interpreting the Latent Space of GANs for Semantic Face Editing.
Yujun Shen, Jinjin Gu, Xiaoou Tang, Bolei Zhou.
CVPR 2020. [PDF] [Project] [Github]

Image2StyleGAN++: How to Edit the Embedded Images?
Rameen Abdal, Yipeng Qin, Peter Wonka.
CVPR 2020. [PDF]

Semantic Photo Manipulation with a Generative Image Prior.
David Bau, Hendrik Strobelt, William Peebles, Jonas, Bolei Zhou, Jun-Yan Zhu, Antonio Torralba.
SIGGRAPH 2019. [PDF]

Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space?
Rameen Abdal, Yipeng Qin, Peter Wonka.
ICCV 2019. [PDF] [Github]

Seeing What a GAN Cannot Generate.
David Bau, Jun-Yan Zhu, Jonas Wulff, William Peebles, Hendrik Strobelt, Bolei Zhou, Antonio Torralba.
ICCV 2019. [PDF] [PDF]

GAN-based Projector for Faster Recovery with Convergence Guarantees in Linear Inverse Problems.
Ankit Raj, Yuqi Li, Yoram Bresler.
ICCV 2019. [PDF]

Inverting Layers of a Large Generator.
David Bau, Jun-Yan Zhu, Jonas Wulff, William Peebles, Hendrik Strobelt, Bolei Zhou, Antonio Torralba.
ICCV 2019. [PDF]

Inverting The Generator Of A Generative Adversarial Network (II).
Antonia Creswell, Anil A Bharath.
TNNLS 2018. [PDF] [Github]

Invertibility of Convolutional Generative Networks from Partial Measurements.
Fangchang Ma, Ulas Ayaz, Sertac Karaman.
NeurIPS 2018. [PDF] [Github]

Metrics for Deep Generative Models.
Nutan Chen, Alexej Klushyn, Richard Kurle, Xueyan Jiang, Justin Bayer, Patrick van der Smagt.
AISTATS 2018. [PDF]

Towards Understanding the Invertibility of Convolutional Neural Networks.
Anna C. Gilbert, Yi Zhang, Kibok Lee, Yuting Zhang, Honglak Lee.
IJCAI 2017. [PDF]

One Network to Solve Them All - Solving Linear Inverse Problems using Deep Projection Models.
J. H. Rick Chang, Chun-Liang Li, Barnabas Poczos, B. V. K. Vijaya Kumar, Aswin C. Sankaranarayanan.
ICCV 2017. [PDF]

Precise Recovery of Latent Vectors from Generative Adversarial Networks.
Zachary C. Lipton, Subarna Tripathi.
ICLR 2017 workshop. [PDF] [Github]

Inverting The Generator Of A Generative Adversarial Network.
Antonia Creswell, Anil Anthony Bharath.
NIPSW 2016. [PDF]

Generative Visual Manipulation on the Natural Image Manifold.
Jun-Yan Zhu, Philipp Krähenbühl, Eli Shechtman, Alexei A. Efros.
ECCV 2016. [PDF]

application

content generation

Paint by Word.
David Bau, Alex Andonian, Audrey Cui, YeonHwan Park, Ali Jahanian, Aude Oliva, Antonio Torralba.
arxiv 2021. [PDF]

Unsupervised Image Transformation Learning via Generative Adversarial Networks.
Kaiwen Zha, Yujun Shen, Bolei Zhou.
arxiv 2021. [PDF] [Project]

TediGAN: Text-Guided Diverse Image Generation and Manipulation.
Weihao Xia, Yujiu Yang, Jing-Hao Xue, Baoyuan Wu.
CVPR 2021. [PDF] [Data] [Github]

LOHO: Latent Optimization of Hairstyles via Orthogonalization.
Rohit Saha, Brendan Duke, Florian Shkurti, Graham W. Taylor, Parham Aarabi.
CVPR 2021. [PDF] [Github]

SAM: Only a Matter of Style-Age Transformation Using a Style-Based Regression Model.
Yuval Alaluf, Or Patashnik, Daniel Cohen-Or.
arxiv 2021. [PDF] [Github]

OSTeC: One-Shot Texture Completion.
Baris Gecer, Jiankang Deng, Stefanos Zafeiriou.
arxiv 2021. [PDF] [Github]

GAN2Shape: Do 2D GANs Know 3D Shape? Unsupervised 3D shape reconstruction from 2D Image GANs.
Xingang Pan, Bo Dai, Ziwei Liu, Chen Change Loy, Ping Luo.
ICLR 2021 (oral). [PDF] [Github] [Project]

Exploring Adversarial Fake Images on Face Manifold.
Dongze Li, Wei Wang, Hongxing Fan, Jing Dong.
arxiv 2021. [PDF]

Generating Images from Caption and Vice Versa via CLIP-Guided Generative Latent Space Search.
Federico A. Galatolo, Mario G.C.A. Cimino, Gigliola Vaglini.
arxiv 2021. [PDF]

Unsupervised Image-to-Image Translation via Pre-trained StyleGAN2 Network.
Jialu Huang, Jing Liao, Sam Kwong.
arxiv 2020. [PDF]

DeepI2I: Enabling Deep Hierarchical Image-to-Image Translation by Transferring from GANs.
yaxing wang, Lu Yu, Joost van de Weijer.
NeurIPS 2020. [PDF] [Github]

DeepLandscape: Adversarial Modeling of Landscape Videos.
E. Logacheva, R. Suvorov, O. Khomenko, A. Mashikhin, and V. Lempitsky.
ECCV 2020. [PDF] [Github] [Project]

image restoration

GLEAN: Generative Latent Bank for Large-Factor Image Super-Resolution.
Kelvin C.K. Chan, Xintao Wang, Xiangyu Xu, Jinwei Gu, Chen Change Loy.
CVPR 2021. [PDF] [Project] [Github]

GFP-GAN: Towards Real-World Blind Face Restoration with Generative Facial Prior.
Xintao Wang, Yu Li, Honglun Zhang, Ying Shan.
arxiv 2021. [PDF] [Project]

image understanding

Repurposing GANs for One-shot Semantic Part Segmentation.
Nontawat Tritrong, Pitchaporn Rewatbowornwong, Supasorn Suwajanakorn.
CVPR 2021 (oral). [PDF] [Project] [Github]

compressed sensing

Generator Surgery for Compressed Sensing.
Niklas Smedemark-Margulies, Jung Yeon Park, Max Daniels, Rose Yu, Jan-Willem van de Meent, Paul Hand.
arxiv 2021. [PDF] [Github]

Task-Aware Compressed Sensing with Generative Adversarial Networks.
Maya Kabkab, Pouya Samangouei, Rama Chellappa.
AAAI 2018. [PDF]

acknowledgement

Thanks for the feedback from Jun-Yan Zhu, Andrey Voynov, and Rushil Anirudh.