Deep Illuminator is a data augmentation tool designed for image relighting. It can be used to easily and efficiently generate a wide range of illumination variants of a single image.

Overview

Deep Illuminator

Deep Illuminator is a data augmentation tool designed for image relighting. It can be used to easily and efficiently generate a wide range of illumination variants of a single image. It has been tested with several datasets and models and has been shown to succesfully improve performance. It has a built in visualizer created with Streamlit to preview how the target image can be relit. This tool has an accompanying paper.

Example Augmentations

Usage

The simplest method to use this tool is through Docker Hub:

docker pull kartvel/deep-illuminator

Visualizer

Once you have the Deep Illuminator image run the following command to launch the visualizer:

docker run -it --rm  --gpus all \
-p 8501:8501 --entrypoint streamlit \ 
kartvel/deep-illuminator run streamlit/streamlit_app.py

You will be able to interact with it on localhost:8501. Note: If you do not have NVIDIA gpu support enabled for docker simply remove the --gpus all option.

Generating Variants

It is possible to quickly generate multiple variants for images contained in a directory by using the following command:

docker run -it --rm --gpus all \                                                                                               ─╯
-v /path/to/input/images:/app/probe_relighting/originals \
-v /path/to/save/directory:/app/probe_relighting/output \
kartvel/deep-illuminator --[options]

Options

Option Values Description
mode ['synthetic', 'mid'] Selecting the style of probes used as a relighting guide.
step int Increment for the granularity of relighted images. max mid: 24, max synthetic: 360

Buidling Docker image or running without a container

Please read the following for other options: instructions

Benchmarks

Improved performance of R2D2 for [email protected] on HPatches

Training Dataset Overall Viewpoint Illumination
COCO - Original 71.0 65.4 77.1
COCO - Augmented 72.2 (+1.7%) 65.7 (+0.4%) 79.2 (+2.7%)
VIDIT - Original 66.7 60.5 73.4
VIDIT - Augmented 69.2 (+3.8%) 60.9 (+0.6%) 78.1 (+6.4%)
Aachen - Original 69.4 64.1 75.0
Aachen - Augmented 72.6 (+4.6%) 66.1 (+3.1%) 79.6 (+6.1%)

Improved performance of R2D2 for the Long-Term Visual Localization challenge on Aachen v1.1

Training Dataset 0.25m, 2° 0.5m, 5° 5m, 10°
COCO - Original 62.3 77.0 79.5
COCO - Augmented 65.4 (+5.0%) 83.8 (+8.8%) 92.7 (+16%)
VIDIT - Original 40.8 53.4 61.3
VIDIT - Augmented 53.9 (+32%) 71.2 (+33%) 83.2(+36%)
Aachen - Original 60.7 72.8 83.8
Aachen - Augmented 63.4 (+4.4%) 81.7 (+12%) 92.1 (+9.9%)

Acknowledgment

The developpement of the VAE for the visualizer was made possible by the PyTorch-VAE repository.

Bibtex

If you use this code in your project, please consider citing the following paper:

@misc{chogovadze2021controllable,
      title={Controllable Data Augmentation Through Deep Relighting}, 
      author={George Chogovadze and Rémi Pautrat and Marc Pollefeys},
      year={2021},
      eprint={2110.13996},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}
The code for 'Deep Residual Fourier Transformation for Single Image Deblurring'

Deep Residual Fourier Transformation for Single Image Deblurring Xintian Mao, Yiming Liu, Wei Shen, Qingli Li and Yan Wang code will be released soon

145 Dec 13, 2022
Crab is a flexible, fast recommender engine for Python that integrates classic information filtering recommendation algorithms in the world of scientific Python packages (numpy, scipy, matplotlib).

Crab - A Recommendation Engine library for Python Crab is a flexible, fast recommender engine for Python that integrates classic information filtering r

python-recsys 1.2k Dec 21, 2022
PatchMatch-RL: Deep MVS with Pixelwise Depth, Normal, and Visibility

PatchMatch-RL: Deep MVS with Pixelwise Depth, Normal, and Visibility Jae Yong Lee, Joseph DeGol, Chuhang Zou, Derek Hoiem Installation To install nece

31 Apr 19, 2022
SSD: Single Shot MultiBox Detector pytorch implementation focusing on simplicity

SSD: Single Shot MultiBox Detector Introduction Here is my pytorch implementation of 2 models: SSD-Resnet50 and SSDLite-MobilenetV2.

Viet Nguyen 149 Jan 07, 2023
Alex Pashevich 62 Dec 24, 2022
This repository contains the source codes for the paper AtlasNet V2 - Learning Elementary Structures.

AtlasNet V2 - Learning Elementary Structures This work was build upon Thibault Groueix's AtlasNet and 3D-CODED projects. (you might want to have a loo

Théo Deprelle 123 Nov 11, 2022
Implementation of Hierarchical Transformer Memory (HTM) for Pytorch

Hierarchical Transformer Memory (HTM) - Pytorch Implementation of Hierarchical Transformer Memory (HTM) for Pytorch. This Deepmind paper proposes a si

Phil Wang 63 Dec 29, 2022
CLIP-GEN: Language-Free Training of a Text-to-Image Generator with CLIP

CLIP-GEN [简体中文][English] 本项目在萤火二号集群上用 PyTorch 实现了论文 《CLIP-GEN: Language-Free Training of a Text-to-Image Generator with CLIP》。 CLIP-GEN 是一个 Language-F

75 Dec 29, 2022
Single/multi view image(s) to voxel reconstruction using a recurrent neural network

3D-R2N2: 3D Recurrent Reconstruction Neural Network This repository contains the source codes for the paper Choy et al., 3D-R2N2: A Unified Approach f

Chris Choy 1.2k Dec 27, 2022
The Python ensemble sampling toolkit for affine-invariant MCMC

emcee The Python ensemble sampling toolkit for affine-invariant MCMC emcee is a stable, well tested Python implementation of the affine-invariant ense

Dan Foreman-Mackey 1.3k Dec 31, 2022
Code for the paper "Functional Regularization for Reinforcement Learning via Learned Fourier Features"

Reinforcement Learning with Learned Fourier Features State-space Soft Actor-Critic Experiments Move to the state-SAC-LFF repository. cd state-SAC-LFF

Alex Li 10 Nov 11, 2022
PyTorch implementation of paper “Unbiased Scene Graph Generation from Biased Training”

A new codebase for popular Scene Graph Generation methods (2020). Visualization & Scene Graph Extraction on custom images/datasets are provided. It's also a PyTorch implementation of paper “Unbiased

Kaihua Tang 824 Jan 03, 2023
TorchGRL is the source code for our paper Graph Convolution-Based Deep Reinforcement Learning for Multi-Agent Decision-Making in Mixed Traffic Environments for IV 2022.

TorchGRL TorchGRL is the source code for our paper Graph Convolution-Based Deep Reinforcement Learning for Multi-Agent Decision-Making in Mixed Traffi

XXQQ 42 Dec 09, 2022
A curated list of awesome papers for Semantic Retrieval (TOIS Accepted: Semantic Models for the First-stage Retrieval: A Comprehensive Review).

A curated list of awesome papers for Semantic Retrieval (TOIS Accepted: Semantic Models for the First-stage Retrieval: A Comprehensive Review).

Yinqiong Cai 189 Dec 28, 2022
Pytorch implementation of Depth-conditioned Dynamic Message Propagation forMonocular 3D Object Detection

DDMP-3D Pytorch implementation of Depth-conditioned Dynamic Message Propagation forMonocular 3D Object Detection, a paper on CVPR2021. Instroduction T

Li Wang 32 Nov 09, 2022
PyKaldi GOP-DNN on Epa-DB

PyKaldi GOP-DNN on Epa-DB This repository has the tools to run a PyKaldi GOP-DNN algorithm on Epa-DB, a database of non-native English speech by Spani

18 Dec 14, 2022
Source code for the paper "SEPP: Similarity Estimation of Predicted Probabilities for Defending and Detecting Adversarial Text" PACLIC 2021

Adversarial text generator Refer to "adversarial_text_generator"[https://github.com/quocnsh/SEPP_generator] project for generating adversarial texts A

0 Oct 05, 2021
Python utility to generate filesystem content for Obsidian.

Security Vault Generator Quickly parse, format, and output common frameworks/content for Obsidian.md. There is a strong focus on MITRE ATT&CK because

Justin Angel 73 Dec 02, 2022
Source code and data in paper "MDFEND: Multi-domain Fake News Detection (CIKM'21)"

MDFEND: Multi-domain Fake News Detection This is an official implementation for MDFEND: Multi-domain Fake News Detection which has been accepted by CI

Rich 40 Dec 18, 2022
Code for one-stage adaptive set-based HOI detector AS-Net.

AS-Net Code for one-stage adaptive set-based HOI detector AS-Net. Mingfei Chen*, Yue Liao*, Si Liu, Zhiyuan Chen, Fei Wang, Chen Qian. "Reformulating

Mingfei Chen 45 Dec 09, 2022