Overview

Like the previous blog, this paper largely draws on the idea of ​​the pioneering work "Context Encoders: Feature Learning by Inpainting", using the encoder-decoder structure for image generation, but the previous oneThe paper has big flaws, the most obvious of which is that the result of the completion is rather blurry (with poor texture technology), and the scale of the input image is fixed at 128*128, and it cannot handle high-resolution images (please note that, for context encoders, high-resolution results are obtained by direct upsampling from low-resolution outputs.).In view of this, this paper innovatively proposes a multi-scale neural patch, which performs both content learning and texture learning, and finally forms a model with excellent content and texture.
Note: This idea is similar to the "Globally and Locally Consistent Image Completion" published at the ACM summit in the same year. The core innovation of this paper is to use the CE generator to innovatively propose twoA discriminator, global and local, is actually a global consideration of the correctness of content filling, and a local (blank area and a small part of the surrounding) considering the texture, which can be understood as the fit of the details.

I. Method details

1. Network Structure

There are two parts of network classification: (1) content generation network (the missing square mask in the center of the image is filled with the average pixel color and then input into the network); (2) texture generation network, the content generation network adopts pioneering workThe CE generator method, the texture generation network adopts VGG-19 pre-trained using ImageNet.
The relu3_1 and relu4_1 layers are used in the texture generation network to calculate the texture.