What is an automatic encoder?

The automatic encoder (AutoEncoder) was originally used as a data compression method. Its features are:

The degree of correlation with the data is very high, which means that the automatic encoder can only compress data similar to the training data. This is obviously obvious, because the features extracted using the neural network are generally highly related to the original training set, using face training. The automatic encoder will perform poorly in compressing pictures of natural animals, because it only learns the characteristics of the face, but does not have the ability to learn the characteristics of the natural picture;

The compressed data is lossy because it is inevitable to lose information in the process of dimensionality reduction;

In 2012, it was discovered that using autoencoders in convolutional networks for layer-by-layer pre-training can train deeper networks, but soon people find that good initialization strategies are more effective than strenuous layer-by-layer pre-training, 2014. The Batch Normalization technology that emerged is also a deeper network that can be effectively trained. By the end of the 15th, we can basically train any depth of neural network through ResNet.

So now the main application of automatic encoder has two aspects, the first is data denoising, and the second is visual reduction. However, an automatic encoder has a function to generate data.

We talked about GAN before, it has some advantages compared with GAN, but also has some shortcomings. Let us first talk about its advantages compared with GAN.

First, the bad thing about using GAN to generate images is that we generate random Gaussian noise for the images, which means we can't generate any images of the type we specify, which means we can't decide which one to use. Random noise can produce the image we want, unless we can try it all out. But using an automatic encoder, we can get the distribution after encoding of this type of image through the encoding process of the output image. It is equivalent to knowing the noise distribution corresponding to each image, we can generate specific noise by selecting specific noise. The image to be generated.

Secondly, this is not only the advantage of generating a network but also has certain limitations. This is to generate a network to distinguish between "real" pictures and "false" pictures by confrontation process. However, the pictures obtained in this way are just as true as possible. But this does not guarantee that the content of the image is what we want. In other words, it is possible to generate a network to generate some background patterns as much as possible to make it as realistic as possible, but there is no actual object inside.