Content deleted Content added
link to reparameterization trick !w |
→Overview of architecture and operation: I am pretty sure I made the right references, but since I am learning this and missed it, please validate. |
||
Line 10:
== Overview of architecture and operation ==
A variational autoencoder is a generative model with a prior and noise distribution respectively. Usually such models are trained using the [[Expectation–maximization algorithm|expectation-maximization]] meta-algorithm (e.g. [[Principal_component_analysis|probabilistic PCA]], (spike & slab) sparse coding). Such a scheme optimizes a lower bound of the data likelihood, which is usually intractable, and in doing so requires the discovery of q-distributions, or variational [[Posterior_probability|posteriors]]. These q-distributions are normally parameterized for each individual data point in a separate optimization process. However, variational autoencoders use a neural network as an amortized approach to jointly optimize across data points. This neural network takes as input the data points themselves, and outputs parameters for the variational distribution. As it maps from a known input space to the low-dimensional latent space, it is called the encoder.
The decoder is the second neural network of this model. It is a function that maps from the latent space to the input space, e.g. as the means of the noise distribution. It is possible to use another neural network that maps to the variance, however this can be omitted for simplicity. In such a case, the variance can be optimized with gradient descent.
| |||