Content deleted Content added
Revert reversion for introduction of errors into the article, introduction of citations that are unrelated to the text, and eterochronistic citation of Introduction to VAEs instead of Autoencoding Variational Bayes by Kingma and Welling. |
→Architecture: Added citation. |
||
Line 14:
The decoder is the second neural network of this model. It is a function that maps from the latent space to the input space, e.g. as the means of the noise distribution. It is possible to use another neural network that maps to the variance, however this is can be omitted for simplicity. In such a case, the variance can be optimized with gradient descent.
To optimize this model, one needs to know two terms: the "reconstruction error", and the [[Kullback–Leibler divergence]]. Both terms are derived from the free energy expression of the probabilistic model, and therefore differ depending on the noise distribution and the assumed prior of the data. The KL-D from the free energy expression maximizes the probability mass of the q distribution that overlaps with the p distribution, which unfortunately can result in mode-seeking behaviour. The "reconstruction" term is the remainder of the free energy expression, and requires a sampling approximation to compute its expectation value.<ref>{{cite journal |last1=Kingma |first1=Diederik |title=Autoencoding Variational Bayes |journal=Arxiv |date=2013}}</ref>
== Formulation ==
| |||