Content deleted Content added
MishchenkoA (talk | contribs) mNo edit summary |
MishchenkoA (talk | contribs) mNo edit summary |
||
Line 24:
: <math>p_\theta(\mathbf{x}) = \int_{\mathbf{z}}p_\theta(\mathbf{x\mid z})p_\theta(\mathbf{z}) \, d\mathbf{z}</math>
In the vanilla variational autoencoder, we assume <math>\mathbf{z}</math> with finite dimension and that <math>p_\theta(\mathbf{x|z})</math> is a [[Gaussian distribution]], then <math>p_\theta(\mathbf{x})</math> is a mixture of Gaussian distributions.
It is now possible to define the set of the relationships between the input data and its latent representation as
Line 44:
For variational autoencoders the idea is to jointly minimize the generative model parameters <math>\theta</math> to reduce the reconstruction error between the input and the output of the network, and <math>\Phi</math> to have <math>q_\Phi(\mathbf{z\mid x})</math> as close as possible to <math>p_\theta(\mathbf{z}\mid\mathbf{x})</math>.
As reconstruction loss, [[mean squared error]] and [[cross entropy]]
As distance loss between the two distributions the reverse Kullback–Leibler divergence <math>D_{KL}(q_\Phi(\mathbf{z\mid x})\parallel p_\theta(\mathbf{z\mid x}))</math> is a good choice to squeeze <math>q_\Phi(\mathbf{z\mid x})</math> under <math>p_\theta(\mathbf{z}\mid\mathbf{x})</math>.<ref name=":0" /><ref>{{cite web |title=From Autoencoder to Beta-VAE |url=https://lilianweng.github.io/lil-log/2018/08/12/from-autoencoder-to-beta-vae.html |website=Lil'Log |language=en |date=2018-08-12}}</ref>
| |||