Variational autoencoder: Difference between revisions

Revision as of 10:11, 19 July 2022

Revision as of 10:14, 19 July 2022

Line 31:

:<math>q_\phi({z| x}) \approx p_\theta({z| x})</math>

with <math>\phi</math> defined as the set of real values that parametrize <math>q</math>. This is sometimes called ''amortized inference'', since by "investing" in finding a good <math>q_\phi</math>, one can later infer <math>z</math> from <math>x</math> quickly without doing any integrals.

In this way, the overall problem canis beof easilyfinding translateda intogood theprobabilistic autoencoder ___domain, in which the conditional likelihood distribution <math>p_\theta(x|z)</math> is carriedcomputed by the ''probabilistic decoder'', whileand the approximated posterior distribution <math>q_\phi(z|x)</math> is computed by the ''probabilistic encoder''.

== Evidence lower bound (ELBO) ==

Line 40:

As in every [[deep learning]] problem, it is necessary to define a differentiable loss function in order to update the network weights through [[backpropagation]].

For variational autoencoders, the idea is to jointly optimize the generative model parameters <math>\theta</math> to reduce the reconstruction error between the input and the output, and <math>\phi</math> to make <math>q_\phi({z| x})</math> as close as possible to <math>p_\theta(z|x)</math>. As reconstruction loss, [[mean squared error]] and [[cross entropy]] are often used.

As reconstruction loss, [[mean squared error]] and [[cross entropy]] are often used.

As distance loss between the two distributions the reverse Kullback–Leibler divergence <math>D_{KL}(q_\phi({z| x})\parallel p_\theta({z| x}))</math> is a good choice to squeeze <math>q_\phi({z| x})</math> under <math>p_\theta(z|x)</math>.<ref name=":0" /><ref>{{cite web |title=From Autoencoder to Beta-VAE |url=https://lilianweng.github.io/lil-log/2018/08/12/from-autoencoder-to-beta-vae.html |website=Lil'Log |language=en |date=2018-08-12}}</ref>