Flow-based generative model: Difference between revisions

Revision as of 22:20, 11 March 2021

Revision as of 22:35, 11 March 2021

Line 10:

== Method ==

=== Derivation of log probability ===

Let <math>z_0</math> be a (possibly multivariate) [[random variable]] with distribution <math>p_0(z_0)</math>.

Let <math>z_1 = f_1(z_0)</math> be a random variable which is a function of <math>z_0</math>. The function <math>f_1</math> should be invertible, i.e. the [[inverse function]] <math>f^{-1}_1</math> exists. Note that <math>z_0 = f^{-1}_1(z_1)</math>.

By the [[Probability density function#Function of random variables and change of variables in the probability density function|change of variable]] formula, the distribution of <math>z_1</math> is:

: <math>p_1(z_1) = p_0(f^{-1}_1(z_1)z_0)\left|\det \frac{df_1^{-1}(z_1)}{dz_1}\right|</math>

Where <math>\det \frac{df_1^{-1}(z_1)}{dz_1}</math> is the [[determinant]] of the [[Jacobian matrix and determinant|Jacobian matrix]] of <math>f^{-1}_1</math>.

By the [[inverse function theorem]]:

: <math>p_1(z_1) = p_0(z_0)\left|\det \left(\frac{df_1(z_0)}{dz_0}\right)^{-1}\right|</math>

By the identity <math>\det(A^{-1}) = \det(A)^-1</math> (where <math>A</math> is an invertible matrix), we have:

: <math>p_1(z_1) = p_0(z_0)\left|\det \frac{df_1(z_0)}{dz_0}\right|^{-1}</math>

The log probability is thus:

: <math>\log p_1(z_1) = \log p_0(z_0) - \log \left|\det \frac{df_1(z_0)}{dz_0}\right|</math>

== Examples ==