Content deleted Content added
→masked autoregressive flow (MAF): touchup |
|||
Line 79:
=== Real Non-Volume Preserving (Real NVP) ===
The Real Non-Volume Preserving model generalizes NICE model by:<ref name=":2" /><math display="block">x = \begin{bmatrix}
x_1 \\ x_2
\end{bmatrix}= f_\theta(z) = \begin{bmatrix}
Line 85:
\end{bmatrix} + \begin{bmatrix}
0 \\ m_\theta(z_1)
\end{bmatrix}</math>
\end{bmatrix}</math>Its inverse is <math>z_1 = x_1, z_2 = e^{-s_\theta (x_1)}\odot (x_2 - m_\theta (x_1))</math>, and its Jacobian is <math>\prod^n_{i=1} e^{s_\theta(z_{1, })}</math>.▼
▲
Since the Real NVP map keeps the first and second halves of the vector <math>x</math> separate, it's usually required to add a permutation <math>(x_1, x_2) \mapsto (x_2, x_1)</math> after every Real NVP layer.
=== Generative Flow (Glow) ===
In generative flow model,<ref name="glow" />
* channel-wise affine transform<math display="block">y_{cij} = s_c(x_{cij} + b_c)</math>with Jacobian <math>\prod_c s_c^{HW}</math>.
* invertible 1x1 convolution<math display="block">z_{cij} = \sum_{c'} K_{cc'} y_{cij}</math>with Jacobian <math>\det(K)^{HW}</math>. Here <math>K</math> is
* Real NVP, with Jacobian as described in Real NVP.
Line 101:
The idea of using the invertible 1x1 convolution is to permute all layers in general, instead of merely permuting the first and second half, as in Real NVP.
===
An autoregressive model of a distribution on <math>\R^n</math> is defined as the following stochastic process:<ref>{{Cite journal |last=Papamakarios |first=George |last2=Pavlakou |first2=Theo |last3=Murray |first3=Iain |date=2017 |title=Masked Autoregressive Flow for Density Estimation |url=https://proceedings.neurips.cc/paper/2017/hash/6c1da886822c67822bcf3679d04369fa-Abstract.html |journal=Advances in Neural Information Processing Systems |publisher=Curran Associates, Inc. |volume=30}}</ref>
<math display="block">\begin{align}
| |||