An autoencoder is defined by the following components: <blockquote>Two sets: the space of decoded messages <math>\mathcal X</math>; the space of encoded messages <math>\mathcal Z</math>. Almost always, bothTypically <math>\mathcal X</math> and <math>\mathcal Z</math> are Euclidean spaces, that is, <math>\mathcal X = \R^m, \mathcal Z = \R^n</math> for somewith <math>m, > n.</math>. </blockquote><blockquote>Two parametrized families of functions: the encoder family <math>E_\phi:\mathcal{X} \rightarrow \mathcal{Z}</math>, parametrized by <math>\phi</math>; the decoder family <math>D_\theta:\mathcal{Z} \rightarrow \mathcal{X}</math>, parametrized by <math>\theta</math>.</blockquote>For any <math>x\in \mathcal X</math>, we usually write <math>z = E_\phi(x)</math>, and refer to it as the code, the [[latent variable]], latent representation, latent vector, etc. Conversely, for any <math>z\in \mathcal Z</math>, we usually write <math>x' = D_\theta(z)</math>, and refer to it as the (decoded) message.
Usually, both the encoder and the decoder are defined as [[multilayer perceptron]]s. For example, a one-layer-MLP encoder <math>E_\phi</math> is:
Line 18:
:<math>E_\phi(\mathbf x) = \sigma(Wx+b)</math>
where <math>\sigma</math> is an element-wise [[activation function]] such as a [[sigmoid function]] or a [[rectified linear unit]], <math>W</math> is a matrix called "weight" matrix, and <math>b</math> is a vector called "bias" vector.
