Recurrent neural network: Difference between revisions

Revision as of 07:32, 16 November 2024

Revision as of 07:27, 17 November 2024

Line 85:

=== PixelRNN ===

An RNN may process data with more than one dimension. PixelRNN processes two-dimensional data, with many possible directions.<ref>{{Cite journal |last1=Oord |first1=Aäron van den |last2=Kalchbrenner |first2=Nal |last3=Kavukcuoglu |first3=Koray |date=2016-06-11 |title=Pixel Recurrent Neural Networks |url=https://proceedings.mlr.press/v48/oord16.html |journal=Proceedings of the 33rd International Conference on Machine Learning |publisher=PMLR |pages=1747–1756}}</ref> For example, the row-by-row direction processes an <math>n \times n</math> grid of vectors <math>x_{i, j}</math> in the following order: <math display="block">x_{1, 1}, x_{1, 2}, \dots, x_{1, n}, x_{2, 1}, x_{2, 2}, \dots, x_{2, n}, \dots, x_{n, n}</math>The '''diagonal BiLSTM''' uses two LSTMLSTMs to process the same grid. One processes it from the top-left corner to the bottom-right, such that it processes <math>x_{i, j}</math> depending on its hidden state and cell state on the top and the left side: <math>h_{i-1, j}, c_{i-1, j}</math> and <math>h_{i, j-1}, c_{i, j-1}</math>. The other processes it from the top-right corner to the bottom-left.

== Architectures ==