Bidirectional recurrent neural networks: Difference between revisions

Revision as of 22:17, 8 March 2016

Revision as of 22:18, 8 March 2016

Line 1:

'''Bidirectional Recurrent Neural Networks'''('''BRNN''') were invented in 1997 by Schuster & Paliwal <ref name="Schuster"> Schuster, Mike, and Kuldip K. Paliwal. "Bidirectional recurrent neural networks." Signal Processing, IEEE Transactions on 45.11 (1997): 2673-2681.2. Awni Hannun, Carl Case, Jared Casper, Bryan Catanzaro, Greg Diamos, Erich Elsen, Ryan </ref>. BRNN is introduced to increase the amount of input information available to be referred to. For example, [[multilayer perceptron]](MLP) and [[time delay neural network]](TDNN) have limitation on the input data flexibility, as they require their input data to be fixed. Standard [[recurrent neural network]](RNN) also has restrictions as its future input information cannot be reached from theirits current state. On the contrary, BRNN do not require their input data to be fixed. Moreover, their future input information is reachable from the current state. The basic idea of BRNN is to connect two hidden layers of opposite directions to the same output. By this structure, the output layer can get information from past and future states.

BRNN are especially useful when the context of the input is needed. For example, in handwriting recognition, the performance can be enhanced by knowledge of the letters located before and after the current letter.