Bidirectional recurrent neural networks: Difference between revisions

Revision as of 18:35, 18 April 2021

Revision as of 18:35, 18 April 2021

Line 1:

{{Orphan|date=March 2016}}

'''Bidirectional [[recurrent neural networks]]''' ('''BRNN''') connect two hidden layers of opposite directions to the same output. With this form of [[Generative model|generative deep learning]], the output layer can get information from past (backwards) and future (forward) states simultaneously. Invented in 1997 by Schuster and Paliwal,<ref name="Schuster">Schuster, Mike, and Kuldip K. Paliwal. "[https://www.researchgate.net/profile/Mike_Schuster/publication/3316656_Bidirectional_recurrent_neural_networks/links/56861d4008ae19758395f85c.pdf Bidirectional recurrent neural networks]." Signal Processing, IEEE Transactions on 45.11 (1997): 2673-2681.2. Awni Hannun, Carl Case, Jared Casper, Bryan Catanzaro, Greg Diamos, Erich Elsen, Ryan</ref> BRNNs were introduced to increase the amount of input information available to the network. For example, [[multilayer perceptron]] (MLPs) and [[time delay neural network]] (TDNNs) have limitations on the input data flexibility, as they require their input data to be fixed. Standard [[recurrent neural network]] (RNNs) also have restrictions as the future input information cannot be reached from the current state. On the contrary, BRNNs do not require their input data to be fixed. Moreover, their future input information is reachable from the current state. <ref>{{Cite document|title=Recent Advances in Recurrent Neural Networks|arxiv = 1801.01078|last1 = Salehinejad|first1 = Hojjat|last2 = Sankar|first2 = Sharan|last3 = Barfett|first3 = Joseph|last4 = Colak|first4 = Errol|last5 = Valaee|first5 = Shahrokh|year = 2017|bibcode = 2018arXiv180101078S}}</ref>

Line 16 ⟶ 17:

Applications of BRNN include :

*Speech Recognition (Combined with [[Long short-term memory]])<ref>Graves, Alex, Santiago Fernández, and Jürgen Schmidhuber. "[https://mediatum.ub.tum.de/doc/1290195/file.pdf Bidirectional LSTM networks for improved phoneme classification and recognition]." Artificial Neural Networks: Formal Models and Their Applications–ICANN 2005. Springer Berlin Heidelberg, 2005. 799-804.

Line 22:

*Translation<ref>Sundermeyer, Martin, et al. "[https://www.aclweb.org/anthology/D14-1003 Translation modeling with bidirectional recurrent neural networks]." Proceedings of the Conference on Empirical Methods on Natural Language Processing, October. 2014.</ref>

*Spoken Language Identification<ref>Garain, Avishek, et al. {{cite journal |title=FuzzyGCP: A deep learning architecture for automatic spoken language identification from speech signals |journal=Expert Systems with Applications |date=15 April 2021 |volume=168 |pages=114416 |doi=10.1016/j.eswa.2020.114416}}</ref>

*Handwritten Recognition<ref>Liwicki, Marcus, et al. "[https://mediatum.ub.tum.de/doc/1289961/file.pdf A novel approach to on-line handwriting recognition based on bidirectional long short-term memory networks]." Proc. 9th Int. Conf. on Document Analysis and Recognition. Vol. 1. 2007.</ref>

*Protein Structure Prediction<ref>Baldi, Pierre, et al. "[https://academic.oup.com/bioinformatics/article-pdf/15/11/937/693153/150937.pdf Exploiting the past and the future in protein secondary structure prediction]." Bioinformatics 15.11 (1999): 937-946.</ref><ref>Pollastri, Gianluca, and Aoife Mclysaght. "[https://academic.oup.com/bioinformatics/article/21/8/1719/250163 Porter: a new, accurate server for protein secondary structure prediction]." Bioinformatics 21.8 (2005): 1719-1720.</ref>

*Part-of-speech tagging

*Dependency Parsing<ref>{{Cite journal|last1=Kiperwasser|first1=Eliyahu|last2=Goldberg|first2=Yoav|date=2016|title=Simple and Accurate Dependency Parsing Using Bidirectional LSTM Feature Representations|url=https://www.aclweb.org/anthology/Q16-1023/|journal=Transactions of the Association for Computational Linguistics|language=en-us|volume=4|pages=313–327|doi=10.1162/tacl_a_00101|arxiv=1603.04351|bibcode=2016arXiv160304351K|s2cid=1642392}}</ref>