Recurrent neural network: Difference between revisions

Revision as of 21:07, 10 March 2025

Revision as of 13:29, 29 March 2025

Line 3:

{{Machine learning|Artificial neural network}}

'''Recurrent neural networks''' ('''RNNs)''') are a class of artificial neural networks designed for processing sequential data, such as text, speech, and [[time series]],<ref>{{Cite journal |last1=Tealab |first1=Ahmed |date=2018-12-01 |title=Time series forecasting using artificial neural networks methodologies: A systematic review |journal=Future Computing and Informatics Journal |volume=3 |issue=2 |pages=334–340 |doi=10.1016/j.fcij.2018.10.003 |issn=2314-7288 |doi-access=free}}</ref> where the order of elements is important. Unlike [[feedforward neural network]]s, which process inputs independently, RNNs utilize recurrent connections, where the output of a neuron at one time step is fed back as input to the network at the next time step. This enables RNNs to capture temporal dependencies and patterns within sequences.

The fundamental building block of RNNs is the '''recurrent unit''', which maintains a '''hidden state'''—a form of memory that is updated at each time step based on the current input and the previous hidden state. This feedback mechanism allows the network to learn from past inputs and incorporate that knowledge into its current processing. RNNs have been successfully applied to tasks such as unsegmented, connected [[handwriting recognition]],<ref>{{cite journal |last1=Graves |first1=Alex |author-link1=Alex Graves (computer scientist) |last2=Liwicki |first2=Marcus |last3=Fernandez |first3=Santiago |last4=Bertolami |first4=Roman |last5=Bunke |first5=Horst |last6=Schmidhuber |first6=Jürgen |author-link6=Jürgen Schmidhuber |year=2009 |title=A Novel Connectionist System for Improved Unconstrained Handwriting Recognition |url=http://www.idsia.ch/~juergen/tpami_2008.pdf |journal=IEEE Transactions on Pattern Analysis and Machine Intelligence |volume=31 |issue=5 |pages=855–868 |citeseerx=10.1.1.139.4502 |doi=10.1109/tpami.2008.137 |pmid=19299860 |s2cid=14635907}}</ref> [[speech recognition]],<ref name="sak2014">{{Cite web |last1=Sak |first1=Haşim |last2=Senior |first2=Andrew |last3=Beaufays |first3=Françoise |year=2014 |title=Long Short-Term Memory recurrent neural network architectures for large scale acoustic modeling |url=https://research.google.com/pubs/archive/43905.pdf |publisher=Google Research}}</ref><ref name="liwu2015">{{cite arXiv |eprint=1410.4281 |class=cs.CL |first1=Xiangang |last1=Li |first2=Xihong |last2=Wu |title=Constructing Long Short-Term Memory based Deep Recurrent Neural Networks for Large Vocabulary Speech Recognition |date=2014-10-15}}</ref> [[natural language processing]], and [[neural machine translation]].<ref>{{Cite journal |last=Dupond |first=Samuel |date=2019 |title=<!-- for sure correct title? not found, nor in archive.org (for 2020-02-13), nor Volume correct? 2019 is vol 47-48 and 41 from 2016--> A thorough review on the current advance of neural network structures. |url=https://www.sciencedirect.com/journal/annual-reviews-in-control |journal=Annual Reviews in Control |volume=14 |pages=200–230}}</ref><ref>{{Cite journal |last1=Abiodun |first1=Oludare Isaac |last2=Jantan |first2=Aman |last3=Omolara |first3=Abiodun Esther |last4=Dada |first4=Kemi Victoria |last5=Mohamed |first5=Nachaat Abdelatif |last6=Arshad |first6=Humaira |date=2018-11-01 |title=State-of-the-art in artificial neural network applications: A survey |journal=Heliyon |volume=4 |issue=11 |pages=e00938 |bibcode=2018Heliy...400938A |doi=10.1016/j.heliyon.2018.e00938 |issn=2405-8440 |pmc=6260436 |pmid=30519653 |doi-access=free}}</ref>