In [[artificial neural networks]], '''recurrent neural networks''' ('''RNNs''') are designed for processing sequential data, such as text, speech, and [[time series]],<ref>{{Cite journal |last1=Tealab |first1=Ahmed |date=2018-12-01 |title=Time series forecasting using artificial neural networks methodologies: A systematic review |journal=Future Computing and Informatics Journal |volume=3 |issue=2 |pages=334–340 |doi=10.1016/j.fcij.2018.10.003 |issn=2314-7288 |doi-access=free}}</ref> where the order of elements is important. Unlike [[feedforward neural network]]s, which process inputs independently, RNNs utilize recurrent connections, where the output of a neuron at one time step is fed back as input to the network at the next time step. This enables RNNs to capture temporal dependencies and patterns within sequences.
The fundamental building block of RNNsRNN is the ''recurrent unit'', which maintains a ''hidden state''—a form of memory that is updated at each time step based on the current input and the previous hidden state. This feedback mechanism allows the network to learn from past inputs and incorporate that knowledge into its current processing. RNNs have been successfully applied to tasks such as unsegmented, connected [[handwriting recognition]],<ref>{{cite journal |last1=Graves |first1=Alex |author-link1=Alex Graves (computer scientist) |last2=Liwicki |first2=Marcus |last3=Fernandez |first3=Santiago |last4=Bertolami |first4=Roman |last5=Bunke |first5=Horst |last6=Schmidhuber |first6=Jürgen |author-link6=Jürgen Schmidhuber |year=2009 |title=A Novel Connectionist System for Improved Unconstrained Handwriting Recognition |url=http://www.idsia.ch/~juergen/tpami_2008.pdf |journal=IEEE Transactions on Pattern Analysis and Machine Intelligence |volume=31 |issue=5 |pages=855–868 |citeseerx=10.1.1.139.4502 |doi=10.1109/tpami.2008.137 |pmid=19299860 |s2cid=14635907}}</ref> [[speech recognition]],<ref name="sak2014">{{Cite web |last1=Sak |first1=Haşim |last2=Senior |first2=Andrew |last3=Beaufays |first3=Françoise |year=2014 |title=Long Short-Term Memory recurrent neural network architectures for large scale acoustic modeling |url=https://research.google.com/pubs/archive/43905.pdf |publisher=Google Research}}</ref><ref name="liwu2015">{{cite arXiv |eprint=1410.4281 |class=cs.CL |first1=Xiangang |last1=Li |first2=Xihong |last2=Wu |title=Constructing Long Short-Term Memory based Deep Recurrent Neural Networks for Large Vocabulary Speech Recognition |date=2014-10-15}}</ref> [[natural language processing]], and [[neural machine translation]].<ref>{{Cite journal |last=Dupond |first=Samuel |date=2019 |title=<!-- for sure correct title? not found, nor in archive.org (for 2020-02-13), nor Volume correct? 2019 is vol 47-48 and 41 from 2016--> A thorough review on the current advance of neural network structures. |url=https://www.sciencedirect.com/journal/annual-reviews-in-control |journal=Annual Reviews in Control |volume=14 |pages=200–230}}</ref><ref>{{Cite journal |last1=Abiodun |first1=Oludare Isaac |last2=Jantan |first2=Aman |last3=Omolara |first3=Abiodun Esther |last4=Dada |first4=Kemi Victoria |last5=Mohamed |first5=Nachaat Abdelatif |last6=Arshad |first6=Humaira |date=2018-11-01 |title=State-of-the-art in artificial neural network applications: A survey |journal=Heliyon |volume=4 |issue=11 |pages=e00938 |bibcode=2018Heliy...400938A |doi=10.1016/j.heliyon.2018.e00938 |issn=2405-8440 |pmc=6260436 |pmid=30519653 |doi-access=free}}</ref>
However, traditional RNNs suffer from the [[vanishing gradient problem]], which limits their ability to learn long-range dependencies. This issue was addressed by the development of the [[long short-term memory]] (LSTM) architecture in 1997, making it the standard RNN variant for handling long-term dependencies. Later, [[gated recurrent unit]]s (GRUs) were introduced as a more computationally efficient alternative.
