Recurrent neural network: Difference between revisions

Revision as of 05:17, 11 August 2025

Revision as of 10:12, 11 August 2025

Line 311:

They are in fact [[recursive neural network]]s with a particular structure: that of a linear chain. Whereas recursive neural networks operate on any hierarchical structure, combining child representations into parent representations, recurrent neural networks operate on the linear progression of time, combining the previous time step and a hidden representation into the representation for the current time step.

From a time-series perspective, RNNs can appear as nonlinear versions of [[finite impulse response]] and [[infinite impulse response]] filters and also as a [[nonlinear autoregressive exogenous model]] (NARX).<ref>{{cite journal |url={{google books |plainurl=y |id=830-HAAACAAJ |page=208}} |title=Computational Capabilities of Recurrent NARX Neural Networks |last1=Siegelmann |first1=Hava T. |last2=Horne |first2=Bill G. |last3=Giles |first3=C. Lee |journal= IEEE Transactions on Systems, Man, and Cybernetics - Part B: Cybernetics|volume=27 |issue=2 |pages=208–15 |year=1995 |pmid=18255858 |doi=10.1109/3477.558801 |citeseerx=10.1.1.48.7468 }}</ref> RNN has infinite impulse response whereas [[convolutional neural network]]s havehas [[finite impulse response|finite impulse]] response. Both classes of networks exhibit temporal [[dynamic system|dynamic behavior]].<ref>{{Cite journal |last=Miljanovic |first=Milos |date=Feb–Mar 2012 |title=Comparative analysis of Recurrent and Finite Impulse Response Neural Networks in Time Series Prediction |url=http://www.ijcse.com/docs/INDJCSE12-03-01-028.pdf |journal=Indian Journal of Computer and Engineering |volume=3 |issue=1}}</ref> A finite impulse recurrent network is a [[directed acyclic graph]] that can be unrolled and replaced with a strictly feedforward neural network, while an infinite impulse recurrent network is a [[directed cyclic graph]] that cannot be unrolled.

The effect of memory-based learning for the recognition of sequences can also be implemented by a more biological-based model which uses the silencing mechanism exhibited in neurons with a relatively high frequency spiking activity.<ref>{{Cite journal |last1=Hodassman |first1=Shiri |last2=Meir |first2=Yuval |last3=Kisos |first3=Karin |last4=Ben-Noam |first4=Itamar |last5=Tugendhaft |first5=Yael |last6=Goldental |first6=Amir |last7=Vardi |first7=Roni |last8=Kanter |first8=Ido |date=2022-09-29 |title=Brain inspired neuronal silencing mechanism to enable reliable sequence identification |journal=Scientific Reports |volume=12 |issue=1 |pages=16003 |doi=10.1038/s41598-022-20337-x |pmid=36175466 |pmc=9523036 |arxiv=2203.13028 |bibcode=2022NatSR..1216003H |issn=2045-2322|doi-access=free }}</ref>