Content deleted Content added
→Continuous-time: Linking. |
|||
Line 313:
From a time-series perspective, RNNs can appear as nonlinear versions of [[finite impulse response]] and [[infinite impulse response]] filters and also as a [[nonlinear autoregressive exogenous model]] (NARX).<ref>{{cite journal |url={{google books |plainurl=y |id=830-HAAACAAJ |page=208}} |title=Computational Capabilities of Recurrent NARX Neural Networks |last1=Siegelmann |first1=Hava T. |last2=Horne |first2=Bill G. |last3=Giles |first3=C. Lee |journal= IEEE Transactions on Systems, Man, and Cybernetics - Part B: Cybernetics|volume=27 |issue=2 |pages=208–15 |year=1995 |pmid=18255858 |doi=10.1109/3477.558801 |citeseerx=10.1.1.48.7468 }}</ref> RNN has infinite impulse response whereas [[convolutional neural network]] has [[finite impulse response]]. Both classes of networks exhibit temporal [[dynamic system|dynamic behavior]].<ref>{{Cite journal |last=Miljanovic |first=Milos |date=Feb–Mar 2012 |title=Comparative analysis of Recurrent and Finite Impulse Response Neural Networks in Time Series Prediction |url=http://www.ijcse.com/docs/INDJCSE12-03-01-028.pdf |journal=Indian Journal of Computer and Engineering |volume=3 |issue=1}}</ref> A finite impulse recurrent network is a [[directed acyclic graph]] that can be unrolled and replaced with a strictly feedforward neural network, while an infinite impulse recurrent network is a [[directed cyclic graph]] that cannot be unrolled.
The effect of memory-based learning for the recognition of sequences can also be implemented by a more biological-based model which uses the silencing mechanism exhibited in neurons with a relatively high frequency [[Action potential|spiking activity]].<ref>{{Cite journal |last1=Hodassman |first1=Shiri |last2=Meir |first2=Yuval |last3=Kisos |first3=Karin |last4=Ben-Noam |first4=Itamar |last5=Tugendhaft |first5=Yael |last6=Goldental |first6=Amir |last7=Vardi |first7=Roni |last8=Kanter |first8=Ido |date=2022-09-29 |title=Brain inspired neuronal silencing mechanism to enable reliable sequence identification |journal=Scientific Reports |volume=12 |issue=1 |pages=16003 |doi=10.1038/s41598-022-20337-x |pmid=36175466 |pmc=9523036 |arxiv=2203.13028 |bibcode=2022NatSR..1216003H |issn=2045-2322|doi-access=free }}</ref>
Additional stored states and the storage under direct control by the network can be added to both [[infinite impulse response|infinite-impulse]] and [[finite impulse response|finite-impulse]] networks. Another network or graph can also replace the storage if that incorporates time delays or has feedback loops. Such controlled states are referred to as gated states or gated memory and are part of [[long short-term memory]] networks (LSTMs) and [[gated recurrent unit]]s. This is also called Feedback Neural Network (FNN).
| |||