Time delay neural network: Difference between revisions

Revision as of 14:07, 10 May 2025

Revision as of 20:55, 23 May 2025

Line 10:

The TDNN was introduced in the late 1980s and applied to a task of [[phoneme]] classification for automatic [[speech recognition]] in speech signals where the automatic determination of precise segments or feature boundaries was difficult or impossible. Because the TDNN recognizes phonemes and their underlying acoustic/phonetic features, independent of position in time, it improved performance over static classification.<ref name="phoneme detection" /><ref name=":0">Alexander Waibel, [https://isl.iar.kit.edu/downloads/Pheome_Recognition_Using_Time-Delay_Neural_Networks_SP87-100_6.pdf Phoneme Recognition Using Time-Delay Neural Networks], Procedures of the Institute of Electrical, Information and Communication Engineers (IEICE), December, 1987, Tokyo, Japan.</ref> It was also applied to two-dimensional signals (time-frequency patterns in speech,<ref name=":1">{{cite journal |author=John B. Hampshire |author2=Alex Waibel |url=https://www.researchgate.net/publication/391319411 |title=Connectionist Architectures for Multi-Speaker Phoneme Recognition |journal=Advances in Neural Information Processing Systems |volume=2 |pages=203–210}}</ref> and coordinate space pattern in OCR<ref name=":2">{{cite journal |url=https://www.researchgate.net/publication/220163530 |doi=10.1007/PL00013559 |title=Online handwriting recognition: The NPen++ recognizer |date=2001 |last1=Jaeger |first1=S. |last2=Manke |first2=S. |last3=Reichert |first3=J. |last4=Waibel |first4=A. |journal=International Journal on Document Analysis and Recognition |volume=3 |issue=3 |pages=169–180 }}</ref>).

[[Kunihiko Fukushima]] published the [[neocognitron]] in 1980.<ref name="intro">{{cite journal |last=Fukushima |first=Kunihiko |year=1980 |title=Neocognitron: A Self-organizing Neural Network Model for a Mechanism of Pattern Recognition Unaffected by Shift in Position |url=https://www.cs.princeton.edu/courses/archive/spr08/cos598B/Readings/Fukushima1980.pdf |url-status=live |journal=Biological Cybernetics |volume=36 |issue=4 |pages=193–202 |doi=10.1007/BF00344251 |pmid=7370364 |s2cid=206775608 |archive-url=https://web.archive.org/web/20140603013137/http://www.cs.princeton.edu/courses/archive/spr08/cos598B/Readings/Fukushima1980.pdf |archive-date=3 June 2014 |access-date=16 November 2013}}</ref> [[Max pooling]] appears in a 1982 publication on the neocognitron<ref>{{Cite journal |last1=Fukushima |first1=Kunihiko |last2=Miyake |first2=Sei |date=1982-01-01 |title=Neocognitron: A new algorithm for pattern recognition tolerant of deformations and shifts in position |url=https://www.sciencedirect.com/science/article/abs/pii/0031320382900243 |journal=Pattern Recognition |volume=15 |issue=6 |pages=455–469 |doi=10.1016/0031-3203(82)90024-3 |bibcode=1982PatRe..15..455F |issn=0031-3203|url-access=subscription }}</ref> and was in the 1989 publication in [[LeNet|LeNet-5]].<ref>{{Cite journal |last1=LeCun |first1=Yann |last2=Boser |first2=Bernhard |last3=Denker |first3=John |last4=Henderson |first4=Donnie |last5=Howard |first5=R. |last6=Hubbard |first6=Wayne |last7=Jackel |first7=Lawrence |date=1989 |title=Handwritten Digit Recognition with a Back-Propagation Network |url=https://proceedings.neurips.cc/paper/1989/hash/53c3bce66e43be4f209556518c2fcb54-Abstract.html |journal=Advances in Neural Information Processing Systems |publisher=Morgan-Kaufmann |volume=2}}</ref>

In 1990, Yamaguchi et al. used max pooling in TDNNs in order to realize a speaker independent isolated word recognition system.<ref name="Yamaguchi111990">{{cite conference |title=A Neural Network for Speaker-Independent Isolated Word Recognition |last1=Yamaguchi |first1=Kouichi |last2=Sakamoto |first2=Kenji |last3=Akabane |first3=Toshio |last4=Fujimoto |first4=Yoshiji |date=November 1990 |___location=Kobe, Japan |conference=First International Conference on Spoken Language Processing (ICSLP 90) |url=https://www.isca-speech.org/archive/icslp_1990/i90_1077.html |access-date=2019-09-04 |archive-date=2021-03-07 |archive-url=https://web.archive.org/web/20210307233750/https://www.isca-speech.org/archive/icslp_1990/i90_1077.html |url-status=dead }}</ref>