Content deleted Content added
Clarified that two functions are learned by the autoencoder. Added one citation and removed a spurious citation in history section. |
Kramer (1992) was the first to discuss anomaly detection, both to detect sensor failures and replace the failed or missing values. |
||
Line 3:
{{Use dmy dates|date=March 2020|cs1-dates=y}}
{{Machine learning|Artificial neural network}}
An '''autoencoder''' is a type of [[artificial neural network]] used to learn [[Feature learning|efficient codings]] of unlabeled data ([[unsupervised learning]]).<ref name=":12">{{cite journal|doi=10.1002/aic.690370209|title=Nonlinear principal component analysis using autoassociative neural networks|journal=AIChE Journal|volume=37|issue=2|pages=233–243|date=1991|last1=Kramer|first1=Mark A.|url= https://www.researchgate.net/profile/Abir_Alobaid/post/To_learn_a_probability_density_function_by_using_neural_network_can_we_first_estimate_density_using_nonparametric_methods_then_train_the_network/attachment/59d6450279197b80779a031e/AS:451263696510979@1484601057779/download/NL+PCA+by+using+ANN.pdf}}</ref><ref name=":13">{{Cite journal |last=Kramer |first=M. A. |date=1992-04-01 |title=Autoassociative neural networks |url=https://www.sciencedirect.com/science/article/pii/009813549280051A |journal=Computers & Chemical Engineering |series=Neutral network applications in chemical engineering |language=en |volume=16 |issue=4 |pages=313–328 |doi=10.1016/0098-1354(92)80051-A |issn=0098-1354}}</ref> An autoencoder learns two functions: an encoding function that transforms the input data, and a decoding function that recreates the input data from the encoded representation. The autoencoder learns an [[Feature learning|efficient representation]] (encoding) for a set of data, typically for [[dimensionality reduction]].
Variants exist, aiming to force the learned representations to assume useful properties.<ref name=":0" /> Examples are regularized autoencoders (''Sparse'', ''Denoising'' and ''Contractive''), which are effective in learning representations for subsequent [[Statistical classification|classification]] tasks,<ref name=":4" /> and ''Variational'' autoencoders, with applications as [[generative model]]s.<ref name=":11">{{cite journal |arxiv=1906.02691|doi=10.1561/2200000056|bibcode=2019arXiv190602691K|title=An Introduction to Variational Autoencoders|date=2019|last1=Welling|first1=Max|last2=Kingma|first2=Diederik P.|journal=Foundations and Trends in Machine Learning|volume=12|issue=4|pages=307–392|s2cid=174802445}}</ref> Autoencoders are applied to many problems, including [[face recognition|facial recognition]],<ref>Hinton GE, Krizhevsky A, Wang SD. [http://www.cs.toronto.edu/~fritz/absps/transauto6.pdf Transforming auto-encoders.] In International Conference on Artificial Neural Networks 2011 Jun 14 (pp. 44-51). Springer, Berlin, Heidelberg.</ref> feature detection,<ref name=":2">{{Cite book|last=Géron|first=Aurélien|title=Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow|publisher=O’Reilly Media, Inc.|year=2019|___location=Canada|pages=739–740}}</ref> anomaly detection and acquiring the meaning of words.<ref>{{cite journal|doi=10.1016/j.neucom.2008.04.030|title=Modeling word perception using the Elman network|journal=Neurocomputing|volume=71|issue=16–18|pages=3150|date=2008|last1=Liou|first1=Cheng-Yuan|last2=Huang|first2=Jau-Chi|last3=Yang|first3=Wen-Chie|url=http://ntur.lib.ntu.edu.tw//handle/246246/155195 }}</ref><ref>{{cite journal|doi=10.1016/j.neucom.2013.09.055|title=Autoencoder for words|journal=Neurocomputing|volume=139|pages=84–96|date=2014|last1=Liou|first1=Cheng-Yuan|last2=Cheng|first2=Wei-Chen|last3=Liou|first3=Jiun-Wei|last4=Liou|first4=Daw-Ran}}</ref> Autoencoders are also generative models which can randomly generate new data that is similar to the input data (training data).<ref name=":2" />
Line 145:
=== Anomaly detection ===
Another application for autoencoders is [[anomaly detection]].<ref name=":13" /><ref>{{Cite journal |last1=Morales-Forero |first1=A. |last2=Bassetto |first2=S. |date=December 2019 |title=Case Study: A Semi-Supervised Methodology for Anomaly Detection and Diagnosis |url=https://ieeexplore.ieee.org/document/8978509 |journal=2019 IEEE International Conference on Industrial Engineering and Engineering Management (IEEM) |___location=Macao, Macao |publisher=IEEE |pages=1031–1037 |doi=10.1109/IEEM44572.2019.8978509 |isbn=978-1-7281-3804-6|s2cid=211027131 }}</ref><ref>{{Cite journal |last1=Sakurada |first1=Mayu |last2=Yairi |first2=Takehisa |date=December 2014 |title=Anomaly Detection Using Autoencoders with Nonlinear Dimensionality Reduction |url=http://dl.acm.org/citation.cfm?doid=2689746.2689747 |journal=Proceedings of the MLSDA 2014 2nd Workshop on Machine Learning for Sensory Data Analysis - MLSDA'14 |language=en |___location=Gold Coast, Australia QLD, Australia |publisher=ACM Press |pages=4–11 |doi=10.1145/2689746.2689747 |isbn=978-1-4503-3159-3|s2cid=14613395 }}</ref><ref name=":8">An, J., & Cho, S. (2015). [http://dm.snu.ac.kr/static/docs/TR/SNUDM-TR-2015-03.pdf Variational Autoencoder based Anomaly Detection using Reconstruction Probability]. ''Special Lecture on IE'', ''2'', 1-18.</ref><ref>{{Cite journal |last1=Zhou |first1=Chong |last2=Paffenroth |first2=Randy C. |date=2017-08-04 |title=Anomaly Detection with Robust Deep Autoencoders |url=https://dl.acm.org/doi/10.1145/3097983.3098052 |journal=Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining |language=en |___location=Halifax NS Canada |publisher=ACM |pages=665–674 |doi=10.1145/3097983.3098052 |isbn=978-1-4503-4887-4|s2cid=207557733 }}</ref><ref>{{Cite journal|doi=10.1016/j.patrec.2017.07.016|title=A study of deep convolutional auto-encoders for anomaly detection in videos|year=2018|last1=Ribeiro|first1=Manassés|last2=Lazzaretti|first2=André Eugênio|last3=Lopes|first3=Heitor Silvério|journal=Pattern Recognition Letters|volume=105|pages=13–22|bibcode=2018PaReL.105...13R}}</ref> By learning to replicate the most salient features in the training data under some of the constraints described previously, the model is encouraged to learn to precisely reproduce the most frequently observed characteristics. When facing anomalies, the model should worsen its reconstruction performance. In most cases, only data with normal instances are used to train the autoencoder; in others, the frequency of anomalies is small compared to the observation set so that its contribution to the learned representation could be ignored. After training, the autoencoder will accurately reconstruct "normal" data, while failing to do so with unfamiliar anomalous data.<ref name=":8" /> Reconstruction error (the error between the original data and its low dimensional reconstruction) is used as an anomaly score to detect anomalies.<ref name=":8" />
Recent literature has however shown that certain autoencoding models can, counterintuitively, be very good at reconstructing anomalous examples and consequently not able to reliably perform anomaly detection.<ref>{{cite arXiv|last1=Nalisnick|first1=Eric|last2=Matsukawa|first2=Akihiro|last3=Teh|first3=Yee Whye|last4=Gorur|first4=Dilan|last5=Lakshminarayanan|first5=Balaji|date=2019-02-24|title=Do Deep Generative Models Know What They Don't Know?|class=stat.ML|eprint=1810.09136}}</ref><ref>{{Cite journal|last1=Xiao|first1=Zhisheng|last2=Yan|first2=Qing|last3=Amit|first3=Yali|date=2020|title=Likelihood Regret: An Out-of-Distribution Detection Score For Variational Auto-encoder|url=https://proceedings.neurips.cc/paper/2020/hash/eddea82ad2755b24c4e168c5fc2ebd40-Abstract.html|journal=Advances in Neural Information Processing Systems|language=en|volume=33|arxiv=2003.02977}}</ref>
| |||