Probabilistic latent semantic analysis: Difference between revisions

Revision as of 00:58, 1 January 2015

Revision as of 09:25, 2 January 2015

Line 10:

: <math>P(w,d) = \sum_c P(c) P(d|c) P(w|c) = P(d) \sum_c P(c|d) P(w|c)</math>

The first formulation is the ''symmetric'' formulation, where <math>w</math> and <math>d</math> are both generated from the latent class <math>c</math> in similar ways (using the conditional probabilities <math>P(d|c)</math> and <math>P(w|c)</math>), whereas the second formulation is the ''asymmetric'' formulation, where, for each document <math>d</math>, a latent class is chosen conditionally to the document according to <math>P(c|d)</math>, and a word is then generated from that class according to <math>P(w|c)</math>. Although we have used words and documents in this example, the co-occurrence of any couple of discrete variables may be modelled in exactly the same way.

So, the number of parameters is equal to <math>cd + wc</math>. The number of parameters grows linearly with the number of documents. In addition, although PLSA is a generative model of the documents in the collection it is estimated on, it is not a generative model of new documents.

Line 19:

PLSA may be used in a discriminative setting, via [[Fisher kernel]]s.<ref>Thomas Hofmann, [http://www.cs.brown.edu/people/th/papers/Hofmann-NIPS99.ps ''Learning the Similarity of Documents : an information-geometric approach to document retrieval and categorization''], [[Advances in Neural Information Processing Systems]] 12, pp-914-920, [[MIT Press]], 2000</ref>

PLSA has applications in [[information retrieval]] and [[information filtering|filtering]], [[natural language processing]], [[machine learning]] from text, and related areas.

It is reported that the [[aspect model]] used in the probabilistic latent semantic analysis has severe [[overfitting]] problems.<ref>{{cite journal|title=Latent Dirichlet Allocation|journal=Journal of Machine Learning Research|year=2003|first=David M.|last=Blei|author2=Andrew Y. Ng |author3=Michael I. Jordan |volume=3|pages=993–1022|id= |url=http://jmlr.csail.mit.edu/papers/volume3/blei03a/blei03a.pdf|doi=10.1162/jmlr.2003.3.4-5.993}}</ref>

Line 37:

==History==

This is an example of a [[latent class model]] (see references therein), and it is related <ref>Chris Ding, Tao Li, Wei Peng (2006). "[http://www.aaai.org/Papers/AAAI/2006/AAAI06-055.pdf Nonnegative Matrix Factorization and Probabilistic Latent Semantic Indexing: Equivalence Chi-Square Statistic, and a Hybrid Method. AAAI 2006 ]</ref> to [[non-negative matrix factorization]]. The present terminology was coined in 1999 by [[Thomas Hofmann]] .<ref>Thomas Hofmann, [http://www.cs.brown.edu/~th/papers/Hofmann-SIGIR99.pdf ''Probabilistic Latent Semantic Indexing''], Proceedings of the Twenty-Second Annual International [[Special Interest Group on Information Retrieval|SIGIR]] Conference on Research and Development in [[Information Retrieval]] (SIGIR-99), 1999</ref>.

== References and notes ==