Concentration parameter: Difference between revisions

Revision as of 20:35, 14 December 2020

Latest revision as of 13:25, 28 December 2023

(3 intermediate revisions by 3 users not shown)

Line 1:

{{for|the inverse of a covariance matrix|Concentration matrix}}

{{Refimprove|date=September 2010}}

In [[probability theory]] and [[statistics]], a '''concentration parameter''' is a special kind of [[numerical parameter]] of a [[parametric family]] of [[probability distribution]]s. Concentration parameters occur in two kinds of distribution: In the [[Von Mises–Fisher distribution]], and in conjunction with distributions whose ___domain is a probability distribution, such as the [[symmetric Dirichlet distribution]] and the [[Dirichlet process]]. The rest of this article focuses on the latter usage.

Line 5 ⟶ 7:

==Dirichlet distribution==

In the case of multivariate Dirichlet distributions, there is some confusion over how to define the concentration parameter. In the topic modelling literature, it is often defined as the sum of the individual Dirichlet parameters,<ref>{{Cite conference|last=Wallach|first=Hanna M.|author-link=Hanna Wallach|author2=Iain Murray|author3=Ruslan Salakhutdinov|author4=David Mimno|date=2009|title=Evaluation methods for topic models|url=|conference=|series=ICML '09|___location=New York, NY, USA|publisher=ACM|volume=|pages=1105–1112|doi=10.1145/1553374.1553515|isbn=978-1-60558-516-1|via=|booktitlebook-title=Proceedings of the 26th Annual International Conference on Machine Learning|citeseerx=10.1.1.149.771}}</ref> when discussing symmetric Dirichlet distributions (where the parameters are the same for all dimensions) it is often defined to be the value of the single Dirichlet parameter used in all dimensions{{Citation needed|date=November 2011}}. This second definition is smaller by a factor of the dimension of the distribution.

A concentration parameter of 1 (or ''k'', the dimension of the Dirichlet distribution, by the definition used in the topic modelling literature) results in all sets of probabilities being equally likely, i.e., in this case the Dirichlet distribution of dimension ''k'' is equivalent to a uniform distribution over a [[Standard simplex|''k-1''-dimensional simplex]]. Note that thisThis is ''not'' the same as what happens when the concentration parameter tends towards infinity. In the former case, all resulting distributions are equally likely (the distribution over distributions is uniform). In the latter case, only near-uniform distributions are likely (the distribution over distributions is highly peaked around the uniform distribution). Meanwhile, in the limit as the concentration parameter tends towards zero, only distributions with nearly all mass concentrated on one of their components are likely (the distribution over distributions is highly peaked around the ''k'' possible [[Dirac delta distribution]]s centered on one of the components, or in terms of the ''k''-dimensional simplex, is highly peaked at corners of the simplex).

==Sparse prior==