Logarithmically concave function: Difference between revisions

Revision as of 21:11, 7 July 2012

Latest revision as of 19:51, 17 July 2025

(40 intermediate revisions by 28 users not shown)

Line 1:

{{Short description|Type of mathematical function}}

In [[convex analysis]], a [[non-negative]] function {{math|''f'' : '''R'''^''n'' → '''R'''₊}} is '''logarithmically concave''' (or '''log-concave''' for short) if its [[___domain of a function|___domain]] is a [[convex set]], and if it satisfies the inequality

: <math>

Line 5 ⟶ 6:

for all {{math|''x'',''y'' ∈ dom ''f''}} and {{math|0&nbsp;<&nbsp;''&theta;''&nbsp;<&nbsp;1}}. If {{math|''f''}} is strictly positive, this is equivalent to saying that the [[logarithm]] of the function, {{math|log ∘ ''f''}}, is [[concave function|concave]]; that is,

: <math>

\log f(\theta x + (1 - \theta) y) \geq \theta \log f(x) + (1-\theta) \log f(y)

</math>

for all {{math|''x'',''y'' ∈ dom ''f''}} and {{math|0&nbsp;<&nbsp;''&theta;''&nbsp;<&nbsp;1}}.

Line 11 ⟶ 12:

Examples of log-concave functions are the 0-1 [[indicator function]]s of convex sets (which requires the more flexible definition), and the [[Gaussian function]].

Similarly, a function is '''[[log-convex]]''' if it satisfies the reverse inequality

: <math>

f(\theta x + (1 - \theta) y) \leq f(x)^{\theta} f(y)^{1 - \theta}

Line 18 ⟶ 19:

==Properties==

* A positive log-concave function is also [[Quasi-concave_functionconcave function| quasi-concave]]. This follows from the fact that the logarithm is monotone implying that the [[Level set#Sublevel and superlevel sets|superlevel sets]] of this function are convex.<ref name=":0" />

* Every concave function that is nonnegative on its ___domain is log-concave. However, the reverse does not necessarily hold. An example is the [[Gaussian function]] {{math|''f''(''x'')}}&nbsp;=&nbsp;{{math|exp(&minus;''x''²/2)}} which is log-concave since {{math|log ''f''(''x'')}}&nbsp;=&nbsp;{{math|&minus;''x''²/2}} is a concave function of {{math|''x''}}. But {{math|''f''}} is not concave since the [[second derivative]] is positive for |{{math|''x''}}|&nbsp;>&nbsp;1:

* Every concave function that is nonnegative on its ___domain is log-concave. However, the reverse does not necessarily hold. An example is the [[Gaussian function]] {{math|''f''(''x'')}}&nbsp;=&nbsp;{{math|exp(&minus;x²/2)}} which is log-concave since {{math|log ''f''(''x'')}}&nbsp;=&nbsp;{{math|&minus;''x''²/2}} is a concave function of {{math|''x''}}. But {{math|''f''}} is not concave since the second derivative is positive for |{{math|''x''}}|&nbsp;>&nbsp;1:

::<math>f''(x)=e^{-\frac{x^2}{2}} (x^2-1) \nleq 0</math>

* From above two points, [[Concave function|concavity]] <math>\Rightarrow</math> log-concavity <math>\Rightarrow</math> [[Quasiconcave function|quasiconcavity]].

* A twice differentiable, nonnegative function with a convex ___domain is log-concave [[if and only if]] for all {{math|''x''}} satisfying {{math|''f''(''x'')&nbsp;>&nbsp;0}},

::<math>f(x)\nabla^2f(x) \preceq \nabla f(x)\nabla f(x)^T</math>,<ref name=":0">{{cite book |first1=Stephen |last1=Boyd |author-link=Stephen P. Boyd |first2=Lieven |last2=Vandenberghe |chapter=Log-concave and log-convex functions |title=Convex Optimization |publisher=Cambridge University Press |year=2004 |isbn=0-521-83378-7 |chapter-url=https://web.stanford.edu/~boyd/cvxbook/ |pages=104–108 }}</ref>

* A twice differentiable, nonnegative function with a convex ___domain is log-concave if and only if for all {{math|''x''}} satisfying {{math|''f''(''x'')&nbsp;>&nbsp;0}},

::<math>f(x)\nabla^2f(x) \preceq \nabla f(x)\nabla f(x)^T</math>, <ref> Stephen Boyd and Lieven Vandenberghe, [http://www.stanford.edu/~boyd/cvxbook/ Convex Optimization] (PDF) p.105</ref>

:i.e.

::<math>f(x)\nabla^2f(x) - \nabla f(x)\nabla f(x)^T</math> is

:[[positive-definite matrix|negative semi-definite]]. For functions of one variable, this condition simplifies to

Line 40:

* Products: The product of log-concave functions is also log-concave. Indeed, if {{math|''f''}} and {{math|''g''}} are log-concave functions, then {{math|log&nbsp;''f''}} and {{math|log&nbsp;''g''}} are concave by definition. Therefore

::<math>\log\,f(x) + \log\,g(x) = \log(f(x)g(x))</math>

:is concave, and hence also {{math|''f''&nbsp;''g''}} is log-concave.

Line 57:

==Log-concave distributions==

Log-concave distributions are necessary for a number of algorithms, e.g. [[adaptive rejection sampling]]. Every distribution with log-concave density is a [[maximum entropy probability distribution]] with specified mean ''μ'' and [[Deviation risk measure]] ''D''.<ref name="Grechuk1">{{cite journal

| last1=Grechuk | first1=Bogdan

| last2=Molyboha | first2=Anton

As it happens, many common [[probability distribution]]s are log-concave. Some examples:<ref>See Mark Bagnoli and Ted Bergstrom (1989), "Log-Concave Probability and Its Applications", University of Michigan.[http://www.econ.ucsb.edu/~tedb/Theory/delta.pdf]</ref>

| last3=Zabarankin | first3=Michael

*The [[normal distribution]] and [[multivariate normal distribution]]s.

| date=May 2009

*The [[exponential distribution]].

| title=Maximum Entropy Principle with General Deviation Measures

*The [[uniform distribution (continuous)|uniform distribution]] over any [[convex set]].

| journal=[[Mathematics of Operations Research]]

*The [[logistic distribution]].

| volume=18734

*The [[extreme value distribution]].

| issue=2

*The [[Laplace distribution]].

| pages=445–467

*The [[chi distribution]].

| doi=10.1287/moor.1090.0377

*The [[Wishart distribution]], where ''n'' >= ''p'' + 1.<ref name="prekopa">András Prékopa (1971), "Logarithmic concave measures with application to stochastic programming". ''Acta Scientiarum Mathematicarum'', 32, pp. 301–316.</ref>

| url=https://www.researchgate.net/profile/Bogdan-Grechuk/publication/220442393_Maximum_Entropy_Principle_with_General_Deviation_Measures/links/59132b61a6fdcc963e7ed4fd/Maximum-Entropy-Principle-with-General-Deviation-Measures.pdf}}</ref>

*The [[Dirichlet distribution]], where all parameters are >= 1.<ref name="prekopa"/>

As it happens, many common [[probability distribution]]s are log-concave. Some examples:<ref name=":1">See {{cite journal |first1=Mark |last1=Bagnoli and |first2=Ted |last2=Bergstrom (1989),|year=2005 "|title=Log-Concave Probability and Its Applications", University|journal=Economic ofTheory Michigan|volume=26 |issue=2 |pages=445–469 |doi=10.[1007/s00199-004-0514-4 |s2cid=1046688 |url=http://www.econ.ucsb.edu/~tedb/Theory/delta.pdf] }}</ref>

*The [[gamma distribution]] if the shape parameter is >= 1.

*Thethe [[chi-squarenormal distribution]] ifand the[[multivariate numbernormal of degrees of freedom is >= 2.distribution]]s,

*Thethe [[betaexponential distribution]] if both shape parameters are >= 1.,

*Thethe [[Weibulluniform distribution (continuous)|uniform distribution]] ifover theany shape[[convex parameter is >= 1.set]],

*Thethe [[exponentialbinomial distribution]].,

*Thethe [[logistic distribution]].,

*Thethe [[extreme value distribution]].,

*Thethe [[Laplace distribution]].,

*Thethe [[chi distribution]].,

*the [[hyperbolic secant distribution]],

*Thethe [[Wishart distribution]], whereif ''n'' >=≥ ''p'' + 1.,<ref name="prekopa">{{cite journal | last1 = Prékopa | first1 = András | author-link = András Prékopa (| year = 1971), "| title = Logarithmic concave measures with application to stochastic programming". ''| journal = [[Acta Scientiarum Mathematicarum'',]] | volume = 32, pp.| issue = 3-4 | pages = 301–316 | url = http://rutcor.rutgers.edu/~prekopa/SCIENT1.pdf}}</ref>

*Thethe [[Dirichlet distribution]], whereif all parameters are >=≥ 1.,<ref name="prekopa"/>

*Thethe [[gamma distribution]] if the [[shape parameter]] is >=≥ 1.,

*the [[chi-square distribution]] if the number of degrees of freedom is ≥ 2,

*the [[beta distribution]] if both shape parameters are ≥ 1, and

*the [[Weibull distribution]] if the shape parameter is ≥ 1.

Note that all of the parameter restrictions have the same basic source: The exponent of non-negative quantity must be non-negative in order for the function to be log-concave.

The following distributions are non-log-concave for all parameters:

*Thethe [[Student's t-distribution]].,

*Thethe [[Cauchy distribution]].,

*Thethe [[Pareto distribution]].,

*Thethe [[log-normal distribution]]., and

*Thethe [[F-distribution]].

Note that the [[cumulative distribution function]] (CDF) of all log-concave distributions is also log-concave. However, some non-log-concave distributions also have log-concave CDF's:

*Thethe [[log-normal distribution]].,

*Thethe [[Pareto distribution]].,

*Thethe [[Weibull distribution]] when the shape parameter < 1., and

*Thethe [[gamma distribution]] when the shape parameter < 1.

The following are among the properties of log-concave distributions:

*If a density is log-concave, so is its [[cumulative distribution function]] (CDF).

*If a multivariate density is log-concave, so is the [[marginal density]] over any subset of variables.

*The sum of two independent log-concave [[random variable]]s is log-concave. This follows from the fact that the convolution of two log-concave functions is log-concave.

*The product of two log-concave functions is log-concave. This means that [[joint distribution|joint]] densities formed by multiplying two probability densities (e.g. the [[normal-gamma distribution]], which always has a shape parameter >=≥ 1) will be log-concave. This property is heavily used in general-purpose [[Gibbs sampling]] programs such as [[Bayesian inference using Gibbs sampling|BUGS]] and [[Just another Gibbs sampler|JAGS]], which are thereby able to use [[adaptive rejection sampling]] over a wide variety of [[conditional distribution]]s derived from the product of other distributions.

* If a density is log-concave, so is its [[survival function]].<ref name=":1" />

* If a density is log-concave, it has a monotone [[hazard rate]] (MHR), and is a [[Regular distribution (economics)|regular distribution]] since the derivative of the logarithm of the survival function is the negative hazard rate, and by concavity is monotone i.e.

::<math>\frac{d}{dx}\log\left(1-F(x)\right) = -\frac{f(x)}{1-F(x)}</math> which is decreasing as it is the derivative of a concave function.

==See also==

*[[logarithmically concave measuresequence]]

*[[logarithmically convexconcave functionmeasure]]

*[[logarithmically convex function]]

*[[convex function]]

==Notes==

Line 99 ⟶ 120:

==References==

* {{cite book|authorlinkauthor-link=Ole Barndorff-Nielsen|last=Barndorff-Nielsen|first=Ole|title=Information and exponential families in statistical theory|series=Wiley Series in Probability and Mathematical Statistics|publisher=John Wiley \& Sons, Ltd.|___location=Chichester|year=1978|pages=ix+238 pp.|isbn=0-471-99545-2|mr=489333}}

* {{cite book|authorlink=Ole Barndorff-Nielsen|last=Barndorff-Nielsen|first=Ole|title=Information and exponential families in statistical theory|series=Wiley Series in Probability and Mathematical Statistics|publisher=John Wiley \& Sons, Ltd.|___location=Chichester|year=1978|pages=ix+238 pp.|isbn=0-471-99545-2|mr=489333}}

* {{cite book|title=Unimodality, convexity, and applications

|last1=Dharmadhikari|first1=Sudhakar

|last2=Joag-Dev

|first2=Kumar|series=Probability and Mathematical Statistics

|series=Probability and Mathematical Statistics

|publisher=Academic Press, Inc.

|___location=Boston, MA

|year=1988

|pages=xiv+278

|isbn=0-12-214690-5|mr=954608}}

* {{cite book|title=Parametric Statistical Theory | last1=Pfanzagl | first1=Johann

|authorlinkauthor-link= <!-- Johann Pfanzagl -->

|last2=with the assistance of R. Hamböker

|year=1994|publisher=Walter de Gruyter

|publisher=Walter de Gruyter

|isbn=3-11-013863-8

|mr=1291393}}

* {{cite book|title=Convex functions, partial orderings, and statistical applications|last1=Pečarić|first1=Josip E.|last2=Proschan|first2=Frank|last3=Tong|first3=Y. L.|<!-- authorlink2|author-link2=Frank Proschan -->

|series=Mathematics in Science and Engineering|volume=187

|volume=187

|publisher=Academic Press, Inc.

|___location=Boston, MA

|year=1992|pages=xiv+467 pp.

|isbn=0-12-549250-2

|mr=1162312}}

==See also==

*[[logarithmically convex function]]

*[[convex function]]

*[[logarithmically concave measure]]

{{DEFAULTSORT:Logarithmically Concave Function}}

[[Category:Mathematical analysis]]

[[Category:Convex analysis]]

[[eo:Logaritme konkava funkcio]]