Convolution of probability distributions: Difference between revisions

Revision as of 10:54, 21 February 2016

Latest revision as of 03:18, 1 July 2025

(37 intermediate revisions by 28 users not shown)

Line 1:

{{short description|Probability distribution of the sum of random variables}}

The '''convolution of probability distributions''' arises in [[probability theory]] and [[statistics]] as the operation in terms of [[probability distribution]]s that corresponds to the addition of [[statistically independent|independent]] [[random variable]]s and, by extension, to forming linear combinations of random variables. The operation here is a special case of [[convolution]] in the context of probability distributions.

The '''convolution/sum of probability distributions''' arises in [[probability theory]] and [[statistics]] as the operation in terms of [[probability distribution]]s that corresponds to the addition of [[statistically independent|independent]] [[random variable]]s and, by extension, to forming linear combinations of random variables. The operation here is a special case of [[convolution]] in the context of probability distributions.

==Introduction==

The [[probability distribution]] of the sum of two or more [[independent (probability)|independent]] [[random variable]]s is the convolution of their individual distributions. The term is motivated by the fact that the [[probability mass function]] or [[probability density function]] of a sum of independent random variables is the [[convolution]] of their corresponding probability mass functions or probability density functions respectively. Many well known distributions have simple convolutions: see [[List of convolutions of probability distributions]] .

The general formula for the distribution of the sum <math>Z=X+Y</math> of two independent integer-valued (and hence discrete) random variables is<ref>[[Susan P. Holmes|Susan Holmes ]](1998). Sums of Random Variables:

Statistics 116. Stanford. https://web.archive.org/web/20210413200454/http://statweb.stanford.edu/~susan/courses/s116/node114.html</ref>

:<math>P(Z=z) = \sum_{k=-\infty}^\infty P(X=k)P(Y=z-k)</math>

For independent, continuous random variables with [[probability density function]]s (PDF) <math>f,g</math> and [[cumulative distribution function]]s (CDF) <math>F,G</math> respectively, we have that the CDF of the sum is:

:<math>H(z)=\int_{-\infty}^\infty F(z-t)g(t) dt = \int_{-\infty}^\infty G(t)f(z-t) dt</math>

If we start with random variables <math>X</math> and <math>Y</math>, related by <math>Z = X + Y</math>, and with no information about their possible independence, then:

:<math>f_Z(z) = \int \limits_{-\infty}^{\infty} f_{XY}(x, z-x)~dx</math>

However, if <math>X</math> and <math>Y</math> are independent, then:

:<math>f_{XY}(x,y) = f_X(x) f_Y(y)</math>

and this formula becomes the convolution of probability distributions:

:<math>f_Z(z) = \int \limits_{-\infty}^{\infty} f_{X}(x)~f_Y(z-x)~dx</math>

== Example derivation ==

Line 9 ⟶ 30:

There are several ways of deriving formulae for the convolution of probability distributions. Often the manipulation of integrals can be avoided by use of some type of [[generating function]]. Such methods can also be useful in deriving properties of the resulting distribution, such as moments, even if an explicit formula for the distribution itself cannot be derived.

One of the straightforward techniques is to use [[Characteristic function (probability theory)|characteristic functions]], which always exists and are unique to a given distribution.{{cncitation needed|date=April 2013}}

=== Convolution of Bernoulli distributions ===

The convolution of two independent identically distributed [[Bernoulli distribution|Bernoulli random variables]] is a Binomialbinomial random variable. That is, in a shorthand notation,

:<math> \sum_{i=1}^2 \mathrm{Bernoulli}(p) \sim \mathrm{Binomial}(2,p).</math>

To show this let

:<math>X_i \sim \mathrm{Bernoulli}(p), \quad 0<p<1, \quad 1 \le i \le 2</math>

and define

:<math>Y=\sum_{i=1}^2 X_i.</math>

Also, let ''Z'' denote a generic binomial random variable:

:<math>Z \sim \mathrm{Binomial}(2,p) \,\! .</math>

====Using probability mass functions====

Line 31 ⟶ 52:

&=p^n\left(1-p\right)^{2-n}\sum_{m\in\mathbb{Z}}\binom{1}{m}\binom{1}{n-m} \\

&=p^n\left(1-p\right)^{2-n}\left[\binom{1}{0}\binom{1}{n}+\binom{1}{1}\binom{1}{n-1}\right]\\

&=\binom{2}{n}p^n\left(1-p\right)^{2-n}=\mathbb{P}[Z=n] .

\end{align}</math>

Here, usewe was made ofused the fact that <math>\tbinom{n}{k}=0</math> for ''k''>''n'' in the last but three equality, and of [[Pascal's rule]] in the second last equality.

==== Using characteristic functions ====

Line 53 ⟶ 74:

== References ==

{{Reflist}}

* {{cite book | last1=Hogg | first1=Robert V. |authorlink1=Robert V. Hogg | last2=McKean | first2=Joseph W. | last3=Craig | first3=Allen T. | title=Introduction to mathematical statistics | edition=6th | publisher=Prentice Hall | url=http://www.pearsonhighered.com/educator/product/Introduction-to-Mathematical-Statistics/9780130085078.page | ___location=Upper Saddle River, New Jersey | year=2004 | pages=692 | ISBN=978-0-13-008507-8|MR=467974|}}

[[Category:Theory of probability distributions]]