Regular conditional probability: Difference between revisions

Revision as of 09:59, 4 April 2021

Latest revision as of 21:14, 3 November 2024

(10 intermediate revisions by 6 users not shown)

Line 5:

=== Conditional probability distribution ===

Consider two random variables <math>X, Y : \Omega \to \mathbb{R}</math>. The ''conditional probability distribution'' of ''Y'' given ''X'' is a two variable function <math>\kappa_{Y|\mid X}: \mathbb{R} \times \mathcal{B}(\mathbb{R}) \to [0,1]</math>

If the random variable ''X'' is discrete

:<math>\kappa_{Y|\mid X}(x, A) = P(Y \in A |\mid X = x) = \begin{cases}

\frac{P(Y \in A, X = x)}{P(X=x)} & \text{ if } P(X = x) > 0 \\[3pt]

\text{arbitrary value} & \text{ otherwise}.

\end{cases}</math>

If the random variables ''X'', ''Y'' are continuous with density <math>f_{X,Y}(x,y)</math>.

:<math>\kappa_{Y|\mid X}(x, A) = \begin{cases}

\frac{\int_A f_{X,Y}(x, y) \, \mathrm{d}y}{\int_\mathbb{R} f_{X,Y}(x, y) \mathrm{d}y} &

\text{ if } \int_\mathbb{R} f_{X,Y}(x, y) \, \mathrm{d}y > 0 \\[3pt]

\text{arbitrary value} & \text{ otherwise}.

\end{cases}</math>

A more general definition can be given in terms of [[conditional expectation]]. Consider a function <math> e_{Y \in A} : \mathbb{R} \to [0,1]</math> satisfying

:<math>e_{Y \in A}(X(\omega)) = \mathbb{operatorname E}[1_{Y \in A} |\mid X](\omega)</math>

for almost all <math>\omega</math>.

Then the conditional probability distribution is given by

:<math>\kappa_{Y|\mid X}(x, A) = e_{Y \in A}(x).</math>

As with conditional expectation, this can be further generalized to conditioning on a sigma algebra <math>\mathcal{F}</math>. In that case the conditional distribution is a function <math>\Omega \times \mathcal{B}(\mathbb{R}) \to [0, 1]</math>:

:<math> \kappa_{Y|\mid\mathcal{F}}(\omega, A) = \mathbb{operatorname E}[1_{Y \in A} |\mid \mathcal{F}](\omega)</math>

=== Regularity ===

For working with <math>\kappa_{Y|\mid X}</math>, it is important that it be ''regular'', that is:

# For almost all ''x'', <math>A \mapsto \kappa_{Y|\mid X}(x, A)</math> is a probability measure

# For all ''A'', <math>x \mapsto \kappa_{Y|\mid X}(x, A)</math> is a measurable function

In other words <math>\kappa_{Y|\mid X}</math> is a [[Markov kernel]].

The firstsecond condition holds trivially, but the proof of the secondfirst is more involved. It can be shown that if ''Y'' is a random element <math>\Omega \to S</math> in a [[Radon space]] ''S'', there exists a <math>\kappa_{Y|\mid X}</math> that satisfies the measurabilityfirst condition.<ref>{{cite book |last1=Klenke |first1=Achim |title=Probability theory : a comprehensive course |date=30 August 2013 |___location=London |isbn=978-1-4471-5361-0 |edition=Second}}</ref> It is possible to construct more general spaces where a regular conditional probability distribution does not exist.<ref>Faden, A.M., 1985. The existence of regular conditional probabilities: necessary and sufficient conditions. ''The Annals of Probability'', 13(1), pp.288-298&nbsp;288–298.</ref>

=== Relation to conditional expectation ===

For discrete and continuous random variables, the [[conditional expectation]] can be expressed as

In probability theory, the theory of [[conditional expectation]] is developed before that of regular conditional distributions.<ref>{{cite book |last1=Durrett |first1=Richard |title=Probability : theory and examples |date=2010 |publisher=Cambridge University Press |___location=Cambridge |isbn=9780521765398 |edition=4th}}</ref>

For discrete and continuous random variables, the conditional expectation can be expressed as

:<math>

\begin{aligned}

\mathbb{operatorname E}[Y|\mid X=x] &= \sum_y y \, P(Y=y|\mid X=x) \\

\mathbb{operatorname E}[Y|\mid X=x] &= \int y \, f_{Y|\mid X}(x, y) \, \mathrm{d}y

\end{aligned}

</math>

where <math>f_{Y|\mid X}(x, y)</math> is the [[conditional density]] of {{mvar|Y}} given {{mvar|X}}.

This result can be extended to measure theoretical conditional expectation using the regular conditional probability distribution:

:<math>\mathbb{operatorname E}[Y|\mid X](\omega) = \int y \, \kappa_{Y|\mid\sigma(X)}(\omega, \mathrm{d}y).</math> .

== Formal definition==

Line 63 ⟶ 60:

* For all <math>A\in\mathcal F</math>, <math>\nu(\cdot, A)</math> (a mapping <math>E \to [0,1]</math>) is <math>\mathcal E</math>-measurable, and

* For all <math>A\in\mathcal F</math> and all <math>B\in\mathcal E</math><ref>D. Leao Jr. et al. ''Regular conditional probability, disintegration of probability and Radon spaces.'' Proyecciones. Vol. 23, No. 1, pp. 15–29, May 2004, Universidad Católica del Norte, Antofagasta, Chile [http://www.scielo.cl/pdf/proy/v23n1/art02.pdf PDF]</ref>

:: <math>P\big(A\cap T^{-1}(B)\big) = \int_B \nu(x,A) \,(P\big(circ T^{-1})(\mathrm{d }x)\big).</math>

where <math>P\circ T^{-1}</math> is the [[pushforward measure]] <math>T_*P</math> of the distribution of the random element <math>T</math>,

<math>x\in\mathrmoperatorname{supp}\,T,</math> i.e. the [[Support (measure theory)|support]] of the <math>T_* P</math>.

Specifically, if we take <math>B=E</math>, then <math>A \cap T^{-1}(E) = A</math>, and so

:<math>P(A) = \int_E \nu(x,A) \, (P\big(circ T^{-1})(\mathrm{d }x)\big),</math>,

where <math>\nu(x, A)</math> can be denoted, using more familiar terms <math>P(A\ |\ T=x)</math>.

==Alternate definition==

{{disputeabout|'''this way leads to irregular conditional probability'''|Non-regular conditional probability|date=September 2009}}

Consider a [[Radon space]] <math> \Omega </math> (that is a probability measure defined on a Radon space endowed with the Borel sigma-algebra) and a real-valued random variable ''T''. As discussed above, in this case there exists a regular conditional probability with respect to ''T''. Moreover, we can alternatively define the '''regular conditional probability''' for an event ''A'' given a particular value ''t'' of the random variable ''T'' in the following manner:

:<math> P (A|\mid T=t) = \lim_{U\supset \{T= t\}} \frac {P(A\cap U)}{P(U)},</math>

where the [[Limit (mathematics)|limit]] is taken over the [[Net (mathematics)|net]] of [[Open set|open]] [[Neighbourhood (mathematics)|neighborhoods]] ''U'' of ''t'' as they become [[Subset|smaller with respect to set inclusion]]. This limit is defined if and only if the probability space is [[Radon space|Radon]], and only in the support of ''T'', as described in the article. This is the restriction of the transition probability to the support of &nbsp;''T''. To describe this limiting process rigorously:

For every <math>\epsilonvarepsilon > 0,</math> there exists an open neighborhood ''U'' of the event {''T''&nbsp;=&nbsp;''t''}, such that for every open ''V'' with <math>\{T=t\} \subset V \subset U,</math>

:<math>\left|\frac {P(A\cap V)}{P(V)}-L\right| < \epsilonvarepsilon,</math>

where <math>L = P (A|\mid T=t)</math> is the limit.

==See also==