Regular conditional probability

Regular conditional probability is a concept that has developed to overcome certain difficulties in formally defining conditional probabilities for continuous probability distributions. It is defined as an alternative probability measure conditioned on a particular value of a random variable.

Motivation

Normally we define the conditional probability of an event A given an event B as:

${\displaystyle P(A|B)={\frac {P(A\cap B)}{P(B)}}.}$ 

The difficulty with this arises when the event B is too small to have a non-zero probability. For example, suppose we have a random variable X with a uniform distribution on ${\displaystyle [0,1],}$  and B is the event that ${\displaystyle X=2/3.}$  Clearly, the probability of B, in this case, is ${\displaystyle P(B)=0,}$  but nonetheless we would still like to assign meaning to a conditional probability such as ${\displaystyle P(A|X=2/3).}$  To do so rigorously requires the definition of a regular conditional probability.

Definition

Let ${\displaystyle (\Omega ,{\mathcal {F}},P)}$  be a probability space, and let ${\displaystyle T:\Omega \rightarrow E}$  be a random variable, defined as a Borel-measurable function from ${\displaystyle \Omega }$  to its state space ${\displaystyle (E,{\mathcal {E}}).}$ . One should think of ${\displaystyle T}$  as a way to "disintegrate" the sample space ${\displaystyle \Omega }$  into ${\displaystyle \{T^{-1}(x)\}_{x\in E}}$ . Using the disintegration theorem from the measure theory, it allows us to "disintegrate" the measure ${\displaystyle P}$  into a collection of measures, one for each ${\displaystyle x\in E}$ . Formally, a regular conditional probability is defined as a function ${\displaystyle \nu :E\times {\mathcal {F}}\rightarrow [0,1],}$  called a "transition probability", where:

• For every ${\displaystyle x\in E}$ , ${\displaystyle \nu (x,\cdot )}$  is a probability measure on ${\displaystyle {\mathcal {F}}}$ . Thus we provide one measure for each ${\displaystyle x\in E}$ .
• For all ${\displaystyle A\in {\mathcal {F}}}$ , ${\displaystyle \nu (\cdot ,A)}$  (a mapping ${\displaystyle E\mapsto [0,1]}$ ) is ${\displaystyle {\mathcal {E}}}$ -measurable, and
• For all ${\displaystyle A\in {\mathcal {F}}}$  and all ${\displaystyle B\in {\mathcal {E}}}$ ^[1]

${\displaystyle P{\big (}A\cap T^{-1}(B){\big )}=\int _{B}\nu (x,A)\,P{\big (}T^{-1}(dx){\big )}.}$ 

where ${\displaystyle P\circ T^{-1}}$  is the pushforward measure ${\displaystyle T_{*}P}$  of the distribution of the random element ${\displaystyle T}$ , ${\displaystyle x\in \mathrm {supp} \,T,}$  i.e. the topological support of the ${\displaystyle T_{*}P}$ . Specifically, if we take ${\displaystyle B=E}$ , then ${\displaystyle A\cap T^{-1}(E)=A}$ , and so

${\displaystyle P(A)=\int _{E}\nu (x,A)\,P{\big (}T^{-1}(dx){\big )}}$ ,

where ${\displaystyle \nu (x,A)}$  can be denoted, using more familiar terms ${\displaystyle P(A\ |\ T=x)}$  (this is "defined" to be conditional probability of ${\displaystyle A}$  given ${\displaystyle x}$ , which can be undefined in elementary constructions of conditional probability). As can be seen from the integral above, the value of ${\displaystyle \nu }$  for points x outside the support of the random variable is meaningless; its significance as a conditional probability is strictly limited to the support of T.

The measurable space ${\displaystyle (\Omega ,{\mathcal {F}})}$  is said to have the regular conditional probability property if for all probability measures ${\displaystyle P}$  on ${\displaystyle (\Omega ,{\mathcal {F}}),}$  all random variables on ${\displaystyle (\Omega ,{\mathcal {F}},P)}$  admit a regular conditional probability. A Radon space, in particular, has this property.

See also conditional probability and conditional probability distribution.

Example

To continue with our motivating example above, we consider a real-valued random variable X and write

${\displaystyle P(A|X=x_{0})=\nu (x_{0},A)=\lim _{\epsilon \rightarrow 0+}{\frac {P(A\cap \{x_{0}-\epsilon <X<x_{0}+\epsilon \})}{P(\{x_{0}-\epsilon <X<x_{0}+\epsilon \})}},}$ 

(where ${\displaystyle x_{0}=2/3}$  for the example given.) This limit, if it exists, is a regular conditional probability for X, restricted to ${\displaystyle \mathrm {supp} \,X.}$ 

In any case, it is easy to see that this limit fails to exist for ${\displaystyle x_{0}}$  outside the support of X: since the support of a random variable is defined as the set of all points in its state space whose every neighborhood has positive probability, for every point ${\displaystyle x_{0}}$  outside the support of X (by definition) there will be an ${\displaystyle \epsilon >0}$  such that ${\displaystyle P(\{x_{0}-\epsilon <X<x_{0}+\epsilon \})=0.}$ 

Thus if X is distributed uniformly on ${\displaystyle [0,1],}$  it is truly meaningless to condition a probability on "${\displaystyle X=3/2}$ ".

See also

• Conditioning (probability)
• Disintegration theorem
• Adherent point
• Limit point

References

1. D. Leao Jr. et al. Regular conditional probability, disintegration of probability and Radon spaces. Proyecciones. Vol. 23, No. 1, pp. 15–29, May 2004, Universidad Católica del Norte, Antofagasta, Chile PDF

External links

• Regular Conditional Probability on PlanetMath