Markov kernel: Difference between revisions

Revision as of 21:41, 22 April 2019

Revision as of 20:37, 18 May 2019

Line 1:

In [[probability theory]], a '''Markov kernel''' (also known as a '''stochastic kernel''' or '''probability kernel''') is a map that plays the role, in the general theory of [[Markov process]]es, plays the role that the [[Stochastic matrix|transition matrix]] does in the theory of Markov processes with a [[finite set|finite]] [[state space]].<ref>{{Cite journal | last1 = Reiss | first1 = R. D. | title = A Course on Point Processes | doi = 10.1007/978-1-4613-9308-5 | series = Springer Series in Statistics | year = 1993 | isbn = 978-1-4613-9310-8 | pmid = | pmc = }}</ref>

== Formal definition ==

Let <math>(X,\mathcal A), and (Y,\mathcal B)</math> be [[measurable space]]s. A ''Markov kernel'' <math>\kappa: X \to Y</math> with source <math>(X,\mathcal A)</math> and target <math>(Y,\mathcal B)</math> is a map <math>\kappa : \mathcal B \times X \to [0,1]</math> with the following properties:

# For every (fixed) <math>B \in \mathcal B</math>, the map <math> x \mapsto \kappa(B, x)</math> is <math>\mathcal A</math>-measurable

# For every (fixed) <math> x \in X</math>, the map <math> B \mapsto \kappa(B, x)</math> is a [[probability measure]] on <math>(Y, \mathcal B)</math>

Line 10:

== Examples ==

===[[Simple random walk]] on the integers ===

Take <math>X=Y=\Z</math>, and <math> \mathcal A = \mathcal B = \mathcal P(\Z)</math> (the [[power set]] of <math>\Z</math>). Then a Markov kernel is fully determined by the probability it assigns to a singleton setsset <math>\{m\}</math> with <math>m \in Y = \Z</math> for each <math>n \in X = \Z</math>.:

:<math>\kappa(B|n )=\sum_{m \in B}\kappa(\{m\}|n), \qquad \forall n \in \mathbb{Z}, \, \forall B \in \mathcal B</math>.

Now the random walk <math>\kappa</math> that goes to the right with probability <math>p</math> and to the left with probability <math>1 - p</math> is defined by

Line 31:

===Markov kernel defined by a kernel function and a measure ===

IfLet <math>\nu</math> isbe a [[Measure (mathematics)#Definition|measure]] on <math>(Y, \mathcal B)</math>, and <math>k: Y \times X\to [0, \infty]</math> is a [[measurable function]] with respect to the [[Sigma-algebra#Product .CF.83-algebra|product <math>\sigma</math>-algebra]] <math>\mathcal A \otimes \mathcal B</math> such that

:<math> \int_Y k(y, x)\nu(\mathrm{d} y) = 1, \qquad \forall x \in X </math>,

then <math> \kappa(dy |x) = k(y, x)\nu(dy) </math> i.e. the mapping

:<math>\begin{cases} \kappa:\mathcal B \times X \to [0,1] \\ \kappa(B|x)=\int_{B}k(y, x)\nu(\mathrm{d} y) \end{cases}</math>

defines a Markov kernel.<ref>{{cite book|last1=Erhan|first1=Cinlar|title=Probability and Stochastics|date=2011|publisher=Springer|___location=New York|isbn=978-0-387-87858-4|pages=37–38}}</ref>. This example generalises the countable Markov process example where <math>\nu</math> was the [[counting measure]],. butMoreover it encompasses other important examples aresuch as the convolution kernels, likein particular the Markov kernelkernels defined by the heat equation e.g. The latter example includes the Gaussian kernel on <math>X = Y = \mathbb R</math> with <math>\nu(dx) = dx </math> standard Lebesgue measure and:

:<math>k_t(y, x) = \frac{1}{\sqrt{2\pi}t}e^{-(y - x)^2/(2t^2)}</math>.

=== Measurable functions. ===

Take <math>(X, \mathcal{A})</math> and <math>(B, \mathcal{B})</math> arbitrary measurable spaces, and let <math>f:X \to Y</math> be a measurable function. Now define <math>\kappa(dy|x) = \delta_{f(x)}(dy)</math> i.e.

:<math> \kappa(B|x) = \mathbf{1}_B(f(x)) = \mathbf{1}_{f^{-1}(B)}(x) = \begin{cases}1 & \text{if } f(x) \in B\\ 0 & \text{otherwise}\end{cases}</math> for all <math>B \in \mathcal{B}</math>.

Note that the indicator function <math>\mathbf{1}_{f^{-1}(B)}</math> is <math>\mathcal{A}</math>-measurable for all <math>B \in \mathcal{B}</math> iff <math>f</math> is measurable.

Line 56:

The composition is associative by [[Fubini's theorem#Tonelli's_theorem_for_non-negative_measurable_function|Tonelli's theorem]] and the identity function considered as a Markov kernel (i.e. the delta measure <math> \kappa_{1}(dx'|x) = \delta_x(dx')</math>) is the unit for this composition.

This composition defines the structure of a [[category (mathematics)|category]] on the measurable spaces with Markov kernels as morphisms first defined by Lawvere <ref>{{cite web|author = F. W. Lawvere|title = The Category of Probabilistic Mappings|date = 1962|url = https://ncatlab.org/nlab/files/lawvereprobability1962.pdf}}</ref>. The category has the empty set as initial object and the one point set <math>*</math> as the terminal object.

== Probability Space defined by Probability Distribution and a Markov Kernel==
A probability measure on a measurable space <math>(X, \mathcal A)</math> is the same thing as a morphism <math>* \to X</math>
in thisthe Markov category also denoted by <math>P</math>. By composition, a probability space <math>(X, \mathcal A, PP_X)</math> and a probability kernel <math>\kappa: (X, \mathcal A) \to (Y, \mathcal B) </math> defines a probability space <math>(Y, \mathcal B, P_Y = \kappa \circ PP_X)</math>. It is concretely defined by

:<math> P_Y(B) = \int_B \int_X \kappa(dy|x) P_X(dx) = \int_X \kappa(B|x)P_X(dx) = \mathbb{E}_{P_X}\kappa(B|\cdot) </math>

== Properties ==