Uniformization (probability theory): Difference between revisions

Revision as of 17:40, 13 December 2010

Revision as of 22:44, 18 April 2011

Line 1:

In [[probability theory]], '''uniformization''' method, (also known as ''Jensen's method''<ref name="stewart" /> or the ''randomization method of randomization''<ref name="ibe">{{cite book |title=Markov processes for stochastic modeling |last=Ibe |first=Oliver C. |year=2009 |publisher=[[Academic Press]] |isbn=0123744512 |page=98}}</ref>) is a method to transformtransient solutions of continuous-time Markov chains. The word '''uniformization''' in its narrower sense involves the transformation of a [[continuous time Markov chain]] to an analgous [[discrete time Markov chain]].<ref name="ibe" />. This chain is then '''randomized''', that is, the times between changes are no longer constant, but exponential. The method is simple to program and efficiently calculates an approximation to the transient distribution at a single point in time (near zero).<ref name="stewart" />.

For a continuous time Markov chain with transition rate matrix ''Q'', the uniformized discrete time Markov chain has probability transition matrix ''P'' calculated by<ref name="stewart">{{cite book |title=Probability, Markov chains, queues, and simulation: the mathematical basis of performance modeling|last=Stewart |first=William J. |year=2009 |publisher=[[Princeton University Press]] |isbn=0691140626 |page=361}}</ref><ref name="cass">{{cite book |title=Introduction to discrete event systems|last=Cassandras |first=Christos G. |last2=Lafortune| first2=Stéphane|year=2008 |publisher=Springer |isbn=0387333320}}</ref><ref name="ross">{{cite book |title=Introduction to probability models|last=Ross |first=Sheldon M. |year=2007 |publisher=Academic Press |isbn=0125980620}}</ref>

Line 6:

with <math>\gamma</math> chosen such that <math>\gamma \geq \max_i |q_{ii}|</math>.

Randomizing the discrete-time Markov chain now results in the following formula for the solution of the <math>P(t)</math>, the transient solution of the continuous-time Markov chain

::<math>P(t) = \sum_{n=0}^{\infty} P^n e^{\gamma t} (\gamma t)^n/n!</math>

==Notes==