Content deleted Content added
CoreSolver (talk | contribs) m Cleared an inconsistency Tags: Reverted Disambiguation links added |
Undid revision 1135252272 by CoreSolver (talk); this links to a disambiguation page and is more ambiguous as "whole numbers" generally include negatives, whereas the only ambiguity of natural numbers (inclusion of 0) is clear from context here |
||
Line 2:
{{About|the mathematical properties of discrete-time Markov chains|the general field of study|Markov chain}}
[[File:Markovkate 01.svg|thumb|A Markov chain with two states, ''A'' and ''E''.]]
In [[probability]], a '''discrete-time Markov chain''' ('''DTMC''') is a sequence of random variables, known as a [[stochastic process]], in which the value of the next variable depends only on the value of the current variable, and not any variables in the past. For instance, a machine may have two states, ''A'' and ''E''. When it is in state ''A'', there is a 40% chance of it moving to state ''E'' and a 60% chance of it remaining in state ''A''. When it is in state ''E'', there is a 70% chance of it moving to ''A'' and a 30% chance of it staying in ''E''. The sequence of states of the machine is a Markov chain. If we denote the chain by <math>X_0, X_1, X_2, ...</math> then <math>X_0</math> is the state which the machine starts in and <math>X_{10}</math> is the [[random variable]] describing its state after 10 transitions. The process continues forever, indexed by the [[
An example of a stochastic process which is not a Markov chain is the model of a machine which has states ''A'' and ''E'' and moves to ''A'' from either state with 50% chance if it has ever visited ''A'' before, and 20% chance if it has never visited ''A'' before (leaving a 50% or 80% chance that the machine moves to ''E''). This is because the behavior of the machine depends on the whole history—if the machine is in ''E'', it may have a 50% or 20% chance of moving to ''A'', depending on its past values. Hence, it does not have the [[Markov property]].
| |||