Piecewise-deterministic Markov process: Difference between revisions

Revision as of 14:25, 16 May 2018

Latest revision as of 14:56, 31 August 2024

(24 intermediate revisions by 13 users not shown)

Line 1:

{{Redirect|PDMP|prescription drug monitoring programs|Prescription monitoring program}}

In [[probability theory]], a '''piecewise-deterministic Markov process (PDMP)''' is a process whose behaviour is governed by random jumps at points in time, but whose evolution is deterministically governed by an [[ordinary differential equation]] between those times. The class of models is "wide enough to include as special cases virtually all the non-diffusion models of [[applied probability]]."<ref name="davis" /> The process is defined by three quantities: the flow, the jump rate, and the transition measure.<ref name="siam2010">{{Cite journal | last1 = Costa | first1 = O. L. V. | last2 = Dufour | first2 = F. | doi = 10.1137/080718541 | title = Average Continuous Control of Piecewise Deterministic Markov Processes | journal = SIAM Journal on Control and Optimization | volume = 48 | issue = 7 | pages = 4262 | year = 2010 | pmid = | pmc = | arxiv = 0809.0477}}</ref>

In [[probability theory]], a '''piecewise-deterministic Markov process (PDMP)''' is a process whose behaviour is governed by random jumps at points in time, but whose evolution is deterministically governed by an [[ordinary differential equation]] between those times. The class of models is "wide enough to include as special cases virtually all the non-diffusion models of [[applied probability]]."<ref name="davis" /> The process is defined by three quantities: the flowflow, the jump rate, and the transition measure.<ref name="siam2010">{{Cite journal | last1 = Costa | first1 = O. L. V. | last2 = Dufour | first2 = F. | doi = 10.1137/080718541 | title = Average Continuous Control of Piecewise Deterministic Markov Processes | journal = SIAM Journal on Control and Optimization | volume = 48 | issue = 7 | pages = 4262 | year = 2010 | pmidarxiv = 0809.0477| pmcs2cid = |14257280 arxiv = 0809.0477}}</ref>

The model was first introduced in a paper by [[Mark H. A. Davis]] in 1984.<ref name="davis">{{Cite journal | last1 = Davis | first1 = M. H. A. | authorlink = Mark H. A. Davis| title = Piecewise-Deterministic Markov Processes: A General Class of Non-Diffusion Stochastic Models | journal = Journal of the Royal Statistical Society. Series B (Methodological)| volume = 46 | issue = 3 | pages = 353–388 | doi = | jstor = 2345677| year = 1984 | pmid = | pmc = }}</ref>

The model was first introduced in a paper by [[Mark H. A. Davis]] in 1984.<ref name="davis">{{Cite journal | last1 = Davis | first1 = M. H. A. | authorlinkauthor-link = Mark H. A. Davis| title = Piecewise-Deterministic Markov Processes: A General Class of Non-Diffusion Stochastic Models | journal = Journal of the Royal Statistical Society. Series B (Methodological)| volume = 46 | issue = 3 | pages = 353–388 | doi = 10.1111/j.2517-6161.1984.tb01308.x| jstor = 2345677| year = 1984 | pmid = | pmc = }}</ref>

==Examples==

Line 9 ⟶ 11:

==Applications==

PDMPs have been shown useful in [[ruin theory]],<ref>{{Cite journal | last1 = Embrechts | first1 = P. | last2 = Schmidli | first2 = H. | title = Ruin Estimation for a General Insurance Risk Model | journal = Advances in Applied Probability | volume = 26 | issue = 2 | pages = 404–422 | doi = 10.2307/1427443 | jstor = 1427443| year = 1994 | pmids2cid = | pmc =124108500 }}</ref> [[queueing theory]],<ref>{{cite journal | last1 = Browne | first1 = Sid | last2 = Sigman | first2 = Karl | year = 1992 | title = Work-Modulated Queues with Applications to Storage Processes | journal = Journal of Applied Probability | volume = 29 | issue = 3 | pages = 699–712 | publisher = Applied Probability Trust | jstor = 3214906 | urldoi = 10.2307/3214906 | formats2cid = | accessdate =122273001 }}</ref><ref>{{Cite journal | last1 = Boxma | first1 = O. | authorlink1author-link1 = Onno Boxma| last2 = Kaspi | first2 = H. |author2-link=Haya Kaspi| last3 = Kella | first3 = O. | last4 = Perry | first4 = D. | title = On/off Storage Systems with State-Dependent Input, Output, and Switching Rates | doi = 10.1017/S0269964805050011 | journal = Probability in the Engineering and Informational Sciences | volume = 19 | year = 2005 | pmidpages = 1–14 | pmcciteseerx = 10.1.1.556.6718 | s2cid = 24065678 }}</ref> for modelling [[biochemistry|biochemical processes]] such as DNA replication in [[eukaryotes]] and subtilin production by the organism [[B. subtilis]] and DNA replication in [[eukaryotes]],<ref>{{cite book|chapter=Chapter 9. Stochastic Hybrid Modeling of Biochemical Processes|chapterurlchapter-url=http://www.nt.ntnu.no/users/skoge/prost/proceedings/hygea-workshop-july07-systems_biology/publications/JL2/Chapter9.pdf|title=Stochastic Hybrid Systems|first1=Christos G.|last1=Cassandras|first2=John|last2=Lygeros|publisher=CRC Press|year=2007|isbn=9780849390838}}</ref> and for modelling [[earthquake]]s.<ref>{{Cite journal | last1 = Ogata | first1 = Y. | last2 = Vere-Jones | first2 = D. | doi = 10.1016/0304-4149(84)90009-7 | title = Inference for earthquake models: A self-correcting model | journal = Stochastic Processes and theirTheir Applications | volume = 17 | issue = 2 | pages = 337 | year = 1984 | pmiddoi-access = | pmc =free }}</ref> Moreover, this class of processes has been shown to be appropriate for biophysical neuron models with stochastic ion channels.<ref>{{cite journal|last=Pakdaman|first=K.|author2=Thieullen, M. |author3=Wainrib, G. |title=Fluid limit theorems for stochastic hybrid systems with application to neuron models|journal=Advances in Applied Probability|date=September 2010|volume=42|issue=3|pages=761–794|doi=10.1239/aap/1282924062|url=https://sites.google.com/site/gwainrib/papers|arxiv=1001.2474|s2cid=18894661 }}</ref>

==Properties==

Löpker and Palmowski have shown conditions under which a [[reversed process|time reversed]] PDMP is a PDMP.<ref>{{Cite journal | last1 = Löpker | first1 = A. | last2 = Palmowski | first2 = Z. | doi = 10.1214/EJP.v18-1958 | title = On time reversal of piecewise deterministic Markov processes | journal = Electronic Journal of Probability | volume = 18 | year = 2013 | arxiv = 1110.3813| pmids2cid = | pmc =1453859 }}</ref> General conditions are known for PDMPs to be stable.<ref>{{Cite journal | last1 = Costa | first1 = O. L. V. | last2 = Dufour | first2 = F. | doi = 10.1137/060670109 | title = Stability and Ergodicity of Piecewise Deterministic Markov Processes | journal = SIAM Journal on Control and Optimization | volume = 47 | issue = 2 | pages = 1053 | year = 2008 | pmidurl = | pmc =http://www.producao.usp.br/bitstream/BDPI/14708/1/art_COSTA_Stability_and_ergodicity_of_piecewise_deterministic_Markov_2008.pdf }}</ref>

Galtier et al.<ref>{{Cite journal | last1 = Galtier | first1 = T. | doi = 10.1051/ps/2019015 | title =On the optimal importance process for piecewise deterministic Markov process | journal = Esaim: Ps | volume = 23 | year = 2019 | pages = 893–921 | s2cid = 198467101 | doi-access = free }}</ref> studied the law of the trajectories of PDMP and provided a reference measure in order to express a '''density of a trajectory''' of the PDMP. Their work opens the way to any application using densities of trajectory. (For instance, they used the density of a trajectories to perform [[importance sampling]], this work was further developed by Chennetier and Al.<ref>{{cite arXiv| last1 = Chennetier | first1 = G. | title =Adaptive importance sampling based on fault tree analysis for piecewise deterministic Markov process | year = 2022 | class = stat.CO | eprint = 2210.16185 }}</ref> to estimate the reliability of industrial systems.)

==See also==

* [[Jump diffusion]], a generalization of piecewise-deterministic Markov processes

* [[Hybrid system]] (in the context of [[dynamical system]]s), a broad class of dynamical systems that includes all jump diffusions (and hence all piecewise-deterministic Markov processes)

==References==

Line 20 ⟶ 28:

[[Category:Markov processes]]

{{probability-stub}}