Content deleted Content added
→Applications: source added that expounds on the importance of exchangeability for conformal prediction |
Link suggestions feature: 2 links added. |
||
| (6 intermediate revisions by 5 users not shown) | |||
Line 19:
== History ==
The concept was introduced by [[William Ernest Johnson]] in his 1924 book ''Logic, Part III: The Logical Foundations of Science''.<ref>
== Exchangeability and the i.i.d. statistical model ==
The property of exchangeability is closely related to the use of [[independent and identically distributed random variables|independent and identically distributed]] (i.i.d.) random variables in statistical models.<ref>{{cite book |last1=Cordani |first1=L. K. |last2=Wechsler |first2=S. |year=2006 |chapter=Teaching independence and exchangeability |title=Proceedings of the International Conference on Teaching Statistics |___location=Den Haag |publisher=International Association for Statistical Education |chapter-url=https://iase-web.org/documents/papers/icots7/3I1_CORD.pdf?1402524964 }}</ref> A sequence of random variables that are i.i.d, conditional on some underlying distributional form, is exchangeable. This follows directly from the structure of the joint probability distribution generated by the i.i.d. form.
Mixtures of exchangeable sequences (in particular, sequences of i.i.d. variables) are exchangeable. The converse can be established for infinite sequences, through an important [[de Finetti's theorem|representation theorem]] by [[Bruno de Finetti]] (later extended by other probability theorists such as [[Paul Halmos|Halmos]] and [[Leonard Jimmie Savage|Savage]]).<ref>{{cite book |first=P. |last=Diaconis |authorlink=Persi Diaconis |chapter=Recent Progress on de Finetti's Notions of Exchangeability |pages=111–125 |year=1988 |title=Bayesian Statistics |volume=3 |editor1-first=J. M. |editor1-last=Bernardo |editor1-link=José-Miguel Bernardo |editor2-first=M. H. |editor2-last=DeGroot |editor3-first=D. V. |editor3-last=Lindley |editor4-first=A. F. M. |editor4-last=Smith |display-editors=1 |publisher=Oxford University Press |isbn=0-19-852220-7 }}</ref> The extended versions of the theorem show that in any infinite sequence of exchangeable random variables, the random variables are conditionally [[independent and identically-distributed random variables|independent and identically-distributed]], given the underlying distributional form. This theorem is stated briefly below. (De Finetti's original theorem only showed this to be true for random indicator variables, but this was later extended to encompass all sequences of random variables.) Another way of putting this is that [[de Finetti's theorem]] characterizes exchangeable sequences as mixtures of i.i.d. sequences—while an exchangeable sequence need not itself be unconditionally i.i.d., it can be expressed as a mixture of underlying i.i.d. sequences.<ref name="ChowTeicher"/>
Line 43:
These equations show the joint distribution or density characterised as a mixture distribution based on the underlying limiting empirical distribution (or a parameter indexing this distribution).
Note that not all finite exchangeable sequences are mixtures of i.i.d. To see this, consider sampling without replacement from a [[finite set]] until no elements are left. The resulting sequence is exchangeable, but not a mixture of i.i.d. Indeed, conditioned on all other elements in the sequence, the remaining element is known.
== Covariance and correlation ==
Exchangeable sequences have some basic [[covariance and correlation]] properties which mean that they are generally positively correlated. For infinite sequences of exchangeable random variables, the covariance between the random variables is equal to the variance of the mean of the underlying distribution function.<ref name="O'Neill"/> For finite exchangeable sequences the covariance is also a fixed value which does not depend on the particular random variables in the sequence. There is a weaker lower bound than for infinite exchangeability and it is possible for negative correlation to exist.
'''Covariance for exchangeable sequences (infinite):''' If the sequence <math>X_1,X_2,X_3,\ldots</math> is exchangeable, then
Line 85:
Exchangeable random variables arise in the study of [[U statistic]]s, particularly in the Hoeffding decomposition.<ref>{{cite book |last=Borovskikh | first=Yu. V. | title=''U''-statistics in Banach spaces | publisher=VSP | ___location=Utrecht | year=1996 | pages=365–376 | isbn=90-6764-200-2 | mr=1419498|chapter=Chapter 10 Dependent variables}}</ref>
Exchangeability is a key assumption of the distribution-free inference method of [[conformal prediction]].<ref>{{cite journal |
==See also==
* [[De Finetti theorem]]
* [[Hewitt-Savage zero-one law]]
* [[Resampling (statistics)|Resampling]]
* {{sectionlink|Resampling (statistics)|Permutation tests}}, statistical tests based on exchanging between groups
==
{{Reflist}}
==
* Aldous, David J., ''Exchangeability and related topics'', in: École d'Été de Probabilités de Saint-Flour XIII — 1983, Lecture Notes in Math. 1117, pp. 1–198, Springer, Berlin, 1985. {{ISBN|978-3-540-15203-3}} {{doi|10.1007/BFb0099421}}
* Chow, Yuan Shih and Teicher, Henry, ''Probability theory. Independence, interchangeability, martingales,'' Springer Texts in Statistics, 3rd ed., Springer, New York, 1997. xxii+488 pp. {{ISBN|0-387-98228-0}}
* {{cite book |last=Dawid |first=A. Philip |chapter=Exchangeability and its ramifications |pages=19–30 |title=Bayesian Theory and Applications |editor-first=Paul |editor-last=Damien |editor2-first=Petros |editor2-last=Dellaportas |editor3-first=Nicholas G. |editor3-last=Polson |editor4-first=David A. |editor4-last=Stephens |display-editors=1 |publisher=Oxford University Press |year=2013 |isbn=978-0-19-969560-7 }}
* [[Olav Kallenberg|Kallenberg, O.]], ''Probabilistic symmetries and invariance principles''. Springer-Verlag, New York (2005). 510 pp. {{ISBN|0-387-25115-4}}.
* Kingman, J. F. C., ''Uses of exchangeability'', Ann. Probability 6 (1978) 83–197 {{MR|494344}} {{JSTOR|2243211}}
* O'Neill, B. (2009) Exchangeability, Correlation and Bayes' Effect. ''International Statistical Review'' '''77(2)''', pp. 241–250. {{ISBN|978-3-540-15203-3}} {{doi|10.1111/j.1751-5823.2008.00059.x}}
* {{cite book|title=Limit theorems for sums of exchangeable random variables|first1=Robert Lee|last1=Taylor|first2=Peter Z.|last2=Daffer|first3=Ronald F.|last3=Patterson|publisher=Rowman and Allanheld|url=https://books.google.com/books?id=6RaoAAAAIAAJ|year=1985|pages=1–152|isbn=9780847674350}}
{{DEFAULTSORT:Exchangeable Random Variables}}
| |||