Exchangeable random variables: Difference between revisions

Revision as of 01:06, 8 January 2021

Revision as of 19:47, 2 September 2021

Line 24:

The property of exchangeability is closely related to the use of [[independent and identically distributed random variables|independent and identically distributed]] (i.i.d.) random variables in statistical models. A sequence of random variables that are i.i.d, conditional on some underlying distributional form, is exchangeable. This follows directly from the structure of the joint probability distribution generated by the i.i.d. form.

MoreoverMixtures of exchangeable sequences (in particular, thesequences of i.i.d. variables) are exchangeable. The converse can be established for infinite sequences, through an important [[de Finetti's theorem|representation theorem]] by [[Bruno de Finetti]] (later extended by other probability theorists such as [[Paul Halmos|Halmos]] and [[Leonard Jimmie Savage|Savage]]). The extended versions of the theorem show that in any infinite sequence of exchangeable random variables, the random variables are conditionally [[independent and identically-distributed random variables|independent and identically-distributed]], given the underlying distributional form. This theorem is stated briefly below. (De Finetti's original theorem only showed this to be true for random indicator variables, but this was later extended to encompass all sequences of random variables.) Another way of putting this is that [[de Finetti's theorem]] characterizes exchangeable sequences as mixtures of i.i.d. sequences — while an exchangeable sequence need not itself be unconditionally i.i.d., it can be expressed as a mixture of underlying i.i.d. sequences.<ref name="ChowTeicher"/>

This means that infinite sequences of exchangeable random variables can be regarded equivalently as sequences of conditionally i.i.d. random variables, based on some underlying distributional form. (Note that this equivalence does not quite hold for finite exchangeability. However, for finite vectors of random variables there is a close approximation to the i.i.d. model.) An infinite exchangeable sequence is [[strictly stationary]] and so a [[law of large numbers]] in the form of [[Birkhoff–Khinchin theorem]] applies.<ref name="Kallenberg"/> This means that the underlying distribution can be given an operational interpretation as the limiting empirical distribution of the sequence of values. The close relationship between exchangeable sequences of random variables and the i.i.d. form means that the latter can be justified on the basis of infinite exchangeability. This notion is central to [[Bruno de Finetti|Bruno de Finetti's]] development of [[predictive inference]] and to [[Bayesian statistics]]. It can also be shown to be a useful foundational assumption in [[frequentist statistics]] and to link the two paradigms.<ref name="O'Neill">