Content deleted Content added
clarify, grammar, link |
Citation bot (talk | contribs) Added article-number. Removed parameters. Some additions/deletions were parameter name changes. | Use this bot. Report bugs. | Suggested by Abductive | Category:Monte Carlo methods | #UCB_Category 35/65 |
||
| (96 intermediate revisions by 61 users not shown) | |||
Line 1:
{{short description|Monte Carlo algorithm}}
[[File:Flowchart-of-Metropolis-Hastings-M-H-algorithm-for-the-parameter-estimation-using-the.png|thumb|300px|The Metropolis-Hastings algorithm sampling a [[Normal distribution|normal]] one-dimensional [[Posterior probability|posterior]] probability distribution.]]
In [[statistics]] and [[statistical physics]], the '''Metropolis–Hastings algorithm''' is a [[Markov chain Monte Carlo]] (MCMC) method for obtaining a sequence of [[pseudo-random number sampling|random samples]] from a [[probability distribution]] from which direct sampling is difficult.
Metropolis–Hastings and other MCMC algorithms are generally used for sampling from multi-dimensional distributions, especially when the number of dimensions is high. ==History==
The algorithm
Some controversy exists with regard to credit for development of the Metropolis algorithm.
This contradicts an account by Edward Teller, who states in his memoirs that the five authors of the 1953
==Description==
The Metropolis–Hastings algorithm can draw samples from any [[probability distribution]] with [[probability density]] <math>P(x)</math>, provided that we know a function <math>f(x)</math> proportional to the [[Probability density function|density]] <math>P</math> and the values of <math>f(x)</math> can be calculated. The requirement that <math>f(x)</math> must only be proportional to the density, rather than exactly equal to it, makes the Metropolis–Hastings algorithm particularly useful, because it removes the need to calculate the density's normalization factor, which is often extremely difficult in practice.
The Metropolis–Hastings algorithm generates a sequence of sample values in such a way that, as more and more sample values are produced, the distribution of values more closely approximates the desired distribution. These sample values are produced iteratively in such a way, that the distribution of the next sample depends only on the current sample value, which makes the sequence of samples a [[Markov chain]]. Specifically, at each iteration, the algorithm proposes a candidate for the next sample value based on the current sample value. Then, with some probability, the candidate is either accepted, in which case the candidate value is used in the next iteration, or it is rejected in which case the candidate value is discarded, and the current value is reused in the next iteration. The probability of acceptance is determined by comparing the values of the function <math>f(x)</math> of the current and candidate sample values with respect to the desired distribution.
The method used to propose new candidates is characterized by the probability distribution <math>g(x\mid y)</math> (sometimes written <math>Q(x\mid y)</math>) of a new proposed sample <math>x</math> given the previous sample <math>y</math>. This is called the ''proposal density'', ''proposal function'', or ''jumping distribution''. A common choice for <math>g(x\mid y)</math> is a [[Gaussian distribution]] centered at <math>y</math>, so that points closer to <math>y</math> are more likely to be visited next, making the sequence of samples into a [[Gaussian random walk]]. In the original paper by Metropolis et al. (1953), <math>g(x\mid y)</math> was suggested to be a uniform distribution limited to some maximum distance from <math>y</math>. More complicated proposal functions are also possible, such as those of [[Hamiltonian Monte Carlo]], [[Langevin Monte Carlo]], or [[preconditioned Crank–Nicolson]].
For the purpose of illustration, the Metropolis algorithm, a special case of the Metropolis–Hastings algorithm where the proposal function is symmetric, is described below.
<!---The sample values are linked in a [[Markov chain]], which means that the probability of each sample is conditionally independent of any earlier sample, given the sample immediately before it. In other words,
general idea is to generate a sequence of samples which are linked in a [[Markov chain]]; in other words, where each sample in the sequence is conditionally independent of any earlier sample, given the sample immediately before it. The procedure for choosing successive samples guarantees that the distribution of sample values will match the desired distribution
Let <math>f(x)</math> be a function that is proportional to the desired probability density function <math>P(x)</math> (a.k.a. a target distribution){{efn|In the original paper by Metropolis et al. (1953), <math>f</math> was taken to be the [[Boltzmann distribution]] as the specific application considered was [[Monte Carlo integration]] of [[equation of state|equations of state]] in [[physical chemistry]]; the extension by Hastings generalized to an arbitrary distribution <math>f</math>.}}.
# Initialization: Choose an arbitrary point <math>x_t</math> to be the first observation in the sample and choose a proposal function <math>g(x\mid y)</math>. In this section, <math>g</math> is assumed to be symmetric; in other words, it must satisfy <math>g(x\mid y) = g(y\mid x)</math>.
# For each iteration ''t'':
#* ''Propose'' a candidate <math>x'</math> for the next sample by picking from the distribution <math>g(x'\mid x_t)</math>.
#* ''Calculate'' the ''acceptance ratio'' <math>\alpha = f(x')/f(x_t)</math>, which will be used to decide whether to accept or reject the candidate{{efn|In the original paper by Metropolis et al. (1953), <math>f</math> was actually the [[Boltzmann distribution]], as it was applied to physical systems in the context of [[statistical mechanics]] (e.g., a maximal-entropy distribution of microstates for a given temperature at thermal equilibrium). Consequently, the acceptance ratio was itself an exponential of the difference in the parameters of the numerator and denominator of this ratio.}}. Because ''f'' is proportional to the density of ''P'', we have that <math>\alpha = f(x')/f(x_t) = P(x')/P(x_t)</math>.
#* ''Accept or reject'':
#** Generate a uniform random number <math>u \in [0, 1]</math>.
#** If <math>u \le \alpha</math>, then ''accept'' the candidate by setting <math>x_{t+1} = x'</math>,
#** If <math>u > \alpha</math>, then ''reject'' the candidate and set <math>x_{t+1} = x_t</math> instead.
This algorithm proceeds by randomly attempting to move about the sample space, sometimes accepting the moves and sometimes remaining in place. <math>P(x)</math> at specific point <math>x</math> is proportional to the iterations spent on the point by the algorithm. Note that the acceptance ratio <math>\alpha</math> indicates how probable the new proposed sample is with respect to the current sample, according to the distribution whose density is <math>P(x)</math>. If we attempt to move to a point that is more probable than the existing point (i.e. a point in a higher-density region of <math>P(x)</math> corresponding to an <math>\alpha > 1 \ge u</math>), we will always accept the move. However, if we attempt to move to a less probable point, we will sometimes reject the move, and the larger the relative drop in probability, the more likely we are to reject the new point. Thus, we will tend to stay in (and return large numbers of samples from) high-density regions of <math>P(x)</math>, while only occasionally visiting low-density regions. Intuitively, this is why this algorithm works and returns samples that follow the desired distribution with density <math>P(x)</math>.
Compared with an algorithm like [[adaptive rejection sampling]]<ref name=":0">{{Cite journal |last1=Gilks |first1=W. R. |last2=Wild |first2=P. |date=1992-01-01 |title=Adaptive Rejection Sampling for Gibbs Sampling |journal=Journal of the Royal Statistical Society. Series C (Applied Statistics) |volume=41 |issue=2 |pages=337–348 |doi=10.2307/2347565 |jstor=2347565}}</ref> that directly generates independent samples from a distribution, Metropolis–Hastings and other MCMC algorithms have a number of disadvantages:
* The samples are [[autocorrelation|autocorrelated]]. Even though over the long term they do correctly follow <math>P(x)</math>, a set of nearby samples will be correlated with each other and not correctly reflect the distribution. This means that effective sample sizes can be significantly lower than the number of samples actually taken, leading to large errors.
* Although the Markov chain eventually converges to the desired distribution, the initial samples may follow a very different distribution, especially if the starting point is in a region of low density. As a result, a ''burn-in'' period is typically necessary,<ref>{{Cite book |title=Bayesian data analysis |date=2004 |publisher=Chapman & Hall / CRC |others=Gelman, Andrew |isbn=978-1584883883 |edition=2nd |___location=Boca Raton, Fla. |oclc=51991499}}</ref> where an initial number of samples are thrown away.
On the other hand, most simple [[rejection sampling]] methods suffer from the "[[curse of dimensionality]]", where the probability of rejection increases exponentially as a function of the number of dimensions. Metropolis–Hastings, along with other MCMC methods, do not have this problem to such a degree, and thus are often the only solutions available when the number of dimensions of the distribution to be sampled is high. As a result, MCMC methods are often the methods of choice for producing samples from [[hierarchical Bayesian model]]s and other high-dimensional statistical models used nowadays in many disciplines.
In [[multivariate distribution|multivariate]] distributions, the classic Metropolis–Hastings algorithm as described above involves choosing a new multi-dimensional sample point. When the number of dimensions is high, finding the suitable jumping distribution to use can be difficult, as the different individual dimensions behave in very different ways, and the jumping width (see above) must be "just right" for all dimensions at once to avoid excessively slow mixing. An alternative approach that often works better in such situations, known as [[Gibbs sampling]], involves choosing a new sample for each dimension separately from the others, rather than choosing a sample for all dimensions at once. That way, the problem of sampling from potentially high-dimensional space will be reduced to a collection of problems to sample from small dimensionality.<ref>{{Cite journal |last=Lee |first=Se Yoon |year=2021 |title=Gibbs sampler and coordinate ascent variational inference: A set-theoretical review |journal=Communications in Statistics - Theory and Methods |volume=51 |issue=6 |pages=1549–1568 |arxiv=2008.01006 |doi=10.1080/03610926.2021.1921214 |s2cid=220935477}}</ref> This is especially applicable when the multivariate distribution is composed of a set of individual [[random variable]]s in which each variable is conditioned on only a small number of other variables, as is the case in most typical [[hierarchical Bayesian model|hierarchical models]]. The individual variables are then sampled one at a time, with each variable conditioned on the most recent values of all the others. Various algorithms can be used to choose these individual samples, depending on the exact form of the multivariate distribution: some possibilities are the [[adaptive rejection sampling]] methods,<ref name=":0" /> the adaptive rejection Metropolis sampling algorithm,<ref>{{Cite journal |last1=Gilks |first1=W. R. |last2=Best |first2=N. G. |author-link2=Nicky Best |last3=Tan |first3=K. K. C. |date=1995-01-01 |title=Adaptive Rejection Metropolis Sampling within Gibbs Sampling |journal=Journal of the Royal Statistical Society. Series C (Applied Statistics) |volume=44 |issue=4 |pages=455–472 |doi=10.2307/2986138 |jstor=2986138}}</ref> a simple one-dimensional Metropolis–Hastings step, or [[slice sampling]].
==Formal derivation==
The purpose of the Metropolis–Hastings algorithm is to generate a collection of states according to a desired distribution <math>P(x)</math>. To accomplish this, the algorithm uses a [[Markov process]], which asymptotically reaches a unique [[Markov chain#Steady-state analysis and limiting distributions|stationary distribution]] <math>\pi(x)</math> such that <math>\pi(x) = P(x)</math>
A Markov process is uniquely defined by its transition probabilities
# ''
# ''
The Metropolis–Hastings algorithm involves designing a Markov process (by constructing transition probabilities)
: <math>P(x'
which is re-written as
: <math>\frac{P(x'
The approach is to separate the transition in two sub-steps; the proposal and the acceptance-rejection. The
: <math>P(x'
Inserting this relation in the previous equation, we have
: <math>\frac{A(x'
The next step in the derivation is to choose an acceptance ratio that fulfills the condition above. One common choice is the Metropolis choice:
: <math>A(x'
For this Metropolis acceptance ratio <math>A</math>, either <math>A(x', x) = 1</math> or <math>A(x, x') = 1</math> and, either way, the condition is satisfied.
The Metropolis–Hastings algorithm can thus be written as follows:
# Initialise
## Pick an initial state <math>x_0</math>
## Set <math>t = 0</math>
# Iterate
##
##
##
### generate a uniform random number <math>u \in [0, 1]</math>;
### if <math>u \le A(x' , x_t)</math>, then ''accept'' the new state and set <math>x_{t+1} = x'</math>;
### if <math>u > A(x' , x_t)</math>, then ''reject'' the new state, and copy the old state forward <math>x_{t+1} = x_{t}</math>
##
Provided that specified conditions are met, the empirical distribution of saved states <math>x_0, \ldots, x_T</math> will approach <math>P(x)</math>. The number of iterations (<math>T</math>) required to effectively estimate <math>P(x)</math> depends on the number of factors, including the relationship between <math>P(x)</math> and the proposal distribution and the desired accuracy of estimation.<ref>Raftery, Adrian E., and Steven Lewis. "How Many Iterations in the Gibbs Sampler?
It is important to notice that it is not clear, in a general problem, which distribution <math>
==Use in numerical integration==
{{main|Monte Carlo integration}}
A common use of Metropolis–Hastings algorithm is to compute an integral. Specifically, consider a space <math>\Omega \subset \mathbb{R}</math> and a probability distribution <math>P(x)</math> over <math>\Omega</math>, <math>x \in \Omega</math>.
: <math>P(E) = \int_\Omega A(x) P(x) \,dx,</math>
where <math>A(x)</math> is a (measurable) function of interest.
For example, consider a [[statistic]] <math>E(x)</math> and its probability distribution <math>P(E)</math>, which is a [[marginal distribution]]. Suppose that the goal is to estimate <math>P(E)</math> for <math>E</math> on the tail of <math>P(E)</math>. Formally, <math>P(E)</math> can be written as
: <math>
P(E) = \int_\Omega P(E\mid x) P(x) \,dx = \int_\Omega \delta\big(E - E(x)\big) P(x) \,dx = E \big(P(E\mid X)\big)
</math>
and, thus, estimating <math>P(E)</math> can be accomplished by estimating the expected value of the [[indicator function]] <math>A_E(x) \equiv \mathbf{1}_E(x)</math>, which is 1 when <math>E(x) \in [E, E + \Delta E]</math> and zero otherwise.
Because <math>E</math> is on the tail of <math>P(E)</math>, the probability to draw a state <math>x</math> with <math>E(x)</math> on the tail of <math>P(E)</math> is proportional to <math>P(E)</math>, which is small by definition.
==Step-by-step instructions==
[[File:3dRosenbrock.png|thumb|300px|Three [[Markov chain]]s running on the 3D [[Rosenbrock function]] using the Metropolis–Hastings algorithm. The chains converge and mix in the region where the function is high. The approximate position of the maximum has been illuminated. The red points are the ones that remain after the burn-in process. The earlier ones have been discarded.]]
Suppose that the most recent value sampled is <math>x_t
: <math>a = a_1 a_2,</math>
where
: <math>a_1 = \frac{P(x')}{P(x_t)}</math>
is the probability (e.g., Bayesian posterior) ratio between the proposed sample <math>x'
: <math>a_2 = \frac{g(x_t \mid x')}{g(x' \mid x_t)}</math>
is the ratio of the proposal density in two directions (from <math>x_t
This is equal to 1 if the proposal density is symmetric.
Then the new state <math>
: If <math>a \geq 1{:}</math>
:: <math>x_{t+1} = x',</math>
: else:
\begin{cases}
x' & \text{with probability } a, \\
x_t & \text{with probability } 1-a.
\end{cases}
</math>
The Markov chain is started from an arbitrary initial value <math>
The algorithm works best if the proposal density matches the shape of the target distribution <math>
If a Gaussian proposal density <math>
This is usually done by calculating the ''acceptance rate'', which is the fraction of proposed samples that is accepted in a window of the last <math>
The desired acceptance rate depends on the target distribution, however it has been shown theoretically that the ideal acceptance rate for a one-dimensional Gaussian distribution is
If <math>
if <math>
== Bayesian Inference ==
{{main article|Bayesian Inference}}
MCMC can be used to draw samples from the [[posterior distribution]] of a statistical model.
The acceptance probability is given by:
<math>P_{acc}(\theta_i \to \theta^*)=\min\left(1, \frac{\mathcal{L}(y|\theta^*)P(\theta^*)}{\mathcal{L}(y|\theta_i)P(\theta_i)}\frac{Q(\theta_i|\theta^*)}{Q(\theta^*|\theta_i)}\right),</math>
where <math>\mathcal{L}</math> is the [[likelihood]], <math>P(\theta)</math> the prior probability density and <math>Q</math> the (conditional) proposal probability.
==See also==
* [[Genetic algorithm]]s
* [[
* [[Metropolis light transport]]
* [[Multiple-try Metropolis]]
* [[Parallel tempering]]
* [[Particle filter|Sequential Monte Carlo]]
* [[Simulated annealing]]
Line 181 ⟶ 166:
==References==
{{Reflist|refs=
<ref name="Hastings">{{Cite journal |last=Hastings |first=W.K. |year=1970 |title=Monte Carlo Sampling Methods Using Markov Chains and Their Applications |journal=[[Biometrika]] |volume=57 |issue=1 |pages=97–109 |bibcode=1970Bimka..57...97H |doi=10.1093/biomet/57.1.97 |jstor=2334940 |zbl=0219.65008}}</ref>
<ref name="Teller">Teller, Edward. ''Memoirs: A Twentieth-Century Journey in Science and Politics''. [[Perseus Publishing]], 2001, p. 328</ref>
<ref name="Barth">Rosenbluth, Marshall. [https://www.aip.org/history-programs/niels-bohr-library/oral-histories/28636-1 "Oral History Transcript"]. American Institute of Physics</ref>
<ref name="Gubernatis">{{Cite journal |last=J.E. Gubernatis |year=2005 |title=Marshall Rosenbluth and the Metropolis Algorithm |url=https://zenodo.org/record/1231899 |journal=[[Physics of Plasmas]] |volume=12 |issue=5 |article-number=057303 |bibcode=2005PhPl...12e7303G |doi=10.1063/1.1887186}}</ref>
<ref name="Rosenbluth">{{Cite journal |last=M.N. Rosenbluth |year=2003 |title=Genesis of the Monte Carlo Algorithm for Statistical Mechanics |journal=[[AIP Conference Proceedings]] |volume=690 |pages=22–30 |bibcode=2003AIPC..690...22R |doi=10.1063/1.1632112}}</ref>
<!--<ref name="Dyson">{{Cite journal |last=F. Dyson |year=2006 |title=Marshall N. Rosenbluth |journal=[[Proceedings of the American Philosophical Society]] |volume=250 |pages=404}}</ref>-->
<ref name="Roberts">{{Cite journal |last1=Roberts |first1=G.O. |last2=Gelman |first2=A. |last3=Gilks |first3=W.R. |year=1997 |title=Weak convergence and optimal scaling of random walk Metropolis algorithms |url=http://www.stat.columbia.edu/~gelman/research/published/theory7.ps |journal=[[Ann. Appl. Probab.]] |volume=7 |issue=1 |pages=110–120 |citeseerx=10.1.1.717.2582 |doi=10.1214/aoap/1034625254}}</ref>
<ref name="Roberts_Casella">{{Cite book |last1=Robert |first1=Christian |url=https://archive.org/details/springer_10.1007-978-1-4757-4145-2 |title=Monte Carlo Statistical Methods |last2=Casella |first2=George |publisher=Springer |year=2004 |isbn=978-0387212395}}</ref>
<ref name="Newman_Barkema">{{Cite book |last1=Newman |first1=M. E. J. |title=Monte Carlo Methods in Statistical Physics |last2=Barkema |first2=G. T. |publisher=Oxford University Press |year=1999 |isbn=978-0198517979 |___location=USA}}</ref>
}}
==Notes==
{{notelist}}
== Further reading ==
* [[Bernd A. Berg]]. ''Markov Chain Monte Carlo Simulations and Their Statistical Analysis''. Singapore, [[World Scientific]], 2004.
*Chib, Siddhartha;
* [http://www.tandfonline.com/doi/abs/10.1080/03610918.2013.777455#.VOk8J1PF9_c David D. L. Minh and Do Le Minh. "Understanding the Hastings Algorithm." Communications in Statistics - Simulation and Computation, 44:2
* Bolstad, William M. (2010) ''Understanding Computational Bayesian Statistics'', [[John Wiley & Sons]] {{ISBN|0-470-04609-0}}
{{DEFAULTSORT:Metropolis-Hastings Algorithm}}
| |||