Content deleted Content added
Citation bot (talk | contribs) Altered template type. Add: doi-access, pages, issue, volume, journal, doi, arxiv, class, date, title, eprint, authors 1-4. Removed URL that duplicated identifier. Changed bare reference to CS1/2. Removed parameters. Some additions/deletions were parameter name changes. | Use this bot. Report bugs. | Suggested by Headbomb | #UCB_toolbar |
m →Software: HTTP to HTTPS for SourceForge |
||
(10 intermediate revisions by 7 users not shown) | |||
Line 2:
{{Bayesian statistics}}
'''Approximate Bayesian computation''' ('''ABC''') constitutes a class of [[Computational science|computational methods]] rooted in [[Bayesian statistics]] that can be used to estimate the posterior distributions of model parameters.
In all model-based [[statistical inference]], the [[likelihood|likelihood function]] is of central importance, since it expresses the probability of the observed data under a particular [[statistical model]], and thus quantifies the support data lend to particular values of parameters and to choices among different models. For simple models, an analytical formula for the likelihood function can typically be derived. However, for more complex models, an analytical formula might be elusive or the likelihood function might be computationally very costly to evaluate.
Line 17:
Although Diggle and Gratton's approach had opened a new frontier, their method was not yet exactly identical to what is now known as ABC, as it aimed at approximating the likelihood rather than the posterior distribution. An article of [[Simon Tavaré]] and co-authors was first to propose an ABC algorithm for posterior inference.<ref name="Tavare" /> In their seminal work, inference about the genealogy of DNA sequence data was considered, and in particular the problem of deciding the posterior distribution of the time to the [[most recent common ancestor]] of the sampled individuals. Such inference is analytically intractable for many demographic models, but the authors presented ways of simulating coalescent trees under the putative models. A sample from the posterior of model parameters was obtained by accepting/rejecting proposals based on comparing the number of segregating sites in the synthetic and real data. This work was followed by an applied study on modeling the variation in human Y chromosome by [[Jonathan K. Pritchard]] and co-authors using the ABC method.<ref name="Pritchard1999" /> Finally, the term approximate Bayesian computation was established by Mark Beaumont and co-authors,<ref name="Beaumont2002" /> extending further the ABC methodology and discussing the suitability of the ABC-approach more specifically for problems in population genetics. Since then, ABC has spread to applications outside population genetics, such as systems biology, epidemiology, and [[phylogeography]].
Approximate Bayesian computation can be understood as a kind of Bayesian version of [[indirect inference]].<ref>{{cite arXiv | eprint=1803.01999 | author1=Christopher C Drovandi | title=ABC and Indirect Inference | date=2018 | class=stat.CO }}</ref><ref name="Peters 2009">{{Cite journal |last=Peters |first=Gareth |date=2009 |title=Advances in Approximate Bayesian Computation and Trans-Dimensional Sampling Methodology |url=https://www.ssrn.com/abstract=3785580 |journal=SSRN Electronic Journal |language=en |doi=10.2139/ssrn.3785580 |issn=1556-5068|hdl=1959.4/50086 |hdl-access=free }}</ref>
Several efficient Monte Carlo based approaches have been developed to perform sampling from the ABC posterior distribution for purposes of estimation and prediction problems. A popular choice is the SMC Samplers
==Method==
Line 30:
where <math>p(\theta|D)</math> denotes the posterior, <math>p(D|\theta)</math> the likelihood, <math>p(\theta)</math> the prior, and <math>p(D)</math> the evidence (also referred to as the [[marginal likelihood]] or the prior predictive probability of the data). Note that the denominator <math>p(D)</math> is normalizing the total probability of the posterior density <math>p(\theta|D)</math> to one and can be calculated that way.
The prior represents beliefs or knowledge (such as
===The ABC rejection algorithm===
All ABC-based methods approximate the likelihood function by simulations, the outcomes of which are compared with the observed data.<ref>{{Cite journal |last=Hunter |first=Dawn |date=2006-12-08 |title=Bayesian inference, Monte Carlo sampling and operational risk |url=https://www.risk.net/journal-of-operational-risk/2160915/bayesian-inference-monte-carlo-sampling-and-operational-risk |journal=Journal of Operational Risk |volume=1 |issue=3 |pages=27–50 |language=en |doi=10.21314/jop.2006.014|url-access=subscription }}</ref><ref
:<math>\rho (\hat{D},D)\le\epsilon</math>,
Line 182:
===Approximation of the posterior===
A non-negligible <math>\epsilon</math> comes with the price that one samples from <math>p(\theta|\rho(\hat{D},D)\le\epsilon)</math> instead of the true posterior <math>p(\theta|D)</math>. With a sufficiently small tolerance, and a sensible distance measure, the resulting distribution <math>p(\theta|\rho(\hat{D},D)\le\epsilon)</math> should often approximate the actual target distribution <math>p(\theta|D)</math> reasonably well. On the other hand, a tolerance that is large enough that every point in the parameter space becomes accepted will yield a replica of the prior distribution. There are empirical studies of the difference between <math>p(\theta|\rho(\hat{D},D)\le\epsilon)</math> and <math>p(\theta|D)</math> as a function of <math>\epsilon</math>,<ref name="Sisson" />
As an attempt to correct some of the error due to a non-zero <math>\epsilon</math>, the usage of local linear weighted regression with ABC to reduce the variance of the posterior estimates has been suggested.<ref name="Beaumont2002" /> The method assigns weights to the parameters according to how well simulated summaries adhere to the observed ones and performs linear regression between the summaries and the weighted parameters in the vicinity of observed summaries. The obtained regression coefficients are used to correct sampled parameters in the direction of observed summaries. An improvement was suggested in the form of nonlinear regression using a feed-forward neural network model.<ref name="Blum2010" /> However, it has been shown that the posterior distributions obtained with these approaches are not always consistent with the prior distribution, which did lead to a reformulation of the regression adjustment that respects the prior distribution.<ref name="Leuenberger2009" />
Line 195:
An algorithm has been proposed for identifying a representative subset of summary statistics, by iteratively assessing whether an additional statistic introduces a meaningful modification of the posterior.<ref name="Joyce" /> One of the challenges here is that a large ABC approximation error may heavily influence the conclusions about the usefulness of a statistic at any stage of the procedure. Another method<ref name="Nunes" /> decomposes into two main steps. First, a reference approximation of the posterior is constructed by minimizing the [[Entropy (statistical thermodynamics)|entropy]]. Sets of candidate summaries are then evaluated by comparing the ABC-approximated posteriors with the reference posterior.
With both of these strategies, a subset of statistics is selected from a large set of candidate statistics. Instead, the [[partial least squares regression]] approach uses information from all the candidate statistics, each being weighted appropriately.<ref name="Wegmann" /> Recently, a method for constructing summaries in a semi-automatic manner has attained a considerable interest.<ref name="Fearnhead" /> This method is based on the observation that the optimal choice of summary statistics, when minimizing the quadratic loss of the parameter point estimates, can be obtained through the posterior mean of the parameters, which is approximated by performing a linear regression based on the simulated data. Summary statistics for model selection have been obtained using [[multinomial logistic regression]] on simulated data, treating competing models as the label to predict.<ref name="Prangle" />
Methods for the identification of summary statistics that could also simultaneously assess the influence on the approximation of the posterior would be of substantial value.<ref name="Marjoram" /> This is because the choice of summary statistics and the choice of tolerance constitute two sources of error in the resulting posterior distribution. These errors may corrupt the ranking of models and may also lead to incorrect model predictions
===Bayes factor with ABC and summary statistics===
Line 296:
| <ref name="Wegmann2010" />
|-
| [
| Open source software package consisting of several C and R programs that are run with a Perl "front-end". Hierarchical coalescent models. Population genetic data from multiple co-distributed species.
| <ref name="Hickerson07" />
Line 322:
| [https://abcpy.readthedocs.io/en/latest/ ABCpy]
| Python package for ABC and other likelihood-free inference schemes. Several state-of-the-art algorithms available. Provides quick way to integrate existing generative (from C++, R etc.), user-friendly parallelization using MPI or Spark and summary statistics learning (with neural network or linear regression).
| <ref>{{cite journal |title=ABCpy: A High-Performance Computing Perspective to Approximate Bayesian Computation |last1=Dutta |first1=R |last2=Schoengens |first2=M |last3=Pacchiardi |first3=L |last4=Ummadisingu |first4=A |last5=Widmer |first5=N |last6=Onnela |first6=J. P. |last7=Mira |first7=A|journal=Journal of Statistical Software |author7-link=Antonietta Mira |year=2021|volume=100 |issue=7 |doi=10.18637/jss.v100.i07 |doi-access=free|arxiv=1711.04694 |s2cid=88516340 }}</ref>
|}
The suitability of individual software packages depends on the specific application at hand, the computer system environment, and the algorithms required.
Line 339 ⟶ 338:
<ref name="Bharti">{{cite journal | last1 = Bharti | first1 = A | last2 = Briol | first2 = F.-X. | last3 = Pedersen | first3 = T | year = 2021 | title = A General Method for Calibrating Stochastic Radio Channel Models with Kernels | journal = IEEE Transactions on Antennas and Propagation | volume = 70 | issue = 6 | pages = 3986–4001 | doi=10.1109/TAP.2021.3083761| arxiv = 2012.09612 | s2cid = 233880538 }}</ref>
<ref name="Bertorelle">{{cite journal | last1 = Bertorelle | first1 = G | last2 = Benazzo | first2 = A | last3 = Mona | first3 = S | year = 2010 | title = ABC as a flexible framework to estimate demography over space and time: some cons, many pros | journal = Molecular Ecology | volume = 19 | issue = 13| pages = 2609–2625 | doi=10.1111/j.1365-294x.2010.04690.x| pmid = 20561199 | bibcode = 2010MolEc..19.2609B | s2cid = 12129604 | doi-access = free }}</ref>
<ref name="Csillery">{{cite journal | last1 = Csilléry | first1 = K | last2 = Blum | first2 = MGB | last3 = Gaggiotti | first3 = OE | last4 = François | first4 = O | year = 2010 | title = Approximate Bayesian Computation (ABC) in practice | journal = Trends in Ecology & Evolution | volume = 25 | issue = 7| pages = 410–418 | doi=10.1016/j.tree.2010.04.001| pmid = 20488578 | bibcode = 2010TEcoE..25..410C | s2cid = 13957079 }}</ref>
<ref name="Rubin">{{cite journal | last1 = Rubin | first1 = DB | year = 1984 | title = Bayesianly Justifiable and Relevant Frequency Calculations for the Applied Statistician | journal = The Annals of Statistics | volume = 12 | issue = 4| pages = 1151–1172 | doi=10.1214/aos/1176346785| doi-access = free }}</ref>
<ref name="Marjoram">{{cite journal | last1 = Marjoram | first1 = P | last2 = Molitor | first2 = J | last3 = Plagnol | first3 = V | last4 = Tavare | first4 = S | year = 2003 | title = Markov chain Monte Carlo without likelihoods | journal = Proc Natl Acad Sci U S A | volume = 100 | issue = 26| pages = 15324–15328 | doi=10.1073/pnas.0306899100| pmid = 14663152 | pmc = 307566 | bibcode = 2003PNAS..10015324M | doi-access = free }}</ref>
Line 369 ⟶ 368:
<ref name="Templeton2010">{{cite journal | last1 = Templeton | first1 = AR | year = 2010 | title = Coherent and incoherent inference in phylogeography and human evolution | journal = Proceedings of the National Academy of Sciences of the United States of America | volume = 107 | issue = 14| pages = 6376–6381 | doi=10.1073/pnas.0910647107| pmid = 20308555 | pmc = 2851988 | bibcode = 2010PNAS..107.6376T| doi-access = free }}</ref>
<!--<ref name="Fagundes">{{cite journal | last1 = Fagundes | first1 = NJR | last2 = Ray | first2 = N | last3 = Beaumont | first3 = M | last4 = Neuenschwander | first4 = S | last5 = Salzano | first5 = FM |display-authors=et al | year = 2007 | title = Statistical evaluation of alternative models of human evolution | journal = Proceedings of the National Academy of Sciences of the United States of America | volume = 104 | pages = 17614–17619 | doi=10.1073/pnas.0708280104 | pmid=17978179 | pmc=2077041}}</ref>-->
<!-- <ref name="Gelfand">{{cite journal | last1 = Gelfand | first1 = AE | last2 = Dey | first2 = DK | year = 1994 | title = Bayesian model choice: Asymptotics and exact calculations | journal = J R
<!-- <ref name="Bernardo">Bernardo JM, Smith AFM (1994) Bayesian Theory: John Wiley.</ref> -->
<!-- <ref name="Box">Box G, Draper NR (1987) Empirical Model-Building and Response Surfaces: John Wiley and Sons, Oxford.</ref> -->
Line 383 ⟶ 382:
<ref name="Dean">{{cite arXiv | eprint=1103.5399 | last1=Dean | first1=Thomas A. | last2=Singh | first2=Sumeetpal S. | last3=Jasra | first3=Ajay | last4=Peters | first4=Gareth W. | title=Parameter Estimation for Hidden Markov Models with Intractable Likelihoods | date=2011 | class=math.ST }}</ref>
<ref name="Fearnhead">{{cite arXiv | eprint=1004.1112 | last1=Fearnhead | first1=Paul | last2=Prangle | first2=Dennis | title=Constructing Summary Statistics for Approximate Bayesian Computation: Semi-automatic ABC | date=2010 | class=stat.ME }}</ref>
<ref name="Wilkinson">{{cite journal | arxiv=0811.3355 | doi=10.1515/sagmb-2013-0010 | title=Approximate Bayesian computation (ABC) gives exact results under the assumption of model error | date=2013 | last1=Wilkinson | first1=Richard David | journal=Statistical Applications in Genetics and Molecular Biology | volume=12 | issue=2 | pmid=23652634 }}</ref>
<ref name="Nunes">{{cite journal | last1 = Nunes | first1 = MA | last2 = Balding | first2 = DJ | year = 2010 | title = On optimal selection of summary statistics for approximate Bayesian computation | journal = Stat Appl Genet Mol Biol | volume = 9 | page = Article 34 | doi=10.2202/1544-6115.1576| pmid = 20887273 | s2cid = 207319754 }}</ref>
<ref name="Joyce">{{cite journal | last1 = Joyce | first1 = P | last2 = Marjoram | first2 = P | year = 2008 | title = Approximately sufficient statistics and bayesian computation | journal = Stat Appl Genet Mol Biol | volume = 7 | issue = 1| page = Article 26 | doi=10.2202/1544-6115.1389| pmid = 18764775 | s2cid = 38232110 }}</ref>
Line 390 ⟶ 389:
<ref name="Toni">{{cite journal | last1 = Toni | first1 = T | last2 = Welch | first2 = D | last3 = Strelkowa | first3 = N | last4 = Ipsen | first4 = A | last5 = Stumpf | first5 = M | year = 2007 | title = Approximate Bayesian computation scheme for parameter inference and model selection in dynamical systems | journal = J R Soc Interface | volume = 6 | issue = 31| pages = 187–202 | pmid = 19205079 | pmc = 2658655 | doi = 10.1098/rsif.2008.0172 }}</ref>
<ref name="Tavare">{{cite journal | last1 = Tavaré | first1 = S | last2 = Balding | first2 = DJ | last3 = Griffiths | first3 = RC | last4 = Donnelly | first4 = P | year = 1997 | title = Inferring Coalescence Times from DNA Sequence Data | journal = Genetics | volume = 145 | issue = 2 | pages = 505–518 | doi = 10.1093/genetics/145.2.505 | pmc = 1207814 | pmid=9071603}}</ref>
<ref name="Toni2010">{{cite journal | doi=10.1093/bioinformatics/btp619 | title=Simulation-based model selection for dynamical systems in systems and population biology | date=2010 | last1=Toni | first1=Tina | last2=Stumpf | first2=Michael P. H. | journal=Bioinformatics | volume=26 | issue=1 | pages=104–110 | pmid=19880371 | pmc=2796821 | arxiv=0911.1705 }}</ref>
.<ref name="Pritchard1999">{{cite journal | last1 = Pritchard | first1 = JK | last2 = Seielstad | first2 = MT | last3 = Perez-Lezaun | first3 = A |display-authors=et al | year = 1999 | title = Population Growth of Human Y Chromosomes: A Study of Y Chromosome Microsatellites | journal = Molecular Biology and Evolution | volume = 16 | issue = 12| pages = 1791–1798 | doi=10.1093/oxfordjournals.molbev.a026091| pmid = 10605120 | doi-access = free }}</ref>
<ref name="Diggle">{{cite journal | last1 = Diggle | first1 = PJ | year = 1984 | title = Monte Carlo Methods of Inference for Implicit Statistical Models | journal = Journal of the Royal Statistical Society, Series B | volume = 46 | issue = 2 | pages = 193–227 | doi = 10.1111/j.2517-6161.1984.tb01290.x }}</ref>
Line 413 ⟶ 412:
<ref name="Kangas16">{{cite journal |last1= Kangasrääsiö |first1= Antti |last2= Lintusaari |first2= Jarno |last3= Skytén |first3= Kusti |last4= Järvenpää |first4= Marko |last5= Vuollekoski |first5= Henri |last6= Gutmann |first6= Michael |last7= Vehtari |first7= Aki |last8= Corander |first8= Jukka |last9= Kaski |first9= Samuel|year= 2016 |title= ELFI: Engine for Likelihood-Free Inference |url=http://approximateinference.org/accepted/KangasraasioEtAl2016.pdf |journal= NIPS 2016 Workshop on Advances in Approximate Bayesian Inference|bibcode= 2017arXiv170800707L |arxiv= 1708.00707 }}</ref>
<ref name="Klinger2017">Klinger, E.; Rickert, D.; Hasenauer, J. (2017). pyABC: distributed, likelihood-free inference.</ref>
<ref name="Salvatier2016">{{cite journal | doi=10.7717/peerj-cs.55 | doi-access=free | title=Probabilistic programming in Python using PyMC3 | date=2016 | last1=Salvatier | first1=John | last2=Wiecki | first2=Thomas V. | last3=Fonnesbeck | first3=Christopher | journal=PeerJ Computer Science | volume=2 | pages=e55 | arxiv=1507.08050 }}</ref>
<ref name="Prangle">{{cite journal | doi=10.1515/sagmb-2013-0012 | title=Semi-automatic selection of summary statistics for ABC model choice| date=2014 | last1=Prangle | first1=Dennis | last2=Fearnhead | first2=Paul | last3=Cox | first3=Murray P. | last4=Biggs | first4=Patrick J. | last5=French | first5=Nigel P. | journal=Stat Appl Genet Mol Biol | volume=13| issue=1| pages=67–82 | pmid=24323893| arxiv=1302.5624 }}</ref>
}}
|