Wikipedia

Probabilistic programming: Difference between revisions

Browse history interactively
← Previous edit
Content deleted Content added
VisualWikitext
Copy lead from Probabilistic logic program as summary for this page.
OAbot (talk | contribs)
Bots
643,717 edits
m Open access bot: url-access updated in citation with #oabot.
 
(15 intermediate revisions by 13 users not shown)
Line 1:
{{Short description|Software system for statistical models}}
{{Use mdy dates|date=September 2015}}
{{primary sources|date=December 2014}}
 
'''Probabilistic programming''' ('''PP''') is a [[programming paradigm]] inbased whichon the declarative specification of [[probabilistic model]]s are specified and inference, for thesewhich modelsinference is performed automatically.<ref name=physorg>{{cite news
| url=http://phys.org/news/2015-04-probabilistic-lines-code-thousands.html
| title=Probabilistic programming does in 50 lines of code what used to take thousands
Line 8 ⟶ 9:
| date=April 13, 2015
| access-date=2015-04-13 }}
</ref> ItProbabilistic represents anprogramming attemptattempts to unify probabilistic modeling and traditional general purpose programming in order to make the former easier and more widely applicable.<ref>{{cite web|url=http://probabilistic-programming.org/wiki/Home|title=Probabilistic Programming|work=probabilistic-programming.org|access-date=December 24, 2013|archive-url=https://web.archive.org/web/20160110035042/http://probabilistic-programming.org/wiki/Home|archive-date=January 10, 2016|url-status=dead}}</ref><ref name="Pfeffer2014">Pfeffer, Avrom (2014), ''Practical Probabilistic Programming'', Manning Publications. p.28. {{ISBN|978-1 6172-9233-0}}</ref> It can be used to create systems that help make decisions in the face of uncertainty. Programming languages following the probabilistic programming paradigm are referred to as "probabilistic programming languages" (PPLs).
 
Programming languages used for probabilistic programming are referred to as "probabilistic programming languages" (PPLs).
 
== Applications ==
Line 19 ⟶ 18:
The [[Gen (probabilistic programming)|Gen]] probabilistic programming library (also written in Julia) has been applied to vision and robotics tasks.<ref>{{Cite web|url=https://venturebeat.com/2019/06/27/mits-gen-programming-system-allows-users-to-easily-create-computer-vision-statistical-ai-and-robotics-programs/|title=MIT's Gen programming system flattens the learning curve for AI projects|date=2019-06-27|website=VentureBeat|language=en-US|access-date=2019-06-27}}</ref>
 
More recently, the probabilistic programming system [[Turing (probabilistic programming)|Turing.jl]] has been applied in various pharmaceutical<ref>{{Cite journal|last1=Semenova|first1=Elizaveta|last2=Williams|first2=Dominic P.|last3=Afzal|first3=Avid M.|last4=Lazic|first4=Stanley E.|date=2020-11-01|title=A Bayesian neural network for toxicity prediction|url=https://www.sciencedirect.com/science/article/pii/S2468111320300438|journal=Computational Toxicology|language=en|volume=16|pages=100133|doi=10.1016/j.comtox.2020.100133|s2cid=225362130|issn=2468-1113|url-access=subscription}}</ref> and economics applications.<ref name="pharma-turing">{{Citation|title= Predicting Drug-Induced Liver Injury with Bayesian Machine Learning |year= 2020 |doi= 10.1021/acs.chemrestox.9b00264 |url= https://pubs.acs.org/doi/10.1021/acs.chemrestox.9b00264|last1= Williams |first1= Dominic P. |last2= Lazic |first2= Stanley E. |last3= Foster |first3= Alison J. |last4= Semenova |first4= Elizaveta |last5= Morgan |first5= Paul |journal= Chemical Research in Toxicology |volume= 33 |issue= 1 |pages= 239–248 |pmid= 31535850 |s2cid= 202689667 |url-access= subscription }}</ref>
 
Probabilistic programming in Julia has also been combined with [[differentiable programming]] by combining the Julia package Zygote.jl with Turing.jl. <ref name="diffprog-zygote">{{cite arXiv|date=2019|title=∂P: A Differentiable Programming System to Bridge Machine Learning and Scientific Computing|eprint=1907.07587|last1=Innes|first1=Mike|last2=Edelman|first2=Alan|last3=Fischer|first3=Keno|last4=Rackauckas|first4=Chris|last5=Saba|first5=Elliot|author6=Viral B Shah|last7=Tebbutt|first7=Will|class=cs.PL }}</ref>
Line 26 ⟶ 25:
 
== Probabilistic programming languages ==
PPLs often extend from a basic language. The choice of underlying basic language depends on the similarity of the model to the basic language's [[Ontology (information science)|ontology]], as well as commercial considerations and personal preference. For instance, DimpleTuring.jl<ref name="DMPLTuringjl"/> and Chimple<ref name="CHMPL"/> areis based on [[JavaJulia (programming language)|JavaJulia]], [[Infer.NET]] is based on [[.NET Framework]],<ref name="INFET"/> while PRISM extends from [[Prolog]].<ref name="PRISM"/> However, some PPLs, such as [[WinBUGS]], offer a self-contained language, that maps closely to the mathematical representation of the statistical models, with no obvious origin in another programming language.<ref name="BUGS"/><ref name="Stan"/>
 
The language for winBUGSWinBUGS was implemented to perform Bayesian computation using Gibbs Sampling (and related algorithms). Although implemented in a relatively unknown programming language (Component Pascal), this language permits [[Bayesian_inference|Bayesian inference]] for a wide variety of statistical models using a flexible computational approach. The same BUGS language may be used to specify Bayesian models for inference via different computational choices ("samplers") and conventions or defaults, using a standalone packageprogram winBUGSWinBUGS (or related R packages, rbugs and r2winbugs) and JAGS (Just Another Gibbs Sampler, another standalone program with related R packagepackages including rjags, R2jags, and runjags). More recently, other languages to support Bayesian model specification and inference allow different or more efficient choices for the underlying Bayesian computation, and are accessible from the R data analysis and programming environment, e.g.: [[Stan (software)|Stan]], NIMBLE and NUTS. The influence of the BUGS language is evident in these later languages, which even use the same syntax for some aspects of model specification.
 
Several PPLs are in active development, including some in beta test. Two popular tools are Stan and [[PyMC]].<ref>{{Cite web|url=http://blog.fastforwardlabs.com/2017/01/30/the-algorithms-behind-probabilistic-programming.html|title=The Algorithms Behind Probabilistic Programming|access-date=2017-03-10}}</ref>
Line 55 ⟶ 54:
|-
| Bean Machine<ref name="beanmachine"/> || |[[PyTorch]] || [[Python (programming language)|Python]]
|-▼
| CuPPL<ref name = "CuPPL"/> || NOVA<ref name="nova"/> ||
|-
| Venture<ref name="Venture"/> || [[Scheme (programming language)|Scheme]] || C++
|-
| Probabilistic-C<ref name="Probabilistic-C"/> || [[C (programming language)|C]] || C
|-
| Anglican<ref name="Anglican"/> || [[Clojure]] || Clojure
|-
| IBAL<ref name="IBAL"/> || [[OCaml]] ||
|-
| BayesDB<ref name="BAYESDB"/> || [[SQLite]], [[Python (programming language)|Python]] ||
Line 71 ⟶ 62:
|-
| [[Infer.NET]]<ref name="INFET"/> || .NET Framework || .NET Framework
|-
| dimple<ref name="DMPL"/> || [[MATLAB]], [[Java (programming language)|Java]] ||
|-
| chimple<ref name="CHMPL"/> || MATLAB, Java ||
|-
| BLOG<ref name="BLOG"/> || Java ||
|-
| diff-SAT<ref name="diff-SAT"/> || [[Answer set programming]], [[Boolean satisfiability problem|SAT (DIMACS CNF)]] ||
Line 84 ⟶ 69:
| BUGS<ref name="BUGS"/> || ||Component Pascal
|-
| FACTORIE<ref name="FACTORIE"/> || [[Scala (programming language)|Scala]] ||Scala
|-
| PMTK<ref name="PMTK"/> || MATLAB || MATLAB
|-
| Alchemy<ref name="Alchemy"/> || [[C++]] ||
|-
| Dyna<ref name="Dyna"/> || [[Prolog]] ||
|-
| Figaro<ref name="Figaro"/> || Scala ||Scala
|-
| [[Church (programming language)|Church]]<ref name="Church"/> || Scheme || Various: JavaScript, Scheme
|-
| [[ProbLog]]<ref name="ProbLog"/> || Prolog ||Python
Line 108 ⟶ 85:
| ProbCog<ref name="ProbCog"/> || || Java, Python
|-
|-Pomegranate || Python || Cython▼
| Gamble<ref name="Gamble"/> || || Racket
|-
|-Infergo|| Go || Go
| PWhile<ref name="PWhile"/> || While || Python
|-
| Tuffy<ref name="Tuffy"/> || || Java
|-
▲|-Pomegranate || Python || Cython
|-
| [[PyMC3|PyMC]]<ref name="PyMC"/> || Python|| Python
Line 132 ⟶ 105:
|-
| WebPPL<ref name="WebPPL"/>||JavaScript||JavaScript
|-
|Let's Chance<ref>{{Cite book|title=Let's Chance: Playful Probabilistic Programming for Children {{!}} Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems|url=https://dl.acm.org/doi/abs/10.1145/3334480.3383071|access-date=2020-08-01|website=dl.acm.org|series=Chi Ea '20|date=April 25, 2020|pages=1–7|language=EN|doi=10.1145/3334480.3383071|isbn=9781450368193|s2cid=216079395}}</ref>
|Scratch
|JavaScript
|-
| Picture<ref name="kurzweilai"/>
Line 145 ⟶ 114:
| Julia
|-
| Gen<ref>{{Cite web|url=https://probcompwww.githubgen.io/Gendev/|title=Gen: A General Purpose Probabilistic Programming Language with Programmable Inference|access-date=20192024-06-1711}}</ref>
| [[Julia (programming language)|Julia]]
| [[Julia (programming language)|Julia]]
|-
|-
| Low-level First-order PPL<ref name="LFPPL"/> || Python, Clojure, Pytorch || Various: Python, Clojure
|-
|Troll<ref name="Troll"/>
|
|Moscow ML
|-
|Edward<ref>{{Cite web|url=http://edwardlib.org/|title=Edward – Home|website=edwardlib.org|access-date=2017-01-17}}</ref>
Line 164 ⟶ 126:
|Python
|-
|Edward2<ref>{{Cite web|url=https://github.com/tensorflow/probability/tree/master/tensorflow_probability/pythongoogle/edward2|title='Edward2' TensorFlow Probability module|website=GitHub|language=en|access-date=20182024-1006-0211}}</ref>
|TensorFlow Probability
|Python
Line 175 ⟶ 137:
|[[Google JAX|JAX]]
|Python
|-
|Saul<ref>{{Cite web | url= https://cogcomp.org/page/software_view/Saul|title=CogComp - Home}}</ref>
|Scala
|Scala
|-
|RankPL<ref>{{Citation|last=Rienstra|first=Tjitze|title=RankPL: A qualitative probabilistic programming language based on ranking theory|date=2018-01-18|url=https://github.com/tjitze/RankPL|access-date=2018-01-18}}</ref>
|
|Java
|-
|Birch<ref>{{Cite web|url=http://birch-lang.org/|title=Probabilistic Programming in Birch|website=birch-lang.org|access-date=2018-04-20}}</ref>
Line 199 ⟶ 153:
|Python
|Python
▲|-
|Anglican<ref>{{Cite web |title=The Anglican Probabilistic Programming System |url=https://probprog.github.io/anglican/ |access-date=2024-10-07 |website=probprog.github.io}}</ref>
|[[Clojure]]
|[[Clojure]]
|}
 
== Difficulty ==
 
* Reasoning about variables as probability distributions causes difficulties for novice programmers, but these difficulties can be addressed through use of Bayesian network visualisationsvisualizations and graphs of variable distributions embedded within the source code editor.<ref>{{Cite book|last1=Gorinova|first1=Maria I.|last2=Sarkar|first2=Advait|last3=Blackwell|first3=Alan F.|last4=Syme|first4=Don|title=Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems |chapter=A Live, Multiple-Representation Probabilistic Programming Environment for Novices |date=2016-01-01|series=CHI '16|___location=New York, NY, USA|publisher=ACM|pages=2533–2537|doi=10.1145/2858036.2858221|isbn=9781450333627|s2cid=3201542}}</ref>
* As many PPLs rely on the specification of priors on the variables of interest, specifying informed priors is often difficult for novices. In some cases, libraries such as PyMC provide automated methods to find the parameterization of informed priors.<ref>{{Cite web |title=pymc.find_constrained_prior — PyMC dev documentation |url=https://www.pymc.io/projects/docs/en/latest/api/generated/pymc.find_constrained_prior.html |access-date=2024-10-23 |website=www.pymc.io}}</ref>
 
==See also==
Line 212 ⟶ 172:
==Notes==
{{Reflist|30em|refs=
<ref name="nova">{{cite book|url=https://dl.acm.org/citation.cfm?id=2627375|work=acm.org|series=Array'14|date=June 9, 2014|pages=8–13|doi=10.1145/2627373.2627375|isbn=9781450329378|s2cid=6748967 |chapter=NOVA: A Functional Language for Data Parallelism |title=Proceedings of ACM SIGPLAN International Workshop on Libraries, Languages, and Compilers for Array Programming |last1=Collins |first1=Alexander |last2=Grewe |first2=Dominik |last3=Grover |first3=Vinod |last4=Lee |first4=Sean |last5=Susnea |first5=Adriana }}</ref>
<ref name="CuPPL">{{cite web|url=https://popl19.sigplan.org/event/lafi-2019-probabilistic-programming-with-cuppl|title=Probabilistic Programming with CuPPL|work=popl19.sigplan.org}}</ref>
<ref name="BAYESDB">{{cite web|url=https://github.com/probcomp/bayeslite|title=BayesDB on SQLite. A Bayesian database table for querying the probable implications of data as easily as SQL databases query the data itself|work=GitHub|date=December 26, 2021}}</ref>
<ref name="bayesloop">{{Cite web|url=http://bayesloop.com/|title=bayesloop - Probabilistic programming framework|website=bayesloop.com}}</ref>
Line 219 ⟶ 177:
<ref name="Analytica">{{cite web|url=http://www.analytica.com|title=Analytica-- A Probabilistic Modeling Language|work=lumina.com}}</ref>
<ref name="Venture">{{cite web|url=http://probcomp.csail.mit.edu/venture/|title=Venture -- a general-purpose probabilistic programming platform|work=mit.edu|access-date=September 20, 2014|archive-url=https://web.archive.org/web/20160125130827/http://probcomp.csail.mit.edu/venture/|archive-date=January 25, 2016|url-status=dead}}</ref>
<ref name="Anglican">{{cite web|url=https://github.com/probprog/anglican-infcomp|title=The Anglican Probabilistic Programming System|work=ox.ac.uk|date=January 6, 2021}}</ref>
<ref name="Probabilistic-C">{{cite web|url=http://www.robots.ox.ac.uk/~brooks/probabilistic-c/|title=Probabilistic C|work=ox.ac.uk|access-date=March 24, 2015|archive-url=https://web.archive.org/web/20160104201746/http://www.robots.ox.ac.uk/~brooks/probabilistic-c/|archive-date=January 4, 2016|url-status=dead}}</ref>
<ref name="IBAL">{{cite web|url=http://www.eecs.harvard.edu/~avi/IBAL/|title=IBAL Home Page|url-status=dead|archive-url=https://web.archive.org/web/20101226131239/http://www.eecs.harvard.edu/~avi/IBAL/|archive-date=December 26, 2010|df=mdy-all}}</ref>
<ref name="PRISM">{{cite web|url=http://rjida.meijo-u.ac.jp/prism/|title=PRISM: PRogramming In Statistical Modeling|website=rjida.meijo-u.ac.jp|access-date=July 8, 2015|archive-url=https://web.archive.org/web/20150301155729/http://rjida.meijo-u.ac.jp/prism/|archive-date=March 1, 2015|url-status=dead}}</ref>
<ref name="INFET">{{cite web|url=http://research.microsoft.com/en-us/um/cambridge/projects/infernet/|title=Infer.NET|publisher=Microsoft|work=microsoft.com}}</ref>
<ref name="DMPL">{{cite web|url=https://github.com/analog-garage/dimple|title=Dimple Home Page|work=analog.com|date=July 2, 2021}}</ref>
<ref name="CHMPL">{{cite web|url=https://github.com/analog-garage/chimple|title=Chimple Home Page|work=analog.com|date=April 16, 2021}}</ref>
<ref name="BLOG">{{cite web|url=http://people.csail.mit.edu/milch/blog/|title=Bayesian Logic (BLOG)|work=mit.edu|url-status=dead|archive-url=https://web.archive.org/web/20110616214423/http://people.csail.mit.edu/milch/blog/|archive-date=June 16, 2011|df=mdy-all}}</ref>
<ref name="diff-SAT">{{cite web|url=https://github.com/MatthiasNickles/diff-SAT/|title=diff-SAT (probabilistic SAT/ASP)|website=[[GitHub]]|date=October 8, 2021}}</ref>
<ref name="PSQL">{{cite journal|title=PSQL: A query language for probabilistic relational data|doi=10.1016/S0169-023X(98)00015-9 | volume=28|journal=Data & Knowledge Engineering|pages=107–120|year = 1998|last1 = Dey|first1 = Debabrata|last2=Sarkar |first2=Sumit }}</ref>
<ref name="BUGS">{{cite web|url=http://www.mrc-bsu.cam.ac.uk/bugs/|title=The BUGS Project - MRC Biostatistics Unit|work=cam.ac.uk|access-date=January 12, 2011|archive-url=https://web.archive.org/web/20140314080841/http://www.mrc-bsu.cam.ac.uk/bugs/|archive-date=March 14, 2014|url-status=dead}}</ref>
<ref name="FACTORIE">{{cite web|url=http://code.google.com/p/factorie/|title=Factorie - Probabilistic programming with imperatively-defined factor graphs - Google Project Hosting|work=google.com}}</ref>
<ref name="PMTK">{{cite web|url=http://code.google.com/p/pmtk3/|title=PMTK3 - probabilistic modeling toolkit for Matlab/Octave, version 3 - Google Project Hosting|work=google.com}}</ref>
<ref name="Gamble">{{cite web|url=https://github.com/rmculpepper/gamble|title=gamble: Probabilistic Programming|first=Ryan|last=Culpepper|date=January 17, 2017|via=GitHub}}</ref>
<ref name="Alchemy">{{cite web|url=http://alchemy.cs.washington.edu/|title=Alchemy - Open Source AI|work=washington.edu}}</ref>
<ref name="Dyna">{{cite web|url=http://www.dyna.org/|title=Dyna|website=www.dyna.org|access-date=January 12, 2011|archive-url=https://web.archive.org/web/20160117155947/http://dyna.org/|archive-date=January 17, 2016|url-status=dead}}</ref>
<ref name="Figaro">{{cite web|url=http://www.cra.com/figaro|title=Charles River Analytics - Probabilistic Modeling Services|work=cra.com|date=February 9, 2017}}</ref>
<ref name="Church">{{cite web|url=http://projects.csail.mit.edu/church/wiki/Church|title=Church|work=mit.edu|access-date=April 8, 2013|archive-url=https://web.archive.org/web/20160114182510/http://projects.csail.mit.edu/church/wiki/Church|archive-date=January 14, 2016|url-status=dead}}</ref>
<ref name="ProbLog">{{cite web|url=http://dtai.cs.kuleuven.be/problog|title=ProbLog: Probabilistic Programming|website=dtai.cs.kuleuven.be}}</ref>
<ref name="ProBT">{{cite web|url=http://www.probayes.com/fr/Bayesian-Programming-Book/downloads/|title=ProbaYes - Ensemble, nous valorisations vos données|author=ProbaYes|work=probayes.com|access-date=November 26, 2013|archive-url=https://web.archive.org/web/20160305000751/http://www.probayes.com/fr/Bayesian-Programming-Book/downloads/|archive-date=March 5, 2016|url-status=dead}}</ref>
Line 243 ⟶ 190:
<ref name="BAli-Phy">{{cite web|url=http://www.bali-phy.org/|title=BAli-Phy Home Page|work=bali-phy.org}}</ref>
<ref name="ProbCog">{{cite web|url=https://github.com/opcode81/ProbCog/wiki/Features|title=ProbCog|work=GitHub}}</ref>
<ref name="Tuffy">{{cite web|url=http://i.stanford.edu/hazy/tuffy/home|title=Tuffy: A Scalable Markov Logic Inference Engine|work=stanford.edu}}</ref>
<ref name="PyMC">{{cite web|url=https://docs.pymc.io/en/v3/|title=PyMC|author=PyMC devs|work=pymc-devs.github.io}}</ref>
<ref name="Lea">{{cite web|url=https://bitbucket.org/piedenis/lea|title=Lea Home Page|work=bitbucket.org}}</ref>
<ref name="WebPPL">{{cite web|url=http://dippl.org/|title=WebPPL Home Page|work=github.com/probmods/webppl}}</ref>
<ref name="Turingjl">{{cite web|url=https://github.com/yebai/Turing.jl|title=The Turing language for probabilistic programming|website=[[GitHub]]|date=December 28, 2021}}</ref>
<ref name="Troll">{{Cite web|url=http://topps.diku.dk/torbenm/troll.msp|title=Troll dice roller and probability calculator|website=topps.diku.dk}}</ref>
<ref name="PWhile">{{cite web|url=https://github.com/zz5013/pwCompiler|title=PWhile Compiler|work=GitHub|date=May 25, 2020}}</ref>
<ref name="LFPPL">{{cite web|url=https://github.com/bradleygramhansen/PyLFPPL|title=LF-PPL: A Low-Level First Order Probabilistic Programming Language for Non-Differentiable Models|work=ox.ac.uk|date=November 2, 2019}}</ref>
<ref name="beanmachine">{{cite web|url=https://beanmachine.org|title=Bean Machine - A universal probabilistic programming language to enable fast and accurate Bayesian analysis|work=beanmachine.org}}</ref>
}}
 
== External links ==
*[https://www.cambridge.org/core/books/foundations-of-probabilistic-programming/819623B1B5B33836476618AC0621F0EE Foundations of Probabilistic Programming]
*[http://anyall.org/blog/2009/12/list-of-probabilistic-model-mini-language-toolkits/ List of Probabilistic Model Mini Language Toolkits]
*[https://web.archive.org/web/20160110035042/http://probabilistic-programming.org/wiki/Home Probabilistic programming wiki]
 
{{Programming paradigms navbox}}
 
Retrieved from "https://en.wikipedia.org/wiki/Probabilistic_programming"