Differentiable programming: Difference between revisions

Revision as of 08:56, 19 October 2022

Revision as of 08:58, 19 October 2022

Line 3:

'''Differentiable programming''' is a [[programming paradigm]] in which a numeric computer program can be [[Differentiation (mathematics)|differentiated]] throughout via [[automatic differentiation]].<ref name="izzo2016_dCGP">{{cite journal|last1=Izzo|first1=Dario|last2=Biscani|first2=Francesco|last3=Mereta|first3=Alessio|title=Differentiable genetic programming|journal=European conference on genetic programming (EuroGP)|year=2017|volume=18|pages=35-51|url=https://link.springer.com/chapter/10.1007/978-3-319-55696-3_3}}</ref>

<ref name="baydin2018automatic">{{cite journal|last1=Baydin|first1=Atilim Gunes|last2=Pearlmutter|first2=Barak|last3=Radul|first3=Alexey Andreyevich|last4=Siskind|first4=Jeffrey|title=Automatic differentiation in machine learning: a survey|journal=Journal of Machine Learning Research|year=2018|volume=18|pages=1–43|url=http://jmlr.org/papers/v18/17-468.html}}</ref><ref>{{Citation|last1=Wang|first1=Fei|title=Backpropagation with Callbacks: Foundations for Efficient and Expressive Differentiable Programming|date=2018|url=http://papers.nips.cc/paper/8221-backpropagation-with-callbacks-foundations-for-efficient-and-expressive-differentiable-programming.pdf|work=Advances in Neural Information Processing Systems 31|pages=10201–10212|editor-last=Bengio|editor-first=S.|publisher=Curran Associates, Inc.|access-date=2019-02-13|last2=Decker|first2=James|last3=Wu|first3=Xilun|last4=Essertel|first4=Gregory|last5=Rompf|first5=Tiark|editor2-last=Wallach|editor2-first=H.|editor3-last=Larochelle|editor3-first=H.|editor4-last=Grauman|editor4-first=K.}}</ref><ref name="innes">{{Cite journal|last=Innes|first=Mike|date=2018|title=On Machine Learning and Programming Languages|url=http://www.sysml.cc/doc/2018/37.pdf|journal=SysML Conference 2018}}</ref><ref name="diffprog-zygote">{{Citation|date=2019|title=∂P: A Differentiable Programming System to Bridge Machine Learning and Scientific Computing|arxiv=1907.07587|last1=Innes|first1=Mike|last2=Edelman|first2=Alan|last3=Fischer|first3=Keno|last4=Rackauckas|first4=Chris|last5=Saba|first5=Elliot|author6=Viral B Shah|last7=Tebbutt|first7=Will}}</ref> This allows for [[Gradient method|gradient-based optimization]] of parameters in the program, often via [[gradient descent]], as well as other learning approaches that are based on higher order derivative information<ref name="izzo2016_dCGP">{{cite journal|last1=Izzo|first1=Dario|last2=Biscani|first2=Francesco|last3=Mereta|first3=Alessio|title=Differentiable genetic programming|journal=European conference on genetic programming (EuroGP)|year=2017|volume=18|pages=35-51|url=https://link.springer.com/chapter/10.1007/978-3-319-55696-3_3}}</ref>. Differentiable programming has found use in a wide variety of areas, particularly [[scientific computing]] and [[artificial intelligence]].<ref name="diffprog-zygote" />. One of the early proposals to adopt such a framework in a systematic fashion to improve upon learning algorithms was made by the [[Advanced_Concepts_Team]] at the [[European_Space_Agency]] in early 2016.<ref name="differential intelligence">{{Cite web|url=https://www.esa.int/gsp/ACT/projects/differential_intelligence/|title=Differential Intelligence|access-date=2022-10-19}}</ref>

==Approaches==