Differentiable programming: Difference between revisions

Revision as of 06:54, 1 June 2023

Revision as of 07:06, 1 June 2023

Line 2:

{{Programming paradigms}}

'''Differentiable programming''' is a [[programming paradigm]] in which a numeric computer program can be [[Differentiation (mathematics)|differentiated]] throughout via [[automatic differentiation]].<ref name="izzo2016_dCGP">{{cite journal|last1=Izzo|first1=Dario|last2=Biscani|first2=Francesco|last3=Mereta|first3=Alessio|title=Differentiable genetic programming|journal=European Conference on Genetic Programming (EuroGP)|series=Lecture Notes in Computer Science |year=2017|volume=18|pages=35–51|doi=10.1007/978-3-319-55696-3_3 |arxiv=1611.04766 |isbn=978-3-319-55695-6 |s2cid=17786263 |url=https://link.springer.com/chapter/10.1007/978-3-319-55696-3_3}}</ref>

<ref name="baydin2018automatic">{{cite journal|last1=Baydin|first1=Atilim Gunes|last2=Pearlmutter|first2=Barak|last3=Radul|first3=Alexey Andreyevich|last4=Siskind|first4=Jeffrey|title=Automatic differentiation in machine learning: a survey|journal=Journal of Machine Learning Research |year=2018 |volume=18|pages=1–43|url=http://jmlr.org/papers/v18/17-468.html}}</ref><ref>{{cite book |last1=Wang |first1=Fei |chapter=Backpropagation with Callbacks: Foundations for Efficient and Expressive Differentiable Programming |date=2018 |chapter-url=http://papers.nips.cc/paper/8221-backpropagation-with-callbacks-foundations-for-efficient-and-expressive-differentiable-programming.pdf |title=NIPS'18: Proceedings of the 32nd International Conference on Neural Information Processing Systems |pages=10201–12 |access-date=2019-02-13 |last2=Decker |first2=James |last3=Wu |first3=Xilun |last4=Essertel |first4=Gregory |last5=Rompf |first5=Tiark |editor-last=Bengio |editor-first=S. |editor2-last=Wallach |editor2-first=H. |editor3-last=Larochelle |editor3-first=H. |editor4-last=Grauman |editor4-first=K |publisher=Curran Associates |url=https://dl.acm.org/doi/proceedings/10.5555/3327546 |ref={{harvid|NIPS'18}} }}</ref><ref name="innes">{{Cite journal|last=Innes|first=Mike|date=2018|title=On Machine Learning and Programming Languages|url=http://www.sysml.cc/doc/2018/37.pdf|journal=SysML Conference 2018}}</ref><ref name="diffprog-zygote">{{Citation|date=2019|title=∂P: A Differentiable Programming System to Bridge Machine Learning and Scientific Computing|arxiv=1907.07587|last1=Innes|first1=Mike|last2=Edelman|first2=Alan|last3=Fischer|first3=Keno|last4=Rackauckas|first4=Chris|last5=Saba|first5=Elliot|author6=Viral B Shah|last7=Tebbutt|first7=Will}}</ref> This allows for [[Gradient method|gradient-based optimization]] of parameters in the program, often via [[gradient descent]], as well as other learning approaches that are based on higher order derivative information. Differentiable programming has found use in a wide variety of areas, particularly [[scientific computing]] and [[artificial intelligence]].<ref name="diffprog-zygote" /> One of the early proposals to adopt such a framework in a systematic fashion to improve upon learning algorithms was made by the [[Advanced Concepts Team]] at the [[European Space Agency]] in early 2016.<ref name="differential intelligence">{{Cite web|url=https://www.esa.int/gsp/ACT/projects/differential_intelligence/|title=Differential Intelligence|date=October 2016 |access-date=2022-10-19}}</ref>

==Approaches==