Differentiable programming: Difference between revisions

Revision as of 17:09, 5 March 2019

Revision as of 22:49, 1 May 2019

Line 9:

* '''[[Operator overloading]], dynamic graph''' based approaches such as [[PyTorch]] and [[AutoGrad (NumPy)|AutoGrad]]. Their dynamic and interactive nature lets most programs be written and reasoned about more easily. However, they lead to [[interpreter (computing)|interpreter]] overhead (particularly when composing many small operations), poorer scalability, and cannot gain benefit from compiler optimization.<ref name="myia1" /><ref name="pytorchtut" />

Both of these earlierearly attemptsapproaches are also generally only able to differentiate code written in a suitable manner for the framework, limiting their interoperability with other programs.

A more recent framework in the [[Julia (programming language)|Julia]] programming language — called Zygote — resolves the issues that earlier attempts faced by treating the language's syntax as the graph; the [[intermediate representation]] for arbitrary Julia code can then be differentiated directly, [[compiler optimization|optimized]], and compiled.<ref name="flux" /><ref>{{cite arxiv|last=Innes|first=Michael|date=2018-10-18|title=Don't Unroll Adjoint: Differentiating SSA-Form Programs|eprint=1810.07951|class=cs.PL}}</ref> An in-development differentiable programming language called [[Myia (programming language)|Myia]] also uses a similar approach.<ref name="myia1" />