Differential dynamic programming: Difference between revisions

Revision as of 04:45, 15 August 2020

Revision as of 14:41, 22 October 2020

Line 172:

== Monte Carlo version ==

Sampled differential dynamic programming (SaDDP) is a Monte Carlo variant of differential dynamic programming.<ref>{{Cite document|title=Sampled differential dynamic programming - IEEE Conference Publication|language=en-US|doi=10.1109/IROS.2016.7759229|s2cid=1338737}}</ref><ref>{{Cite web|url=https://ieeexplore.ieee.org/document/8430799|title=Regularizing Sampled Differential Dynamic Programming - IEEE Conference Publication|website=ieeexplore.ieee.org|language=en-US|access-date=2018-10-19}}</ref><ref>{{Cite book|last=Joose|first=Rajamäki|date=2018|title=Random Search Algorithms for Optimal Control|url=http://urn.fi/URN:ISBN:978-952-60-8156-4|language=en|issn=1799-4942|isbn=9789526081564|publisher=Aalto University}}</ref> It is based on treating the quadratic cost of differential dynamic programming as the energy of a [[Boltzmann distribution]]. This way the quantities of DDP can be matched to the statistics of a [[Multivariate normal distribution|multidimensional normal distribution]]. The statistics can be recomputed from sampled trajectories without differentiation.

Sampled differential dynamic programming has been extended to Path Integral Policy Improvement with Differential Dynamic Programming.<ref>{{Cite journal|lastlast1=Lefebvre|firstfirst1=Tom|last2=Crevecoeur|first2=Guillaume|date=July 2019|title=Path Integral Policy Improvement with Differential Dynamic Programming|url=https://ieeexplore.ieee.org/abstract/document/8868359|journal=2019 IEEE/ASME International Conference on Advanced Intelligent Mechatronics (AIM)|pages=739–745|doi=10.1109/AIM.2019.8868359|hdl=1854/LU-8623968|isbn=978-1-7281-2493-3|s2cid=204816072|hdl-access=free}}</ref> This creates a link between differential dynamic programming and path integral control,<ref>{{Cite journal|lastlast1=Theodorou|firstfirst1=Evangelos|last2=Buchli|first2=Jonas|last3=Schaal|first3=Stefan|date=May 2010|title=Reinforcement learning of motor skills in high dimensions: A path integral approach|url=https://ieeexplore.ieee.org/document/5509336|journal=2010 IEEE International Conference on Robotics and Automation|pages=2397–2403|doi=10.1109/ROBOT.2010.5509336|isbn=978-1-4244-5038-1|s2cid=15116370}}</ref> which is a framework of stochastic optimal control.

== Constrained problems ==

Interior Point Differential dynamic programming (IPDDP) is an [[interior-point method]] generalization of DDP that can address the optimal control problem with nonlinear state and input constraints. <ref>{{cite arXiv |lastlast1=Pavlov |firstfirst1=Andrei|last2=Shames|first2=Iman| last3=Manzie|first3=Chris|date=2020 |title=Interior Point Differential Dynamic Programming |eprint=2004.12710 |class=math.OC}}</ref>

== See also ==