Levenberg–Marquardt algorithm: Difference between revisions

Revision as of 01:46, 21 May 2022

Latest revision as of 07:50, 26 April 2024

(4 intermediate revisions by 4 users not shown)

Line 31:

&= \left [\mathbf y - \mathbf f\left (\boldsymbol\beta\right )\right ]^{\mathrm T}\left [\mathbf y - \mathbf f\left (\boldsymbol\beta\right )\right ] - 2\left [\mathbf y - \mathbf f\left (\boldsymbol\beta\right )\right ]^{\mathrm T} \mathbf J \boldsymbol\delta + \boldsymbol\delta^{\mathrm T} \mathbf J^{\mathrm T} \mathbf J\boldsymbol\delta.

\end{align}</math>

Taking the derivative of this approximation of <math>S\left (\boldsymbol\beta + \boldsymbol\delta\right )</math> with respect to {{tmath|\boldsymbol\delta}} and setting the result to zero gives

:<math>\left (\mathbf J^{\mathrm T} \mathbf J\right )\boldsymbol\delta = \mathbf J^{\mathrm T}\left [\mathbf y - \mathbf f\left (\boldsymbol\beta\right )\right ],</math>

Line 96:

where <math>\alpha</math> is usually fixed to a value lesser than 1, with smaller values for harder problems.<ref name="Transtrum2012"/>

The addition of a geodesic acceleration term can allow significant increase in convergence speed and it is especially useful when the algorithm is moving through narrow canyons in the landscape of the objective function, where the allowed steps are smaller and the higher accuracy due to the second order term gives significativesignificant improvements.<ref name="Transtrum2012"/>

==Example==

Line 204:

|number = 4

|pages = W1–W16

|url = http://link.aip.org/link/?GPY/72/W1/1

|bibcode = 2007Geop...72W...1P

}}

}}{{Dead link|date=February 2020 |bot=InternetArchiveBot |fix-attempted=yes }}

* {{cite book

| last1 = Nocedal | first1 = Jorge

Line 219 ⟶ 218:

== External links ==

* Detailed description of the algorithm can be found in [httphttps://wwwnumerical.nrbook.com/arecipes/bookcpdfbook.phphtml Numerical Recipes in C, Chapter 15.5: Nonlinear models]

* C. T. Kelley, ''Iterative Methods for Optimization'', SIAM Frontiers in Applied Mathematics, no 18, 1999, {{isbn|0-89871-433-8}}. [http://www.siam.org/books/textbooks/fr18_book.pdf Online copy]

* [https://web.archive.org/web/20140301154319/http://www3.villanova.edu/maple/misc/mtc1093.html History of the algorithm in SIAM news]

Line 227 ⟶ 226:

* H. P. Gavin, [http://people.duke.edu/~hpgavin/ce281/lm.pdf ''The Levenberg-Marquardt method for nonlinear least-squares curve-fitting problems''] ([[MATLAB]] implementation included)

{{Optimization algorithms|unconstrained}}

{{DEFAULTSORT:Levenberg-Marquardt algorithm}}