Levenberg–Marquardt algorithm: Difference between revisions

Revision as of 13:48, 29 October 2004

Revision as of 20:00, 1 November 2004

Line 1:

The '''Levenberg-Marquardt algorithm''' provides a [[numerics|numerical]] solution to the [[mathematics|mathematical]] problem of minimizing a sum of squares of several, generally nonlinear functions that depend on a common set of parameters.

In [[mathematics]] and [[statistics]], minimising a sum

This minimization problem arises especially in [[least squares]] [[curve fitting]] (see also: [[nonlinear programming]]).

The Levenberg-Marquardt algorithm (LMA) interpolates between the [[Gauss-Newton algorithm]] (GNA) and the method of [[steepest descent]]. The LMA is ''robuster'' than the GNA, which means that in many cases it finds a solution even if it starts very far off the final minimum. On the other hand, for well-behaved functions and reasonable starting parameters, the LMA tends to be a bit slower than the GNA. The LMA is the most popular curve-fitting algorithm; it is used in almost any software that provides a generic curve-fitting tool; few users will ever need another curve-fitting algorithm.

== The problem ==

Given is the problem: minimize the sum

:<math>S(\vec{x}) = \sum_{i=1}^{m}[f_{i}(\vec{x})]^2</math>

Line 9 ⟶ 17:

represent the components of the function

:<math>\vec{f}</math>, .

== The solution ==

may be solved by the '''Levenberg-Marquardt algorithm''' according to

The LMA solves the minimization problem according to

:<math>(J^{T}J + \lambda I)\vec{q} = -J^{T}\vec{f}</math>.

Line 18 ⟶ 28:

== Weblinks ==

[http://www.siam.org/siamnews/mtc/mtc1093.htm history of the algorithm]

Public ___domain implementations: