Talk:Levenberg–Marquardt algorithm: Difference between revisions

Content deleted Content added
Choice of damping parameter: should discuss the approach of Moré
Line 41:
 
: A way to address this might be to add a section called (say) "Improved solution" after the "Choice of damping parameter" section, with the new formula. The original "solution" section and this new one should probably be better referenced as well. Thoughts? I may give it a shot soon. [[User:Baccyak4H|Baccyak4H]] ([[User talk:Baccyak4H|Yak!]]) 18:29, 25 July 2007 (UTC)
 
I believe Marquardt's suggested improvement was actually to scale the approximate hessian matrix, <math>J^{T}J</math>. The scaling had an effect similar to replacing the identity matrix with the diagonal of the hessian approximation. Marquardt suggests scaling the hessian approximation by an amount that makes the diagonal elements ones. Adding a constant diagonal matrix to the scaled matrix is similar to adding a proportion of the diagonal elements to the unscaled matrix. The scaling applied to the other elements of the hessian approximation improves the condition of the matrix. I suggest the following:
 
<math>q = \Sigma_J[\hat{J}^T\hat{J} + \lambda{}I]^{-1}\hat{J}^T [y - f(p)]</math>
 
where <math>\Sigma_J</math> is the square diagonal matrix:
 
<math>\Sigma_J = [\mbox{diag}[J^TJ]]^{-\frac{1}{2}}</math>
and the scaled Jacobian, <math>\hat{J}</math>, is:
 
<math>\hat{J} = J\Sigma_J</math>
 
The square matrix, <math>\hat{J}^T\hat{J}</math>, is then a scaled version
of the squared Jacobian, <math>J^{T}J</math>, where the scale factor is
the root mean square of the columns of <math>J^{T}J</math>. The result
of the scaling is ones on all the diagonal elements of <math>\hat{J}^T\hat{J}</math>.
 
Note that more recent implementations that use the Levenberg-Marquardt tag do not include Marquardt's suggestion. It seems that clever choices of <math>\lamda> result in reasonable robustness and less function evaluations.
 
== Reference to nonlinear programming ==