Proximal gradient method: Difference between revisions

Revision as of 19:00, 26 January 2019

Revision as of 23:32, 6 April 2019

Line 1:

{{more footnotes|date=November 2013}}

'''Proximal gradient methods''' are a generalized form of projection used to solve non-differentiable [[convex optimization]] problems.

Many interesting problems can be formulated as convex optimization problems of form

:<math>

\operatorname{min}\limits_{x \in \mathbb{R}^N} \sum_{i=1}^n f_i(x)

Line 7 ⟶ 9:

where <math>f_i,\ i = 1, \dots, n</math> are [[convex functions]] defined from <math>f: \mathbb{R}^N \rightarrow \mathbb{R} </math>

where some of the functions are non-differentiable, this rules out our conventional smooth optimization techniques like

[[Gradient descent|Steepest descent method]], [[conjugate gradient method]] etc. ThereProximal isgradient amethods specificcan classbe ofused [[algorithms]]instead. whichThese canmethods proceed by splitting, in solvethat the abovefunctions optimization<math>f_1, problem. These. methods. proceed, byf_n</math> splitting,are used individually so as to yield an easily [[wikt:implementable|implementable]] algorithm.

in that the functions <math>f_1, . . . , f_n</math> are used individually so as to yield an easily [[wikt:implementable|implementable]] algorithm.

They are called [[proximal]] because each non [[smooth function]] among <math>f_1, . . . , f_n</math> is involved via its proximity

operator. Iterative Shrinkage thresholding algorithm,<ref>