Content deleted Content added
Erel Segal (talk | contribs) |
Erel Segal (talk | contribs) |
||
Line 91:
==Strongly convex functions==
The concept of strong convexity extends and parametrizes the notion of strict convexity. Intuitively, a strongly-convex function is a function that grows as fast as a quadratic function.<ref>{{Cite web |title=Strong convexity · Xingyu Zhou's blog |url=https://xingyuzhou.org/blog/notes/strong-convexity |access-date=2023-09-27 |website=xingyuzhou.org}}</ref> A strongly convex function is also strictly convex, but not vice versa. If <math>f</math> is twice continuously differentiable and the ___domain is the real line, then we can characterize it as follows:
*<math>f</math> convex if and only if <math>f''(x) \ge 0</math> for all <math>x.</math>▼
*<math>f</math> strictly convex if <math>f''(x) > 0</math> for all <math>x</math> (note: this is sufficient, but not necessary).▼
*<math>f</math> strongly convex if and only if <math>f''(x) \ge m > 0</math> for all <math>x.</math>▼
For example, let <math>f</math> be strictly convex, and suppose there is a sequence of points <math>(x_n)</math> such that <math>f''(x_n) = \tfrac{1}{n}</math>. Even though <math>f''(x_n) > 0</math>, the function is not strongly convex because <math>f''(x)</math> will become arbitrarily small.▼
Intuitively, a strongly-convex function is a function that grows as fast as a quadratic function.<ref>{{Cite web |title=Strong convexity · Xingyu Zhou's blog |url=https://xingyuzhou.org/blog/notes/strong-convexity |access-date=2023-09-27 |website=xingyuzhou.org}}</ref> Formally, a differentiable function <math>f</math> is called strongly convex with parameter <math>m > 0</math> if the following inequality holds for all points <math>x, y</math> in its ___domain:<ref name="bertsekas">{{cite book|page=[https://archive.org/details/convexanalysisop00bert_476/page/n87 72]|title=Convex Analysis and Optimization|url=https://archive.org/details/convexanalysisop00bert_476|url-access=limited|author=Dimitri Bertsekas| others= Contributors: Angelia Nedic and Asuman E. Ozdaglar|publisher=Athena Scientific|year=2003|isbn=9781886529458}}</ref><math display=block>(\nabla f(x) - \nabla f(y) )^T (x-y) \ge m \|x-y\|_2^2 </math>▼
▲
or, more generally,
<math display=block>\langle \nabla f(x) - \nabla f(y), x-y \rangle \ge m \|x-y\|^2 </math>
Line 119 ⟶ 124:
<math display=block>x\mapsto f(x) -\frac m 2 \|x\|^2</math>
is convex.
▲*<math>f</math> convex if and only if <math>f''(x) \ge 0</math> for all <math>x.</math>
▲*<math>f</math> strictly convex if <math>f''(x) > 0</math> for all <math>x</math> (note: this is sufficient, but not necessary).
▲*<math>f</math> strongly convex if and only if <math>f''(x) \ge m > 0</math> for all <math>x.</math>
▲For example, let <math>f</math> be strictly convex, and suppose there is a sequence of points <math>(x_n)</math> such that <math>f''(x_n) = \tfrac{1}{n}</math>. Even though <math>f''(x_n) > 0</math>, the function is not strongly convex because <math>f''(x)</math> will become arbitrarily small.
A twice continuously differentiable function <math>f</math> on a compact ___domain <math>X</math> that satisfies <math>f''(x)>0</math> for all <math>x\in X</math> is strongly convex. The proof of this statement follows from the [[extreme value theorem]], which states that a continuous function on a compact set has a maximum and minimum.
| |||