Stochastic gradient descent: Difference between revisions

Revision as of 23:17, 1 November 2018

Revision as of 05:10, 24 November 2018

Line 108:

:<math>w^{new} := w^{old} - \eta \nabla Q_i(w^{new}).</math>

This equation is implicit since <math>w^{new}</math> appears on both sides of the equation. It is also a stochastic form of the [[proximal gradient method]] since the update

can also be written as:

:<math>w^{new} := \arg\min_w \{ Q_i(w) + \frac{1}{2\eta} ||w - w^{old}||^2 \}.</math>