Content deleted Content added
m →Derivation: cite repair; |
→Finding the derivative of the error: Comment on differentiable activation functions |
||
Line 162:
: <math>\frac{\partial o_j}{\partial\text{net}_j} = \frac {\partial}{\partial \text{net}_j} \varphi(\text{net}_j) = \varphi(\text{net}_j)(1-\varphi(\text{net}_j))</math>
This is the reason why backpropagation requires the activation function to be [[Differentiable function|differentiable]]. (Nevertheless, the non-differentiable [[ReLU]] activation function has become quite popular recently, e.g. in [[AlexNet]])
The first factor is straightforward to evaluate if the neuron is in the output layer, because then <math>o_j = y</math> and
| |||