Content deleted Content added
Line 109:
\end{aligned}
</math>where <math>\alpha</math> is a hyperparameter to be optimized on a validation set.
Other works attempt to eliminate BatchNorm, such as the Normalizer-Free ResNet.<ref>https://arxiv.org/abs/2102.06171</ref>
== Layer normalization ==
| |||