Loss functions for classification: Difference between revisions

Revision as of 18:36, 31 July 2019

Revision as of 19:40, 31 July 2019

Line 128:

This function is undefined when <math>p(1\mid x)=1</math> or <math>p(1\mid x)=0</math> (tending toward ∞ and −∞ respectively), but predicts a smooth curve which grows when <math>p(1\mid x)</math> increases and equals 0 when <math>p(1\mid x)= 0.5</math>.<ref name="mitlec" />

It's easy to check that the [[logistic loss]] (above) and binary [[cross entropy]] loss (Log loss) are in fact the same (up to a multiplicative constant <math>\frac{1}{\log(2)}</math>).The cross entropy loss is closely related to the [[Kullback-Leibler divergence]] between the empirical distribution and the predicted distribution. The cross entropy loss is ubiquitous in modern [[deep learning|deep neural networks]].

== Exponential loss ==