This function is undefined when <math>p(1\mid x)=1</math> or <math>p(1\mid x)=0</math> (tending toward ∞ and −∞ respectively), but predicts a smooth curve which grows when <math>p(1\mid x)</math> increases and equals 0 when <math>p(1\mid x)= 0.5</math>.<ref name="mitlec" />
It's easy to check that the [[logistic loss]] (above) and binary [[cross entropy]] loss (Log loss) are in fact the same (up to a multiplicative constant <math>\frac{1}{\log(2)}</math>).The cross entropy loss is closely related to the [[Kullback-Leibler divergence]] between the empirical distribution and the predicted distribution. The cross entropy loss is ubiquitous in modern [[deep learning|deep neural networks]].
== Cross entropy loss (Log Loss) ==
{{main|Cross entropy}}
Using the alternative label convention <math>t=(1+y)/2</math> so that <math>t \in \{0,1\}</math>, the binary cross entropy loss is defined as
It's easy to check that the [[logistic loss]] (above) and binary cross entropy are in fact the same (up to a multiplicative constant <math>1/\log2</math>).
The cross entropy loss is closely related to the [[Kullback-Leibler divergence]] between the empirical distribution and the predicted distribution. This function is not naturally represented as a product of the true label and the predicted value, but is convex and can be minimized using [[stochastic gradient descent]] methods. The cross entropy loss is ubiquitous in modern [[deep learning|deep neural networks]].
