Loss functions for classification: Difference between revisions

Revision as of 03:32, 22 July 2023

Revision as of 02:17, 22 September 2023

Line 138:

This function is undefined when <math>p(1\mid x)=1</math> or <math>p(1\mid x)=0</math> (tending toward ∞ and −∞ respectively), but predicts a smooth curve which grows when <math>p(1\mid x)</math> increases and equals 0 when <math>p(1\mid x)= 0.5</math>.<ref name="mitlec" />

It's easy to check that the logistic loss and binary [[cross -entropy]] loss (Log loss) are in fact the same (up to a multiplicative constant <math>\frac{1}{\log(2)}</math>). The cross -entropy loss is closely related to the [[Kullback–Leibler divergence]] between the empirical distribution and the predicted distribution. The cross -entropy loss is ubiquitous in modern [[deep learning|deep neural networks]].

== Exponential loss ==