Probabilistic classification: Difference between revisions

Revision as of 21:40, 1 May 2015

Revision as of 21:56, 1 May 2015

Line 2:

In [[machine learning]], a '''probabilistic classifier''' is a [[statistical classification|classifier]] that is able to predict, given a sample input, a [[probability distribution]] over a set of classes, rather than only outputting the most likely class that the sample should belong to. Probabilistic classifiers provide classification with a degree of certainty, which can be useful in its own right,<ref>{{cite book |first1=Trevor |last1=Hastie |first2=Robert |last2=Tibshirani |first3=Jerome |last3=Friedman |year=2009 |title=The Elements of Statistical Learning |url=http://statweb.stanford.edu/~tibs/ElemStatLearn/ |page=348 |quote=[I]n [[data mining]] applications the interest is often more in the class probabilities <math>p_\ell(x), \ell = 1, \dots, K</math> themselves, rather than in performing a class assignment.}}</ref> or when combining classifiers into [[ensemble classifier|ensembles]].

Formally, an "ordinary" classifier is some rule, or [[function (mathematics)|function]], that assigns to a sample {{mvar|x}} a class label {{mvar|ŷ}}:

Formally, a probabilistic classifier is a [[conditional probability|conditional]] distribution <math>\operatorname{P}(Y \vert X)</math> over a finite set of classes {{mvar|Y}}, given inputs {{mvar|X}}. Deciding on the best class label <math>\hat{y}</math> for {{mvar|X}} can then be done using the [[Bayes estimator|optimal decision rule]]<ref name="bishop">{{cite book |first=Christopher M. |last=Bishop |year=2006 |title=Pattern Recognition and Machine Learning |publisher=Springer}}</ref>{{rp|39–40}}

:<math>\hat{y} = f(x)</math>

The samples come from some set {{mvar|X}} (e.g., the set of all [[document classification|documents]], or the set of all [[Computer vision#Recognition|images]]), while the class labels form a finite set {{mvar|Y}} defined prior to training.

Formally,Probabilistic aclassifiers probabilisticgeneralize classifierthis isnotion aof classifiers: instead of functions, they are [[conditional probability|conditional]] distributiondistributions <math>\operatorname{P}(Y \vert X)</math>, overmeaning athat finitefor set of classes {{mvar|Y}},a given inputs<math>x \in {{mvar|X}}.</math>, Decidingthey onassign theprobabilities bestto class labelall <math>\hat{y} \in Y</math> for(and {{mvar|X}}these probabilities sum to one). "Hard" classification can then be done using the [[Bayes estimator|optimal decision rule]]<ref name="bishop">{{cite book |first=Christopher M. |last=Bishop |year=2006 |title=Pattern Recognition and Machine Learning |publisher=Springer}}</ref>{{rp|39–40}}

:<math>\hat{y} = \operatorname{\arg\max}_{y} \operatorname{P}(Y=y \vert X)</math>