Content deleted Content added
m Open access bot: url-access updated in citation with #oabot. |
|||
| (41 intermediate revisions by 26 users not shown) | |||
Line 1:
{{Short description|Subfield of machine learning}}
'''Preference learning''' is a subfield of [[machine learning]] that focuses on modeling and predicting preferences based on observed preference information.<ref>{{Cite Mehryar Afshin Ameet 2012}}</ref> Preference learning typically involves [[supervised learning]] using datasets of pairwise preference comparisons, rankings, or other preference information.
==Tasks==
The main task in preference learning concerns problems in "[[learning to rank]]". According to different types of preference information observed, the tasks are categorized as three main problems in the book ''Preference Learning'':<ref>{{Cite book |url=https://books.google.com/books?id=nc3XcH9XSgYC&pg=PA4 |title=Preference learning |date=2010 |publisher=Springer |isbn=978-3-642-14124-9 |editor-last=Fürnkranz |editor-first=Johannes |___location= |pages=3–8 |editor-last2=Hüllermeier |editor-first2=Eyke}}</ref>
===Label ranking===
In label ranking, the model has an instance space <math>X=\{x_i\}\,\!</math> and a finite set of labels <math>Y=\{y_i|i=1,2,\cdots,k\}\,\!</math>. The preference information is given in the form <math>y_i \succ_{x} y_j\,\!</math> indicating instance <math>x\,\!</math> shows preference in <math>y_i\,\!</math> rather than <math>y_j\,\!</math>. A set of preference information is used as training data in the model. The task of this model is to find a preference ranking among the labels for any instance.
It was observed that some conventional [[Classification in machine learning|classification]] problems can be generalized in the framework of label ranking problem:<ref>{{Cite journal |date=2002 |title=Constraint classification for multiclass classification and ranking |url=https://proceedings.neurips.cc/paper_files/paper/2002/file/16026d60ff9b54410b3435b403afd226-Paper.pdf |journal=NeurIPS}}</ref> if a training instance <math>x\,\!</math> is labeled as class <math>y_i\,\!</math>, it implies that <math>\forall j \neq i, y_i \succ_{x} y_j\,\!</math>. In the [[Multi-label classification|multi-label]] case, <math>x\,\!</math> is associated with a set of labels <math>L \subseteq Y\,\!</math> and thus the model can extract a set of preference information <math>\{y_i \succ_{x} y_j | y_i \in L, y_j \in Y\backslash L\}\,\!</math>. Training a preference model on this preference information and the classification result of an instance is just the corresponding top ranking label.
===Instance ranking===
Instance ranking also has the instance space <math>X\,\!</math> and label set <math>Y\,\!</math>. In this task, labels are defined to have a fixed order <math>y_1 \succ y_2 \succ \cdots \succ y_k\,\!</math> and each instance <math>x_l\,\!</math> is associated with a label <math>y_l\,\!</math>. Giving a set of instances as training data, the goal of this task is to find the ranking order for a new set of instances.
===Object ranking===
Object ranking is similar to instance ranking except that no labels are associated with instances. Given a set of pairwise preference information in the form <math>x_i \succ x_j\,\!</math> and the model should find out a ranking order among instances.
==Techniques==
There are two practical representations of the preference information <math>A \succ B\,\!</math>. One is assigning <math>A\,\!</math> and <math>B\,\!</math> with two real numbers <math>a\,\!</math> and <math>b\,\!</math> respectively such that <math>a > b\,\!</math>. Another one is assigning a binary value <math>V(A,B) \in \{0,1\}\,\!</math> for all pairs <math>(A,B)\,\!</math> denoting whether <math>A \succ B\,\!</math> or <math>B \succ A\,\!</math>. Corresponding to these two different representations, there are two different techniques applied to the learning process.
===Utility function===
If we can find a mapping from data to real numbers, ranking the data can be solved by ranking the real numbers. This mapping is called [[utility function]]. For label ranking the mapping is a function <math>f: X \times Y \rightarrow \mathbb{R}\,\!</math> such that <math>y_i \succ_x y_j \Rightarrow f(x,y_i) > f(x,y_j)\,\!</math>. For instance ranking and object ranking, the mapping is a function <math>f: X \rightarrow \mathbb{R}\,\!</math>.
Finding the utility function is a [[Regression analysis|regression]] learning problem{{citation needed|date=March 2025}} which is well developed in machine learning.
===Preference relations===
The binary representation of preference information is called preference relation. For each pair of alternatives (instances or labels), a binary predicate can be learned by conventional supervised learning approach. Fürnkranz and Hüllermeier proposed this approach in label ranking problem.<ref name=":0">{{Cite book |last1=Fürnkranz |first1=Johannes |last2=Hüllermeier |first2=Eyke |chapter=Pairwise Preference Learning and Ranking |series=Lecture Notes in Computer Science |date=2003 |volume=2837 |editor-last=Lavrač |editor-first=Nada |editor2-last=Gamberger |editor2-first=Dragan |editor3-last=Blockeel |editor3-first=Hendrik |editor4-last=Todorovski |editor4-first=Ljupčo |title=Machine Learning: ECML 2003 |chapter-url=https://link.springer.com/chapter/10.1007/978-3-540-39857-8_15 |language=en |___location=Berlin, Heidelberg |publisher=Springer |pages=145–156 |doi=10.1007/978-3-540-39857-8_15 |isbn=978-3-540-39857-8}}</ref> For object ranking, there is an early approach by Cohen et al.<ref>{{Cite journal |last1=Cohen |first1=William W. |last2=Schapire |first2=Robert E. |last3=Singer |first3=Yoram |date=1998-07-31 |title=Learning to order things |url=https://dl.acm.org/doi/10.5555/302528.302736 |journal=NeurIPS |series= |___location=Cambridge, MA, USA |publisher=MIT Press |pages=451–457 |doi= |isbn=978-0-262-10076-2}}</ref>
Using preference relations to predict the ranking will not be so intuitive. Since observed preference relations may not always be transitive due to inconsistencies in the data, finding a ranking that satisfies all the preference relations may not be possible or may result in multiple possible solutions. A more common approach is to find a ranking solution which is maximally consistent with the preference relations. This approach is a natural extension of pairwise classification.<ref name=":0" />
==Uses==
Preference learning can be used in ranking search results according to feedback of user preference. Given a query and a set of documents, a learning model is used to find the ranking of documents corresponding to the [[relevance (information retrieval)|relevance]] with this query. More discussions on research in this field can be found in [[Tie-Yan Liu]]'s survey paper.<ref>{{Cite journal |last=Liu |first=Tie-Yan |date=2007 |title=Learning to Rank for Information Retrieval |url=http://www.nowpublishers.com/article/Details/INR-016 |journal=Foundations and Trends in Information Retrieval |language=en |volume=3 |issue=3 |pages=225–331 |doi=10.1561/1500000016 |issn=1554-0669|url-access=subscription }}</ref>
Another application of preference learning is [[recommender systems]].<ref>{{Citation |last1=Gemmis |first1=Marco de |title=Learning Preference Models in Recommender Systems |date=2010 |work=Preference Learning |pages=387–407 |editor-last=Fürnkranz |editor-first=Johannes |url=http://link.springer.com/10.1007/978-3-642-14125-6_18 |access-date=2024-11-05 |place= |publisher=Springer |language=en |doi=10.1007/978-3-642-14125-6_18 |isbn=978-3-642-14124-9 |last2=Iaquinta |first2=Leo |last3=Lops |first3=Pasquale |last4=Musto |first4=Cataldo |last5=Narducci |first5=Fedelucio |last6=Semeraro |first6=Giovanni |editor2-last=Hüllermeier |editor2-first=Eyke|url-access=subscription }}</ref> Online store may analyze customer's purchase record to learn a preference model and then recommend similar products to customers. Internet content providers can make use of user's ratings to provide more user preferred contents.
==References==
{{Reflist}}
[[Category:Information retrieval techniques]]
[[Category:Machine learning]]
| |||