Matrix factorization (recommender systems): Difference between revisions

Revision as of 20:40, 26 December 2023

Latest revision as of 12:56, 17 April 2025

(One intermediate revision by one other user not shown)

Line 14:

Specifically, the predicted rating user ''u'' will give to item ''i'' is computed as:

:<math>\tilde{r}_{ui} = \sum_{f=0}^{\text{n factors} } H_{u,f}W_{f,i}</math>

It is possible to tune the expressive power of the model by changing the number of latent factors. It has been demonstrated<ref name="Jannach13">{{cite book |last1=Jannach |first1=Dietmar |last2=Lerche |first2=Lukas |last3=Gedikli |first3=Fatih |last4=Bonnin |first4=Geoffray |title=User Modeling, Adaptation, and Personalization |chapter=What Recommenders Recommend – an Analysis of Accuracy, Popularity, and Sales Diversity Effects |volume=7899 |date=2013 |pages=25–37 |doi=10.1007/978-3-642-38844-6_3 |publisher=Springer Berlin Heidelberg |language=en|series=Lecture Notes in Computer Science |isbn=978-3-642-38843-9 |citeseerx=10.1.1.465.96 }}</ref> that a matrix factorization with one latent factor is equivalent to a ''most popular'' or ''top popular'' recommender (e.g. recommends the items with the most interactions without any personalization). Increasing the number of latent factors will improve personalization, therefore recommendation quality, until the number of factors becomes too high, at which point the model starts to [[overfitting|overfit]] and the recommendation quality will decrease. A common strategy to avoid overfitting is to add [[regularization (mathematics)|regularization]] terms to the objective function.<ref name="bi2017">{{cite journal|last1=Bi|first1=Xuan|last2=Qu|first2=Annie|last3=Wang|first3=Junhui|last4=Shen|first4=Xiaotong|year=2017|title=A group-specific recommender system.|url=https://amstat.tandfonline.com/doi/abs/10.1080/01621459.2016.1219261|journal=Journal of the American Statistical Association|volume=112|issue=519|pages=1344–1353|doi=10.1080/01621459.2016.1219261|s2cid=125187672}}</ref><ref>{{cite journal|last1=Zhu|first1=Yunzhang|last2=Shen|first2=Xiaotong|last3=Ye|first3=Changqing|year=2016|title=Personalized prediction and sparsity pursuit in latent factor models.|url=https://figshare.com/articles/journal_contribution/A_Group-Specific_Recommender_System/3803748/1/files/5921967.pdf|journal=Journal of the American Statistical Association|volume=111|issue=513|pages=241–252|doi=10.1080/01621459.2016.1219261|s2cid=125187672}}</ref>

Line 21:

All things considered, Funk MF minimizes the following objective function:

:<math>\underset{H, W}{\operatorname{arg\,min}}\, \|R - \tilde{R}\|_{\rm F} + \alpha\|H\| + \beta\|W\|</math>

Where <math>\|.\|_{\rm F}</math> is defined to be the [[frobenius norm]] whereas the other norms might be either frobenius or another norm depending on the specific recommending problem.<ref name="Paterek07">{{cite journal |last1=Paterek |first1=Arkadiusz |title=Improving regularized singular value decomposition for collaborative filtering |journal=Proceedings of KDD Cup and Workshop |date=2007 |url=https://www.mimuw.edu.pl/~paterek/ap_kdd.pdf}}</ref>

Line 31:

The predicted rating user ''u'' will give to item ''i'' is computed as:

:<math>\tilde{r}_{ui} = \mu + b_i + b_u + \sum_{f=0}^{\text{n factors}} H_{u,f}W_{f,i}</math>

Where <math>\mu</math> refers to the overall average rating over all items and <math>b_i</math> and <math>b_u</math> refers to the observed deviation of the item <math>{{mvar|i</math>}} and the user <math>{{mvar|u</math>}} respectively from the average.<ref>{{Cite journal |last1=Koren |first1=Yehuda |last2=Bell |first2=Robert |last3=Volinsky |first3=Chris |date=August 2009 |title=Matrix factorization techniques for recommender systems |url=https://datajobs.com/data-science-repo/Recommender-Systems-[Netflix].pdf |journal=Computer |pages=45}}</ref> SVD++ has however some disadvantages, with the main drawback being that this method is not ''model-based.'' This means that if a new user is added, the algorithm is incapable of modeling it unless the whole model is retrained. Even though the system might have gathered some interactions for that new user, its latent factors are not available and therefore no recommendations can be computed. This is an example of a [[Cold start (recommender systems)|cold-start]] problem, that is the recommender cannot deal efficiently with new users or items and specific strategies should be put in place to handle this disadvantage.<ref name="Kluver14">{{cite book|last1=Kluver |first1=Daniel |title=Proceedings of the 8th ACM Conference on Recommender systems – Rec ''Sys'' '14 |last2=Konstan |first2=Joseph A. |date=6 October 2014 |pages=121–128 |doi=10.1145/2645710.2645742 |publisher=ACM|chapter=Evaluating recommender behavior for new users |isbn=9781450326681 |s2cid=18509558 }}</ref>

A possible way to address this cold start problem is to modify SVD++ in order for it to become a ''model-based'' algorithm, therefore allowing to easily manage new items and new users.

Line 40:

Note that this does not entirely solve the [[Cold start (recommender systems)|cold-start]] problem, since the recommender still requires some reliable interactions for new users, but at least there is no need to recompute the whole model every time. It has been demonstrated that this formulation is almost equivalent to a SLIM model,<ref name="Zheng14">{{cite book |last1=Zheng |first1=Yong |last2=Mobasher |first2=Bamshad |last3=Burke |first3=Robin |title=CSLIM: contextual SLIM recommendation algorithms |date=6 October 2014 |pages=301–304 |doi=10.1145/2645710.2645756 |publisher=ACM|chapter=CSLIM |isbn=9781450326681 |s2cid=15931532 }}</ref> which is an [[item-item recommender|item-item]] model based recommender.

:<math>\tilde{r}_{ui} = \mu + b_i + b_u + \sum_{f=0}^{\text{n factors}} \biggl( \sum_{j=0}^{\text{n items}} r_{uj} W^T_{j,f} \biggr) W_{f,i}</math>

With this formulation, the equivalent [[item-item recommender]] would be <math>\tilde{R} = R S = R W^{\rm T} W</math>. Therefore the similarity matrix is symmetric.

Line 58:

The predicted rating user ''u'' will give to item ''i'' is computed as:

:<math>\tilde{r}_{ui} = \sum_{f=0}^{\text{n factors}} (H_{u,f}+S_{v_u,f})(W_{f,i}+T_{f,j_i})</math>

Here <math>v_u</math> and <math>j_i</math> represent the group label of user ''u'' and item ''i'', respectively, which are identical across members from the same group. And <math>{{mvar|S</math>}} and <math>{{mvar|T</math>}} are matrices of group effects. For example, for a new user <math>u_{new}</math> whose latent factor <math>H_{u_{new}}</math> is not available, we can at least identify their group label <math>v_{u_{new}}</math>, and predict their ratings as:

:<math>\tilde{r}_{u_{new}i} = \sum_{f=0}^{\text{n factors}} S_{v_{u_{new}},f}(W_{f,i}+T_{f,j_i})</math>

This provides a good approximation to the unobserved ratings.

Line 71:

===Deep-learning MF===

In recent years a number of neural and deep-learning techniques have been proposed, some of which generalize traditional Matrix factorization algorithms via a non-linear neural architecture.<ref>{{cite book |last1=He |first1=Xiangnan |last2=Liao |first2=Lizi |last3=Zhang |first3=Hanwang |last4=Nie |first4=Liqiang |last5=Hu |first5=Xia |last6=Chua |first6=Tat-Seng |title=Proceedings of the 26th International Conference on World Wide Web |chapter=Neural Collaborative Filtering |date=2017 |pages=173–182 |doi=10.1145/3038912.3052569 |chapter-url=https://dl.acm.org/citation.cfm?id=3052569 |accessdate=16 October 2019 |publisher=International World Wide Web Conferences Steering Committee|isbn=9781450349130 |arxiv=1708.05031 |s2cid=13907106 }}</ref>

While deep learning has been applied to many different scenarios: (context-aware, sequence-aware, social tagging, etc.), its real effectiveness when used in a simple [[Collaborative filtering]] scenario has been put into question. Systematic analysis of publications applying deep learning or neural methods to the top-k recommendation problem, published in top conferences (SIGIR, KDD, WWW, RecSys, IJCAI), has shown that on average less than 40% of articles are reproducible, with as little as 14% in some conferences. Overall the studies identify 26 articles, only 12 of them could be reproduced and 11 of them could be outperformed by much older and simpler properly tuned baselines. The articles also highlights a number of potential problems in today's research scholarship and call for improved scientific practices in that area.<ref>{{cite book |last1=Rendle |first1=Steffen |last2=Krichene |first2=Walid |last3=Zhang |first3=Li |last4=Anderson |first4=John |title=Fourteenth ACM Conference on Recommender Systems |chapter=Neural Collaborative Filtering vs. Matrix Factorization Revisited |date=22 September 2020 |pages=240–248 |doi=10.1145/3383313.3412488|arxiv=2005.09683 |isbn=9781450375832 |doi-access=free }}</ref><ref>{{cite journal | last1=Dacrema |last2=Ferrari |title=A Troubling Analysis of Reproducibility and Progress in Recommender Systems Research. |journal=ACM Transactions on Information Systems |pages=39.2 |date=2021 |volume=39 |issue=2 |doi=10.1145/3434185 |arxiv=1911.07698 |s2cid=208138060 }}</ref> Similar issues have been spotted also in sequence-aware recommender systems.<ref>{{cite book |last1=Ludewig |first1=Malte |last2=Mauro |first2=Noemi |last3=Latifi |first3=Sara |last4=Jannach |first4=Dietmar |title=Proceedings of the 13th ACM Conference on Recommender Systems |chapter=Performance comparison of neural and non-neural approaches to session-based recommendation |date=2019 |pages=462–466 |doi=10.1145/3298689.3347041 |publisher=ACM|isbn=9781450362436 |doi-access=free }}</ref>

==See also==