Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Recommender System Wenkai Mo The Recommender Problem Estimate a utility function to predict how a user will like an item. Outline • Collaborative Filtering • Content-based Recommendations • Context-based Recommendations • Ranking • Diversity • Socialization Outline • Collaborative Filtering • Content-based Recommendations • Context-based Recommendations • Ranking • Diversity • Socialization Collaborative Filtering • The task of predicting (filtering) user preferences on new items by collecting taste information from many users (collaborative). • Challenges: • • • • • Many items to choose from Very few recommendations to propose Few data per user No data for new users Very large dataset Collaborative Filtering – Memory-based • User-based Similarity • More dynamic • Precomputing user neighbourhood can lead to poor predictions. • Item-based Similarity • Static (No Personality) • We can precompute item neighbourhood. Online computation of the predicted ratings. Collaborative Filtering – data sparse • Large products set but few users rating, so the user rating matrix is very sparse. • Model-based Collaborative Filtering Model-based CF • Clustering • Association Rules • Matrix Factorization • Restricted Boltzmann Machine • Recurrent Neural Networks Model-based CF • Clustering • Association Rules • Matrix Factorization • Restricted Boltzmann Machine • Recurrent Neural Networks Clustering - CF • Cluster customers into categories based on preferences and past purchases • Computer recommendations at the cluster level: all customers within a cluster receive the same recommendations. • + Easy and faster • - Less personalized Association rules – CF • Past purchases used to find relationships of common purchases. Association rules – CF • + Fast to implement • + Fast to execute • + Not much storage space required • + Very successful in broad applications for large populations, such as shelf layout in retail stores • - Not suitable if preferences change rapidly • - Rules can be used only when enough data validates them. False associations can arise Matrix Factorization – CF Matrix Factorization – CF • Frequentists • SVD Loss Function Regularization • SVD++ (Implicit Feedback) Koren Y. Factorization meets the neighborhood: a multifaceted collaborative filtering model[C]//Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, 2008: 426-434. Matrix Factorization – CF • Frequentists - Sparsity Ning X, Karypis G. Slim: Sparse linear methods for top-n recommender systems[C]//Data Mining (ICDM), 2011 IEEE 11th International Conference on. IEEE, 2011: 497-506. Matrix Factorization – CF • Frequentists - Neighbour Kabbur S, Ning X, Karypis G. Fism: factored item similarity models for top-n recommender systems[C]//Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, 2013: 659-667. Matrix Factorization – CF • Bayesians Salakhutdinov R, Mnih A. Probabilistic matrix factorization[C]. NIPS, 2011. Restricted Boltzmann Machine – CF Contract Divergence Train Predict Can update to Deep Belief Networks Restricted Boltzmann Machine – CF • Salakhutdinov R, Mnih A, Hinton G. Restricted Boltzmann machines for collaborative filtering[C]//Proceedings of the 24th international conference on Machine learning. ACM, 2007: 791-798. • Georgiev K, Nakov P. A non-iid framework for collaborative filtering with restricted boltzmann machines[C]//Proceedings of the 30th International Conference on Machine Learning (ICML-13). 2013: 1148-1156. Recurrent Neural Network – CF • Model Sequences. (music recommendation, film recommendation, etc.) linear softmax linear Tensor Regression - CF Park S T, Chu W. Pairwise preference regression for coldstart recommendation[C]//Proceedings of the third ACM conference on Recommender systems. ACM, 2009: 21-28. Limitations of CF • Cold Start • It needs to have enough users in the system. • New items need to get enough ratings. • Sparsity: • it is hard to find users who rated the same items. • Popularity Bias: • Cannot recommend items to users with unique tastes. • Tends to recommend popular items. Outline • Collaborative Filtering • Content-based Recommendations • Context-based Recommendations • Ranking • Diversity • Socialization Content-based Recommendation • Recommendations are based on the information on the content of items rather than on other users’ opinions. • Use a machine learning algorithm to model the users' preferences from examples based on a description of the content. Content-based Recommendation • Collaborative Topic Regression Wang C, Blei D M. Collaborative topic modeling for recommending scientific articles[C]//Proceedings of the 17th ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, 2011: 448-456. Content-based Recommendation • Deep Learning – Consider Noise Wang H, Wang N, Yeung D Y. Collaborative deep learning for recommender systems[C]//Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 2015: 1235-1244. Content-based Recommendation • Content + CF -> Hybrid -> Cold Start Saveski M, Mantrach A. Item cold-start recommendations: learning local collective embeddings[C]//Proceedings of the 8th ACM Conference on Recommender Systems. ACM, 2014: 89-96. Content-based Recommendation • + No need for data on other users. • + No cold-start or sparsity problems. • + Can recommend to users with unique tastes. • + Can recommend new and unpopular items • + Can provide explanations of recommended items by listing contentfeatures that caused an item to be recommended. Content-based Recommendation • - Only for content that can be encoded as meaningful features. • - Some types of items (e.g. movies, music)are not amenable to easy feature extraction methods • - Even for texts, IR techniques cannot consider multimedia information, aesthetic qualities, download time: a positive rating could be not related to the presence of certain keywords • - Users’ tastes must be represented as a learnable function of these content features. • - Hard to exploit quality judgements of other users. • - Difficult to implement serendipity Outline • Collaborative Filtering • Content-based Recommendations • Context-based Recommendations • Ranking • Diversity • Socialization Context-based Recommendations • Context is a dynamic set of factors describing the state of the user at the moment of the user's experience. • Context factors can rapidly change and affect how the user perceives an item. For example, temporal, spatial, social, etc. Pre-filtering Post-filtering Tensor Factorization Karatzoglou A, Amatriain X, Baltrunas L, et al. Multiverse recommendation: ndimensional tensor factorization for contextaware collaborative filtering[C]//Proceedings of the fourth ACM conference on Recommender systems. ACM, 2010: 79-86. Factorization Machines Rendle S, Gantner Z, Freudenthaler C, et al. Fast context-aware recommendations with factorization machines[C]//Proceedings of the 34th international ACM SIGIR conference on Research and development in Information Retrieval. ACM, 2011: 635-644. Rendle S. Factorization machines[C]//Data Mining (ICDM), 2010 IEEE 10th International Conference on. IEEE, 2010: 9951000. Time-awareness Koren Y. Collaborative filtering with temporal dynamics[J]. Communications of the ACM, 2010, 53(4): 89-97. Context-based Recommendations • Zhao Z, Cheng Z, Hong L, et al. Improving User Topic Interest Profiles by Behavior Factorization[C]//Proceedings of the 24th International Conference on World Wide Web. International World Wide Web Conferences Steering Committee, 2015: 1406-1416. • Yang S H, Long B, Smola A J, et al. Collaborative competitive filtering: learning recommender using context of user choice[C]//Proceedings of the 34th international ACM SIGIR conference on Research and development in Information Retrieval. ACM, 2011: 295-304. • Nguyen T V, Karatzoglou A, Baltrunas L. Gaussian process factorization machines for context-aware recommendations[C]//Proceedings of the 37th international ACM SIGIR conference on Research & development in information retrieval. ACM, 2014: 63-72. • Zhang Y, Zhang M, Zhang Y, et al. Daily-Aware Personalized Recommendation based on Feature-Level Time Series Analysis[C]//Proceedings of the 24th International Conference on World Wide Web. International World Wide Web Conferences Steering Committee, 2015: 1373-1383. Outline • Collaborative Filtering • Content-based Recommendations • Context-based Recommendations • Ranking • Diversity • Socialization Learning to Rank • Pointwise • Ranking function minimizes loss function defined on individual relevance judgment e.g. • Ranking score based on regression or classification • Ordinal regression, Logistic regression, SVM Learning to Rank • Pairwise • Loss function is defined on pair-wise preferences • Goal: minimize number of inversions in ranking BPR Rendle S, Freudenthaler C, Gantner Z, et al. BPR: Bayesian personalized ranking from implicit feedback[C]//Proceedings of the twenty-fifth conference on uncertainty in artificial intelligence. AUAI Press, 2009: 452-461. Pairwise Tensor Regression Park S T, Chu W. Pairwise preference regression for coldstart recommendation[C]//Proceedings of the third ACM conference on Recommender systems. ACM, 2009: 21-28. Pairwise Learning to Rank • RankBoost • RankNet • Frank • EigenRank • pLPA • CR •…… Learning to Rank • List-wise • Direct optimization of ranking metrics • List-wise loss minimization for CF a.k.a Collaborative Ranking Listwise Learning to Rank • CoFiRank: optimizes an upper bound of • NDCG (Smooth version) • CLiMF : optimizes a smooth version of MRR • TFMAP: optimizes a smooth version of MAP • AdaRank: uses boosting to optimize NDCG • ListRank • WLT • GAPfm • xCLiMF •…… Outline • Collaborative Filtering • Content-based Recommendations • Context-based Recommendations • Ranking • Diversity • Socialization Diversity • Recommendations from a music on-line retailer: • Problem: No diversity: pop albums from female singers. • Some are redundant. Diversity • Re-ranking Sub-profile Diversity and Novelty Novelty Model Popularity-based Item Novelty Distance-based Item Novelty Browsing Model • A chosen item must obviously be seen, and relevant items are more likely to be chosen than irrelevant ones. Vargas S, Castells P. Rank and relevance in novelty and diversity metrics for recommender systems[C]//Proceedings of the fifth ACM conference on Recommender systems. ACM, 2011: 109-116. Reference • Smyth B, McClave P. Similarity vs. diversity[M]//Case-Based Reasoning Research and Development. Springer Berlin Heidelberg, 2001: 347361. • Lathia N, Hailes S, Capra L, et al. Temporal diversity in recommender systems[C]//Proceedings of the 33rd international ACM SIGIR conference on Research and development in information retrieval. ACM, 2010: 210-217. Outline • Collaborative Filtering • Content-based Recommendations • Context-based Recommendations • Ranking • Diversity • Socialization Socialization • 1. Trust-based Social Recommendation Edge: Trust Value (user sim., SNS, etc.) Vertex: Item Rating Jamali M, Ester M. Trustwalker: a random walk model for combining trust-based and item-based recommendation[C] //Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, 2009: 397-406. Socialization • 1. Trust-based Social Recommendation • Forsati R, Mahdavi M, Shamsfard M, et al. Matrix factorization with explicit trust and distrust side information for improved social recommendation[J]. ACM Transactions on Information Systems (TOIS), 2014, 32(4): 17. • Jamali M, Ester M. A matrix factorization technique with trust propagation for recommendation in social networks[C]//Proceedings of the fourth ACM conference on Recommender systems. ACM, 2010: 135-142. • J. Golbeck. Computing and Applying Trust in Web-based Social Networks. PhD thesis, University of Maryland College Park, 2005 • P. Massa and P. Avesani. Trust-aware recommender systems. In RecSys 2007, USA. • Levien and Aiken. Advogato’s trust metric. online at http://advogato.org/trust-metric.html, 2002 Socialization • 2. MF-based Social Recommendation User himself Friends Delporte J, Karatzoglou A, Matuszczyk T, et al. Socially enabled preference learning from implicit feedback data[M]//Machine Learning and Knowledge Discovery in Databases. Springer Berlin Heidelberg, 2013: 145-160. Socialization • 2. MF-based Social Recommendation • Jamali M, Ester M. A matrix factorization technique with trust propagation for recommendation in social networks[C]//Proceedings of the fourth ACM conference on Recommender systems. ACM, 2010: 135-142. • Noel J, Sanner S, Tran K N, et al. New objective functions for social collaborative filtering[C]//Proceedings of the 21st international conference on World Wide Web. ACM, 2012: 859-868. • Yang X, Steck H, Guo Y, et al. On top-k recommendation using social networks[C]//Proceedings of the sixth ACM conference on Recommender systems. ACM, 2012: 67-74. • H. Ma, I. King, and M. R. Lyu. Learning to recommend with social trust ensemble. In SIGIR 2009, pages 203–210. • H. Ma, H. Yang, M. R. Lyu, and I. King. Sorec: social recommendation using probabilistic matrix factorization. In CIKM 2008, pages 931–940. ACM, 2008. Socialization • 3. Finding Experts • X. Yang, H. Steck, Y. Guo, and Y. Liu. On top-k recommendation using social networks. In Proc. of RecSys’12, 2012.