1.引言

“We are leaving the information age,Enter the recommended era.”

As with many machine learning techniques,The history of recommendation system based on user behavior to predict.推荐系统是一种信息过滤系统,具体来说,is based on the user's historical behavior、社交关系、兴趣点.to predict user preferences for a set of items.

在过去的几十年里,随着 Youtube、亚马逊、Netflix and the rise of many other such web services,Recommendation system in our life to occupy the position of more and more.从电子商务（Recommend products to buyers that may be of interest to them）The online advertising（Recommend the right content to the user,match their preferences）,Recommendation system now in our daily life is inevitable.一般来说,Recommendation system is designed to recommend to the user related project algorithm（item is a movie to watch、text to read、Want to buy the product or any other project depends on the industry）.

Recommendation systems are very important in some industries,Because they can generate huge income when efficient,Is also a way to stand out from the control of the competition for.As a testament to the importance of recommender systems,we can mention,几年前,Netflix organized a challenge（“Netflix 奖”）,The goal is to produce a performance algorithm is better than your own recommendation system and rewarded100 万美元的奖金.

To build a recommender system,The two most typical methods are

• 基于内容 (content-based)
• 协同过滤 (Collaborative Filtering)

Among them, collaborative filtering is divided into memory-based（memory based）和基于模型（Model based）两种方法,Next we will focus on how they work、优缺点等.

2.基于内容的推荐方法（content-based Filtering）

基于内容的Methods need a lot of information about the project its own characteristics,instead of using user interaction and feedback.例如,可以是类型、年份、导演、Actors and other movie attributes,Can also be applied to natural language processing to extract text content.

• Tag items based on their attributes
• Calculate similarity by label

然后,The method based on the content of idea is to try based on available“特征”构建一个模型,to explain the observed users-项目交互.Still consider the user and movies,例如,We will try to young women tend to some film score is higher、Young men are inclined to other film score higher fact modeling.If we manage to get a model like this,So it's very easy for users to make new prediction,：We just need to look at that user's profile（年龄、性别……）,and based on this information to determine relevant movie recommendations.So based on the recommended method is a method of static content.

基于内容The method of the冷启动问题much less impact：New users or items can pass their characteristics（属性）来分析,Therefore can provide relevant recommendations for the new entity.For only which is a new user or have the characteristics of the project will have a problem in logic,But once the system is long enough,This almost never happens.中,Recommendation problems are often transformed into分类问题（predict whether a user“喜欢”某个项目）或回归问题（Predict user ratings for items）.Content based explanation,在这两种情况下,We will both set up a model,The model will be based on user or the items we can use the feature（That's what we call content-based）.

If our classification（或回归）是基于用户特征的,We say this method is以项目为中心的：建模、Optimization and calculation can“按项目”完成.在这种情况下,We study based on user characteristics build and a model,试图回答“What is the probability of each user like this project？”这个问题. （或者“What is each user's rating for this item？”）.There are usually many users interact with a project,The resulting model is therefore based on robustness.然而,Considering the interaction learning model from each user,even if these users have similar characteristics（特征）,their preferences may also differ.This means that although this method is more鲁棒,But it is after the user-centered approach is less具有个性化.

if we are researching项目特征,则该方法以用户为中心：建模、Optimization and calculation can“由用户”完成.然后,We according to the training of users according to the characters of project a model,试图回答“The user like what is the probability of each project？”.此时,We can train a model for each user,The model obtained than project centered model more个性化推荐,Because it only considers the interaction from the user.然而,Most of the time users interact with a relatively small project,因此,We get the model is much less than the project centered model robust.

从实际的角度来看,在大多数情况下,Get some information from new users（Users don't want to answer too many questions）Than to get a lot of information about the new items is much more difficult.（Used because these items are interested to fill out these information,In order to recommend their projects to the correct user）.我们还可以注意到,Depending on the complexity of the relationship to be expressed,We build the model of may from simple to complex：Linear Regression to Deep Neural Networks.最后,让我们提一下,The method based on content can be user-centered,project-centric：Information about the user and the project can be used in our model,For example, by the stacked two characteristic vector and build neural network architecture.

2.1Item-Centric Bayesian Classifier

Let us first consider the situation of the project centered classification：对于每个项目,We want to train a Bayesian classifier,The classifier to user features as the input and output“喜欢”或“不喜欢”.所以,为了完成分类任务,我们要计算
$$\frac{p_{item}(like|use{r}_{features})}{p_{item}(dislike|use{r}_{features})}$$
With the given characteristics of the user like to consider the probability of project and its probability and the ratio of don't like it.Define our classification rules（with simple threshold）The conditional probability ratio can according to the bayesian formula said
$$\frac{p_{item}(like|use{r}_{features})}{p_{item}(dislike|use{r}_{features})}=\frac{p_{item}(use{r}_{features}|like)\times p_{item}(like)}{p_{item}(use{r}_{features}|dislike)\times p_{item}(dislike)}$$

其中：$p_{item}(like)$是先验概率,Can pass before the data is calculated.

$p_{item}(\cdot |like)$是条件概率,This is the most important part of a Bayesian model,First, Naive Bayes assumes that the features are条件独立的.So that we can be dismantled the conditional probability.

当特征是离散型数据时,只要很方便的统计训练样本中各个划分在每个类别中出现的频率即可用来估计$P(use{r}_{featur{e}_i}|like)$.

下面重点讨论特征属性是连续值的情况.

当特征是连续型数据：通常假定其值服从高斯分布（也称正态分布）.因此只要计算出训练样本中各个类别中此特征项划分的各均值和标准差,Into the formula of normal distribution can be need to estimate.均值与标准差的计算在此不再赘述.

另一个需要讨论的问题就是当$P(use{r}_{featur{e}_i}|like)=0$怎么办,当某个类别下某个特征项划分没有出现时,this phenomenon occurs,这会令分类器质量大大降低.为了解决这个问题,我们引入拉普拉斯平滑法,它的思想非常简单,就是对没类别下所有划分的计数加1,这样如果训练样本集数量充分大时,并不会对结果产生影响,并且解决了上述频率为0的尴尬局面.

2.1User-Centric Linear Regression

Let us now consider to the return of the user as the center of：对于每个用户,We want to train a simple linear regression,The return to the project features as the input and output of the project score.我们将 M 表示为用户-项目交互矩阵,矩阵 X Indicates the user coefficient to be learned,矩阵 Y Said given project features.然后,对于给定的用户 i,We learn by solving the following optimization problem $X_i$中的系数
$$X_i=min\frac{1}{2}\sum _{(i,j)}[(X_i)(Y_j{)}^T-M_{ij}{]}^2+\frac{\lambda }{2}\sum _k(X_{ik}{)}^2$$
注意： i 是固定的,因此,The first sum is only for users with i 相关的 (user, item) 对.我们可以观察到,If we solve model for all users at the same time,Optimization problem and keep the items we introduced later fixed in“Alternating Matrix Factorization”The exact same problem solved in.This shows that model-based协同过滤方法（例如矩阵分解）and content-based methodsboth assume the user-Project interactions exist潜在模型,But based on the model of collaborative filtering must learn two users and project matrix,And based on the contents of the method only needs to learn a user or project a matrix.

3.协同过滤（Collaborative Filtering）

• Collaborative filtering is find similar to your user through data,through their behavior and the content they like.For you to recommend items or content that you might be interested in
• 日常生活中,We will find the same interest friend to help us recommend movies or music

Collaborative filtering is a动态的方法.协同过滤In addition to user preferences of the history of a set of projects,不需要其他任何东西.because it is based on historical data,So here's core assumption is used to the user in the future will be like.in terms of user preferences,Usually expressed in two categories.clear rating, Is the user to the ratio of a sliding scale project,Like the Titanic 5 星评分.This is the most direct feedback from users,Show how much they like a project.Implicit Rating,Indirect response to user preferences,例如页面浏览量、点击量、购买记录、Whether to listen to music tracks, etc..在本文中,We'll take a closer look at collaborative filtering,It is a traditional and powerful tool for recommender systems.

The collaborative filtering recommendation system is based on the records of the past interaction between a user and the project only to create new method recommended by.These interactions are stored in“用户-项目交互矩阵”中.

然后,The main idea of the collaborative filtering method is the past用户-项目交互And to detect similar users/or similar items,According to these estimates predict similar DuDu.

The category of the collaborative filtering algorithm is divided into two sub-categories,Often referred to as the method based on memory and the method based on the model.The method based on memory used directly record the interaction of the values,Assuming no model,and is basically based on nearest neighbor search（例如,Found in the users interested in recent users and recommend the most popular among these users to their project）.Suppose a model-based method潜在的“生成”模型to explain the user-Project interactions and try to find it to make a new prediction.

The main advantage of the collaborative filtering method is that they don't need information about a user or project,so their use cases are wider.此外,The more user interaction with the project,New recommendations are more accurate：For a fixed set of users and projects,With the passage of time records of new interaction will bring new information,And make the system more effective.

然而,Because it only considers the past interaction to recommend,Collaborative filtering exists“冷启动问题”：Impossible to new users recommend anything or to any user to recommend new items,And many to effectively deal with the interaction of the user or project too.This shortcoming can be solved in different ways：Recommend to new users random projects or recommend to random users for the new project（随机策略）,Recommend to new users popular projects or recommend to the most active users for the new project（maximum expectation strategy）,Recommend a different set of new user or a group of different users of the new projects（exploratory strategy）,或者最后,To the user or the project early use of the collaborative filtering method.
下面,We will mainly introduce three classic collaborative filtering methods：Two memory-based methods（用户-用户和项目-项目）and a model-based approach（矩阵分解）.

3.1基于记忆的协同过滤

用户-用户和项目-Project method of main characteristic is that they only use from用户-项目交互矩阵Information and they assume that no model to generate a new recommendation.

3.1.1用户-用户(User-user)

To make new recommendations to users,用户-Roughly attempts to identify the user method has the similar“Interactive overview”的用户（最近的邻居）,to recommend in these neighbors最受欢迎的项目（And this project is new for our users）.

通俗的来说Is first of all, according to the historical data of the user information for classification,Then recommend popular items in the same class users.例如A、B、CThe three users are very similar,现在A、Busers frequently buyD产品,但是Cusers never bought,我们可以向C用户推荐D产品.

这种方法被称为 “以用户为中心” ,Because it is based on user interactions with the project to represent the distance between users and evaluate.

Suppose we want to make recommendations for a given user.首先,Each user can use it with interaction of different project vector（each row of the interaction matrix）来表示.然后,We can calculate between the user and all other users which we are interested in some kind of“相似性”.The similarity measure makes on the same project with similar interaction of two users shall be deemed to be close to.Once the similarity to each user is calculated,我们就可以将 k nearest neighbors,Then suggest the most popular items among them（Just check our reference user has not yet interacts with project）.

请注意,When calculating similarity between users,应该仔细考虑“mutual interaction”的数量（How many items are in common between two users）！事实上,在大多数情况下,We want to avoid with our reference user onlyAn interactionBut this time may have 100% 的匹配,is considered to be more 100 mutual interaction但是只有98%匹配度of users are more similar.因此,If two users in the same way（相似的评分、similar hover time……）Interact with many common project,We think they are similar.

3.1.2项目-项目（Item-item）

To make new recommendations to users,item-item The idea of ​​the method is to find something that the user already has“积极”Items that have interacted with similar items.If interact with two projects of the majority of users to operate in a similar way,the two items are considered to be similar.这种方法被称为**“以项目为中心”**,Because it is based on the user interactions with them to represent the project,and assess the distance between these items.

Suppose we want to make recommendations for a given user.首先,We consider this user's favorite item,and pass it the interaction vector with each user（In the interaction matrix“它的列”）来表示它（like all other projects）.然后,我们可以计算“best project”Similarity with all other projects.Once the similarity is calculated,我们就可以保留kA nearest neighbor project recommend our users interested in these items.

请注意,For more relevant recommendations,We can not only in view of the user's favorite project to do the work,而是考虑 **n preferred items.**在这种情况下,We can recommend the preferred projects in several close to.

3.1.3比较 user-user 和 item-item

用户-User method based on search similar users in terms of interaction with the project.一般来说,Each user interacts with only a few items,This makes the method is sensitive to any interaction records（高方差）.另一方面,Because ultimately recommend only based on similar to the user which we are interested in user record interaction,So we get more personalized results（低偏差）.
相反,item-item method is based on the user-item Search for similar interactionsitem.因为一般来说,Many users interacted with an item,So the sensitivity of the neighborhood search to a single interaction is much lower（较低的方差）.作为对应,Interactions from various users（Even with our reference users very different）is then considered in the recommendation,makes the method less personal（more bias）.因此,This method is not like the user-user method,but more robust.

Complexity and side effects

One of the biggest defect based on the collaborative filtering of memory is that they are not easy to extend：Generate new recommended for large systems can be very time consuming.事实上,For millions of users and hundreds of millions of project of system,If not carefully designed,Nearest neighbor search steps may become difficult to deal with（KNN 算法的复杂度为 O(ndk),其中 n 是用户数,d 是项目数,kConsidering the number of neighbors）.In order to make the calculation of large systems are easier to deal with,We can in the design of algorithm based on the interaction matrix of sparse sex,Approximate nearest neighbor method can also be used (ANN).
in most recommendation algorithms,Must be very careful to avoid the hottest commodity produce“越来越多”的效果,And to avoid lets the user into the so-called**“Information restricted area”.** 换句话说,We don't want our systems tend to recommend more and more popular goods,Don't hope our customers only receive and they have like commodities is very close to the recommended,And didn't have a chance to know they may also like the new project（Because these items are not enough“接近”,无法推荐）.正如我们所提到的,These problems may appear in most of the recommendation algorithm,Especially for the collaborative filtering algorithm based on memory.事实上,由于缺乏“规范化”的模型,This phenomenon can be more outstanding and more frequently observed.

3.2Model-Based Collaborative Filtering

Collaborative filtering method based on the model only relies on the user-Project interaction information,And assuming a potential model should explain these interactions.例如,矩阵分解算法（Matrix Factorization）Including the huge and sparse user-Project interaction matrix is decomposed into two small and dense matrix product：一个用户-factor matrix（Contain the user said）Multiplied by a factor-项目矩阵（Contains item representation）

例如,Consider we have a matrix of user movie ratings.In order to the interaction between the user and film modeling,我们可以假设

• There are some movie features.
• These features can also be used to describe user preferences（If the user likes it, the feature value is high,Otherwise the eigenvalues ​​are low）

但是,We don't want to add these features clearly to our model（Because it can be used in our later will describe the method based on the content）.相反,We prefer to let the system to find these useful feature,And automatically to represent users and items.Since they are learned rather than given,So the separately extracted features have mathematical significance,but no intuitive explanation.然而,The algorithm eventually produce structure is very close to human visual decomposition can think of.事实上,The result of this decomposition is,Close to the user in terms of preferences and close to the characteristics of final project in the potential space has the close to said.

Mathematical Explanation of Matrix Decomposition

Next, we will introduce a simple matrix decomposition summary of mathematical.更具体地说,We described a classic iterative method based on gradient descent,It can be loaded into the computer at the same time in the don't put all the data in memory to acquire very large matrix decomposition.
Let us consider a interaction matrix contains the score M (nxm),Each of these users only to score some project

其中大多数为 None,Indicates that the user has not rated the movie yet,heavy duty in recommender systems,Often this scoring matrix is ​​a十分稀疏的矩阵.Matrix decomposition to do is to predict matrix missing score,Makes the prediction score can reflect the extent to which the user's like.we want to decompose this matrix,使得：
$$M=X{Y}^T$$
其中Xis the user matrix（User Matrix）（n×l）,每一行代表一个用户,Y项目矩阵（Item Matrix）（l×k）,每一列代表一个物品.

这里 l User and the project of the potential space dimensions.因此,we search matrix X 和 Y,Its dot product is closest to the existing interaction.将 E expressed in pairs (i,j) 的集合,使得 $M_{ij}$ 不是空值,We want to find a minimum“Scoring error”的 X 和 Y
$$(X,Y)=min\sum _{M_{ij}\ne 0}[M_{ij}-(X_i)(Y_j{)}^T{]}^2$$
添加一个L2正则化,我们得到:
$$(X,Y)=min\frac{1}{2}\sum _{M_{ij}\ne 0}[M_{ij}-(X_i)(Y_j{)}^T{]}^2+\frac{\lambda }{2}(\sum _{i,k}(X_{ik}{)}^2+\sum _{j,k}(Y_{jk}{)}^2)$$
我们要做的就是最小化上面这个损失函数.这个时候就是一个最优化的过程.to decompose from the scoring matrixUser矩阵和Item矩阵,Only the rating matrix on the leftM是已知的,User矩阵和Itemmatrix is ​​unknown.in order to learnUser矩阵和Item矩阵,使得User矩阵*ItemMatrix and score matrix of the known minimum score difference （最优化问题）

And then can be gained by gradient descent to solve matrix X 和 Y,We can notice two things.首先,You don't have to pair at every step E All computed gradients in,We can consider only a subset of these for,以便我们“批量”Our objective function optimization.其次,X 和 Y The values ​​in do not have to be updated at the same time,And gradient descent can alternately in each step in X 和 Y 上完成（这样做,We consider a matrix to be fixed,And in the opposite for the next step by another matrix iteration before),Suppose the result is as follows：

Once the matrix is ​​decomposed,We can use less information to make a new recommendation：We can simply be user vector multiplied by any project vector,to estimate the corresponding score.请注意,We can also put the user-用户和项目-According to these new to project method for user and project：（近似）Nearest neighbor search won't conducted on large sparse vector,Instead, do it on small dense vectors,This makes easier to deal with some approximation technique.以电影推荐为例：

某个用户u对电影i的预测评分 = User向量和Item向量的内积

Multiply these two matrices,Can get each user ratings for the forecast of each movie,The higher the rating value,表示用户喜欢该电影的可能性越大,该电影就越值得推荐给用户.

Finally we can notice,The concept of this basic decomposition can be extended to more complex models,例如,A more general neural network,We can think of the first is the Boolean value interaction matrix.If we want to reconstruct the boolean interaction,Simple dot product cannot be done well.但是,If we add alogistic函数,we will get a model,它的值在 [0, 1] 中,So you can better solve the problem.在这种情况下,The model to be optimized is
$$min\frac{1}{2}\sum _{M_{ij}\ne 0}[f(X_i,Y_j^T)-M_{ij}{]}^2+\frac{\lambda }{2}(\sum _{i,k}(X_{ik}^2)+\sum _{j,k}(Y_{ik}{)}^2)$$
其中 f(.) 是logistic函数.Deeper neural network model in complex recommendation system usually get close to the most advanced performance（SOTA）.

4.模型、偏差和方差

Let's pay more attention to the main difference between the aforementioned method.Especially, let's take a look at the effects of bias and variance.
在Memory-based collaborative filtering method中,No assumptions underlying model.The algorithm deals directly with the user-项目交互：例如,Users through their interactions with the project,And use the nearest neighbor search for these said to generate Suggestions.Because there is no potential model assumptions,These low deviation method theory but high variance.
在Model-Based Collaborative Filtering中,Some underlying interaction models are assumed.该模型经过训练,From its own users and reconstruction in project said-项目交互值.Then you can according to this model puts forward new suggestion.Model to extract the user and the project potential said with mathematical meaning,Humans can't explain.Due to the assumption that the user-A model of project interaction,So compared with potential model assuming no method,In theory, this method possesses higher deviation but a lower variance.
最后,in a content-based approach中,Some underlying interaction models are also assumed.然而,在这里,Model provides a define the user and the content of the project said：例如,A user is represented by a given feature,We try to like or don't like the project for each project modeling user characteristics.在这里,For model-based filtering methods,assumes the user-Project Interaction Model.然而,The model is more constrained（Because the representation of users and items is given）,So this method has the highest deviation often but minimum variance.

5.推荐系统的评估

for any machine learning algorithm,We all need to be able to assess the performance of the recommendation system,To determine which algorithm is most suitable for our situation.Recommendation system evaluation method mainly can be divided into two groups：Based on well-defined indicators of evaluation and the main estimates based on human judgment and satisfaction assessment.

5.1Based on the evaluation index of evaluation

If our recommender system is based on输出数值的模型,such as scoring predictions or matching probabilities,We can use the error measure metric,例如均方误差（MSE）In a very classic way to assess the quality of these output. 在这种情况下,Model only on the part of the interactive data available for training,and test it on the rest of the interaction data.

If our recommender system is based on预测数值的模型,We can also use the classic method of threshold value of these values for binary（Values ​​above the threshold are positive,Values ​​below the threshold are negative）并以“分类方式”评估模型. 实际上,由于用户-Project in the past data set is binary interaction（Or can be binarized by thresholding）,So we can evaluate model on the test data set of binary output accuracy（and precision and recall）.

最后,If we now consider aNon-Numerical Recommender Systems,it just returns a list of recommendations（例如基于 knn 方法的用户-用户或项目-项目）,We can still define a precision,As we measure estimate real for the proportion of users recommend project.To estimate this accuracy,We can't consider our user didn't interact with the recommended project,We should only consider we have user feedback of test data set project.

5.2person-based assessment

在设计推荐系统时,We not only to obtain the recommended model which we pretty sure interested in,And we can also look forward to some other good features,such as diversity and interpretability of recommendations.
As mentioned in Collaborative Filtering,We want to avoid let users into restricted areas the information we have said before. “偶然性”Is often used to express the concept of model have or not have created such a restricted areas the trend of（Suggested Diversity）.偶然性,Can recommend project by calculating the distance between the estimated,should not be too low,because it creates restricted areas,But it shouldn't be too high,Because it means that we don't have enough consider when recommended users interested in Suggestions**（探索与利用） ** .Exploitation: Showcase past interests.Exploration: Content that showcases diversity.因此,To bring diversity in suggested options,We want to recommend is very suitable for our users and not very similar project.例如,instead of recommending users“start the war1,2,3”,Not as recommended“星球大战1”、“start the trek into the dark”和“Indiana Jones and the Lost Ark Raiders”：The two later may be our system as user interest less chance,但推荐 3 Look too similar projects is not a good choice.
可解释性is another key to the success of the recommendation algorithm.事实上,已经证明,If the user doesn't understand why they are recommended to specific project,They tend to lose confidence in recommender systems.所以,If we can design a clear explanation of the model,We can recommend to add a small sentence explain why recommend a project（“Like the project are also like another project”,“Do you like this project,you might be interested in this”……）.
最后,In addition to diversity and interpretability essentially beyond difficult to assess the fact that,We can notice the appraisal does not belong to the quality of the recommended test data set is very difficult：How do you know if there is a new recommendation before actual recommend to our customers is related？由于所有这些原因,有时在“real condition”It is useful to test models in.Since the goal of recommender systems is to generate behavior（看电影、购买产品、read articles etc.）,We can evaluate its ability to produce desired behavior.例如,The system can follow A/B Test method goes into production,It is also possible to test only on user samples.In the test need to set up the model a certain level of confidence.（$\alpha =0.05$）

6.总结

第一、Recommendation algorithms can be divided into two classic models：基于协同过滤方法（例如用户-用户、项目-Project and matrix decomposition）和基于内容的方法

第二、Collaborative filtering method based on memory, don't assume any潜在模型,thus has low bias but high variance;Based on the assumption of collaborative filtering model is a potential user interaction model to learn from scratch and the characteristics of the project,So it is of higher deviation but lower variance;Based on the assumption of method content around the user and a potential model/or project feature build,thus has the highest bias and lowest variance

第三、Recommendation systems are difficult to evaluate：如果可以使用 MSE、准确率、Some classic metrics such as recall or precision,you should remember,We cannot evaluate some properties in this way,such as diversity（偶然性）和可解释性;In the field（离线测试、Small batch onlineAB测试、 全流量上线）In the end is the only true way to evaluate new recommendation system,But need to set a certain confidence level

第四、我们应该注意,We have not discussed in this introductory article混合方法.These methods combining collaborative filtering and content-based methods,State-of-the-art methods can be implemented in many cases,So now in many large recommendation system using.Hybrid method to make the combination of the main two forms can be used：We can train both models independently（A collaborative filtering model and a model based on the content）,And combined with their proposal.Or directly to build a unified model of the two methods of（通常是神经网络）By using prior information（About users and projects）and interaction information as input.谢谢阅读！The author has limited knowledge,如有错误,subject to your,Welcome to correct in the comment area.

参考资料:

深度学习推荐系统.王喆
https://medium.com/towards-data-science/introduction-to-recommender-systems-6c66cf15ada