Reading notes: meta matrix factorization for federated rating predictions

Meta Matrix Factorization for Federated Rating Predictions

1. What does literature study?

1. design a novel federated learning framework for rating prediction(RP) for mobile environments, introduce MetaMF to generate private item embeddings and RP models with a meta network.
2. main: how to reduce the model scale of matrix factorization methods in order to make them suitable for a federated environment.

2. What’s the innovation?

Past shortcomings

1. Ammad-ud din et al.FCF do not focus on the size of the local models while maintaining performance; importantly, they focus on the ranking tasks, not the rating prediction task.
2. in previous federated learning methods, the global model in the server and the local model in the device have the same size, the local model is a copy of the global model.

Innovation

1. introduce a meta matrix factorization to generate private item embeddings and RP models with a meta work.
2. employ a meta recommender module to generate private item embeddings and a RP model based on the collaborative vector in the server.
3. devise a rise-dimensional generation strategy {that first generates a low-dimensional item embedding matrix and a rise-dimensional matrix, and then multiply them to obtain high-dimensional embeddings} to address the challenge of generation a large number of high-dimensional item embeddings.

3. What was the methodology?

An overview of MetaMF.it consists of three modules. CM module( Cooperative memory module ), MR module with the RG strategy and prediction module.

1. design for MetaMF that deploys a big meta network into the server to exploit CF while deploying a small RP model into the device to predict ratings.

2. MetaMF provides private RPs by generating non-shared and small models as well as item embeddings for individual users.

3. 3.6 loss function is defined as: $$L_{rp}=\frac{1}{|D_{train}|}\sum _{r_u{,}_i\in D_{train}}(r_{u,i}-{\hat{r}}_{u,i}{)}^2$$
   then add the L2 regularization term: $$L_{reg}=\frac{1}{2}{\Vert \Theta \Vert }_2^2$$
   the final loss L is : $$L=L_{rp}+\lambda L_{reg}$$, $$\lambda$$is the weight of $$L_{reg}$$.
   Loss: Note that unlike existing MF methods, the item embeddings and the parameters of RP models are not included in $$\Theta$$, because they are also the outputs of MetaMF, not trainable parameters.

4. Algorithm 1 MetaMF
   

5. datasets and baseline should not be stated?

4. What are the conclusions?

1. on the Douban and Hetrec-movielens datasets, MetaMF outperforms most baselines despite the fact that it is federated while most baselines are centralized. on the Movielens1M and Ciao datasets, MetaMF does not perform well.
2. generating private item embeddings and private RP models for each user can improve the performance of MetaMF. And item embeddings have a greater impact on the performance of MetaMF than RP models.
3. As the model scale increases, there is not sufficient private data to train too many parameters for each user, larger model scales easily lead MetaMF to overfit. For RG strategy, it is unrealistic to generate larger embeddings for too many items .

5. others

1. code of NCF:
   https://github.com/hexiangnan/neural_collaborative_filtering.
2. AutoRec(autoencoders): https://github.com/gtshs2/Autorec.
3. LibRec: https://www.librec.net/.