Foreword

This chapter is actually something that has not been done before, let's make it up, the two are actually not related

Transfer Learning

The following content is learned from Transfer Learning [Stanford Fall 21: Practical Machine Learning ChineseVersion]

What does transfer learning include?

①feature extraction
②train a model on a relate task and reuse it
③fine-tuning from a pertrained model

fine-tuning

(1) Why use a preserved model?

Conducive to accelerating convergence, may improve accuracy

I used a picture from the class, a bit abstract butThis is the truth
Red pentagram: the optimal solution (good generalization performance)
Blue pentagram: obtained solution (the result obtained with poor generalization performance)

(2)How to avoid

①Limit the training range
Give a training radius, that is, the green circle in the picture above

②Freeze training
In fact, what we really want to train is the last fully connected layer of the entire model, which is randomly initialized at this time

Kamin initialization

Derivation of kaiming initialization