Classification and regression tree of machine learning

One . Random forests RF

RF It's a classification tree , It is composed of many decision trees , There is no correlation between different decision trees

1. A fixed number of samples and features are randomly selected to build a decision tree , There's a sample put back .
2. The classifier is trained for each decision tree
3. Average the percentage of predicted results for each tree , Select the result of the largest proportion as the final prediction result
4. Video teaching
   https://www.bilibili.com/video/BV1H5411e73F/?spm_id_from=333.788.recommend_more_video.-1

Two . GBDT

GBDT It's the return tree , The input result of the current... Tree

1. The decision tree is constructed according to the numerical characteristics （ Such as age ）
2. Find the average value of each node
3. Find the difference from the average
4. Build a residual tree through the difference
5. Continuously fit the residual results , Make the residual smaller through iteration
6. The core is to accumulate the results of all trees as the final result

Introduction example , You can see the example in this video , Pretty good ：
https://www.bilibili.com/video/BV1Bt411j7KH?from=search&seid=13485462299533879409&spm_id_from=333.337.0.0

3、 ... and . GBDT + LR

Just like its name ,GBDT+LR It's made up of two parts , among GBDT For training sets The extracted features , These features are used as new training input data ,LR Classifier as input data for new training .

Four . XGBOOST

https://blog.csdn.net/u011094454/article/details/78948989

5、 ... and . Regularization concept

Regularization is to prevent overfitting .
This is very well said ：
https://www.cnblogs.com/jianxinzhou/p/4083921.html

6、 ... and . bagging and boosting summary , More complete

https://blog.csdn.net/u014114990/article/details/50948079