Ridge regression and LASSO regression

Let's talk about the concepts of generalization and regularization. Generalization refers to the performance of a trained machine learning model when dealing with unencountered samples.That is, the ability of the model to process new samples.Many times the model is as fierce as a tiger in the training set, but it is outrageous (high error rate) in the test set, that is, the generalization ability is poor.Because the model only learns the characteristics of the data in the training set. For example, the training set is all cats in the daytime. It is very likely that if you give a photo of a cat in the night, it will not be able to distinguish it. This is also called overfitting.In order to prevent over-fitting and improve generalization ability, regularization came into being. It refers to adding some rules (restrictions) to the objective function that needs to be trained, that is, adding a regular term to the loss function..The commonly used regularization methods are divided into L1 regularization, L2 regularization, and the regularization terms used represent L1 norm and L2 norm respectively.For more details, please refer to this Second to understand regularization.

Adding a regular term after the original loss function can reduce the parameters learned by the model$w$, which can make the model more generalizable.
A linear model that performs L1 norm regularization on the parameter space is called LASSO Regression (LASSO Regression);
L2 norm on the parameter spaceThe linear model of number regularization is called Ridge Regression.

Difference between Ridge Regression and LASSO Regression:
With Regular Variables$\lambda$ change, based on ridge regressionstrong>The fitting curve of the improved polynomial regression algorithm is always a curve, and it is difficult to obtain a sloping straight line; the fitting curve based on the improved polynomial regression algorithm based on LASSO regressionThe resultant curve will quickly turn into a sloping curve and eventually an almost horizontal straight line.