一、The bias and variance of the estimator（Bias and Variance of Estimator）

• 误差（error）的来源：Bias and variance（Bias and Variance）
• 估计量（Estimator）：We found based on the training set $f^{\ast }$ 就是真实$\hat{f}$的估计量
• 估计变量x的均值
  • 假设x的均值是u
  • 假设x的方差是${\sigma }^2$
• 估计均值u
  • 取N个点：{ $x^1,x^2,...,x^N$}
  • $m=\frac{1}{N}\sum _n{x}^n\ne u$
  • $E\left[m\right]=E\left[\frac{1}{N}\sum _n{x}^n\right]=\frac{1}{N}\sum _{n}E\left[x^n\right]=u$,虽然每一个mare spread evenly in the realu的四周,But after taking the average of them, it is equalu
  • 散布程度：$Var[m]=\frac{{\sigma }^2}{N}$,Var为取m的方差,Its value depends on the number of samples.N越大,The tighter the spread;N越小,Scatter more sparsely.
• 估计方差${\sigma }^2$
  • $s^2=\frac{1}{N}\sum _n(x^n-m{)}^2$
  • $E[s^2]=\frac{N-1}{N}{\sigma }^2\ne {\sigma }^2$,N越大,估计量$s^2$with the true variance${\sigma }^2$the smaller the difference.

1.1 方差（Variance）

• 越简单的模型,方差越小;越复杂的模型,方差越大
• Simple models are less affected by the data

1.3 偏置值（Bias）

• 越简单的模型,The larger the offset value is;越复杂的模型,The smaller the offset value
• 对于简单的模型,Because it contains a relatively small set of functions,May not contain the real function at all,So no matter how you look for it,No real model could be found either.That's why the simpler model,The reason for its larger bias value.

1.4 模型优化方向

• When the variance of the model（Variance）比较大时,The model will overfit（Overfitting）.
  • If the model fits well on the training set,And the performance on the test set is very poor,Then the variance is large,过拟合
  • 增大数据集：For example, take more sample data,Or perform data enhancement on the original data,This does not change the bias value of the model（推荐）
    • Handwritten digit recognition：Angular flip the data in the training set
    • 影像辨识：Angular flip the data in the training set
    • 语音辨识：If only boys data,There are no data for girls,You can use a voice changer to convert boys' data into girls' data.If the data is not disturbed by noise,In the real scene, there is noise interference.Then the noise of the scene can be recorded and synthesized with the training set,Then the data in the real scene is obtained.
    • 语音翻译：When there is only data in one national language in hand,却想做10language translation tasks,Then, data in multiple languages ​​can be obtained in the form of machine translation
  • 正则化（Regularization）：Add a term after the loss functionterm,to reduce the sensitivity of the model to the input,Doing so will reduce the model space,It will also increase the bias value of the model.（不推荐）
• When the bias value of the model（Bias）比较大时,The model does not fit（Underfitting）.
  • If the model fits poorly on the training set,Then the bias value is larger,does not fit
  • At this point it is time to redesign our model,For example, increasing the complexity of the model：考虑更多的特征,Consider multiple powers for a single feature

二、交叉验证（Cross Validation）

• 交叉验证
  • Divide the original training set into two,Divided into real training set and validation set.That is, don't optimize your model by exposing the results on the test set,Instead, just optimize your model by the results on the validation set,This way the model does not overfit,It also better reflects the real situation of your model.
• N次交叉验证（N-fold Cross Validation）
  • Divide the original training set into three,One piece of data is taken each time as a validation set,The other two are used as training set.Calculations for different models are given below3The average loss value in each case,Publish the best performing model to the public test set.