Orthogonal-Uncorrelated-Independent

This article describes orthogonality in random variables、不相关、Separate distinctions and connections.

概述

All three are concepts that describe the relationship between random variables,It seems that both can represent the distant relationship between two random variables,But the definitions and constraints are different.

• 考察m维随机变量X,Y之间的关系.

定义

正交

定义R(X, Y) = E[XY]is the correlation function：若R(X, Y)=0,称X,Y正交

不相关

定义 E[XY] = E[X]E[Y],则X,Y不相关

• X,Y的协方差：

Uncorrelation can also be used as covariance0表示

• X,Y的相关系数：

The correlation coefficient can also be used for uncorrelation0表示

独立

Independence is generally represented by their probability density function.The joint distribution is equal to the product of their respective independent marginal distributions,independent：

关系

独立 -> 不相关

Independence is a more stringent requirement for variables,如果两个随机变量独立,must not be relevant,In other words, independence is a sufficient and unnecessary condition for irrelevance.

• 若已知X,Y联合概率密度f(x, y)is equal to the edge density function of bothg(x), h(y)的乘积,则有：

So the independent variables are not correlated,In contrast, uncorrelation cannot directly lead to independence

不相关 --高斯分布–> 独立

When the random variable follows a Gaussian distribution,Uncorrelated can be deduced to be independent：

• We now consider a slightly more complicated situation,X为n维随机变量：

• There is no correlation between random variables,并且服从高斯分布：

• 那么此时X的联合概率密度函数为：

• 其中{\bf{\Sigma } }为协方差矩阵,Because there is no correlation between random variables：

• 其中\sigma_i 为x_i的标准差,Then the joint probability density function can be written as ：

• Therefore, when the random variable follows a Gaussian distribution,不相关与独立等价,互为充要条件.

正交 – 不相关

• 根据定义可以得知： 当E[X],E[Y]至少有一个为0When orthogonal is equivalent to uncorrelated.

参考资料

• http://blog.sciencenet.cn/blog-812827-1096465.html
• https://blog.csdn.net/wjj5881005/article/details/53320403/
• https://www.zhihu.com/question/26583332/answer/33327497
• https://baike.baidu.com/item/Uncorrelated random variables/19126995?fr=aladdin