当前位置:网站首页>4.1 - Support Vector Machines
4.1 - Support Vector Machines
2022-08-11 07:47:00 【A big boa constrictor 6666】
文章目录
The binary classification problem from the previous chapter
- Because the original picture on the right is yellowlossFunctions cannot do gradient descent,So we make an approximation to it(Approximation),用 l l l 来取代 δ δ δ ,此时的 l l l Many different functions can be used,比如:方差,Sigmoid+方差,Sigmoid+交叉熵(Cross entropy),铰链损失(Hinge loss)
y
- When there are outliers in the data(Outlier)时,铰链损失(Hinge loss)Often better than cross entropy(Cross entropy)表现得更好
一、铰链损失(Hinge loss)
- According to the derivation process in the figure below,SVM的lossThe function can be solved by gradient descent,最后并将SVMConverted to a common expression in textbooks.
二、核方法(Kernel Method)
- 对偶表示(Dual Representation):在SVM中, α n ∗ \alpha_n^* αn∗May be sparse,means there is some α n ∗ = 0 \alpha_n^*=0 αn∗=0的xn,而那些 α n ∗ ≠ 0 \alpha_n^*\neq 0 αn∗=0的xn就是支持向量(support vector).这些不是0The points ultimately determine the quality of our entire model,This is also why some outliers in the data are hard to matchSVMcause of the impact.
- 核函数(Kernel Fountion):右图中 K ( x n , x ) K(x^n,x) K(xn,x)就是核函数,也就是做 x n 和 x x^n和x xn和x的内积(inner product)
- 核方法(Kernel Trick):当我们的lossThe function can be written as the blue line on the left,We just need to calculate K ( x n ′ , x n ) K(x^{n'},x^n) K(xn′,xn),And you don't need to know the vectorx的具体值.This is the benefit of the nuclear approach,他不仅可以应用在SVM上,It can also be applied to linear regression and logistic regression.
- We can see in the derivation of the right figurex与zThe inner product after the feature transformation is very complicated,We don't need to do this when we use the kernel method,直接对x,zIt can be squared after inner product.
2.1 径向基函数核(Radial Basis Function Kernel)
- 当x与zmore like,其Kernel值就越大.如果x=z,值为1;x与zcompletely different,值为0.
- It is easy to see from the derivation of the formula in the figure belowRBF KernelIt is to do things on an infinitely multidimensional plane,Therefore, the complexity of the model will be very high,This is very easy to overfit.
2.2 Sigmoid Kernel
- Do it in the picture on the leftSigmoid Kernel时,There is only one hidden layer network,And the weight of each neuron is a piece of data,The number of neurons is the number of support vectors.
- The figure on the right explains how to directly design a kernel functionK(x,z)来代替Φ(x)和Φ(z),以及通过Mercer’s theoryto check whether the kernel function meets the requirements.
三、Support vector machine related methods(SVM related methods)
SVR(支持向量回归):When the difference between the predicted value and the actual value is within a certain range,loss=0
Ranking SVM:When something to consider is an orderinglist时
One-class SVM:He wants to belongpositive的exampleare all in the same category,negative的examplescattered elsewhere
下图是SVMand deep learning similarities between the two
边栏推荐
- Redis source code: how to view the Redis source code, the order of viewing the Redis source code, the sequence of the source code from the external data structure of Redis to the internal data structu
- break pad源码编译--参考大佬博客的总结
- Implementation of FIR filter based on FPGA (5) - FPGA code implementation of parallel structure FIR filter
- 如何选择专业、安全、高性能的远程控制软件
- oracle19c does not support real-time synchronization parameters, do you guys have any good solutions?
- 基于FPGA的FIR滤波器的实现(5)— 并行结构FIR滤波器的FPGA代码实现
- 接入网、承载网、核心网是什么,交换机路由器是什么、这个和网络的协议有什么关系呢?
- 3.2-分类-Logistic回归
- 1091 N-自守数 (15 分)
- 2022-08-10 第四小组 修身课 学习笔记(every day)
猜你喜欢
【软件测试】(北京)字节跳动科技有限公司二面笔试题
tf.cast(), reduce_min(), reduce_max()
1071 小赌怡情 (15 分)
Implementation of FIR filter based on FPGA (5) - FPGA code implementation of parallel structure FIR filter
接口测试的基础流程和用例设计方法你知道吗?
【latex异常和错误】Missing $ inserted.<inserted text>You can‘t use \spacefactor in math mode.输出文本要注意特殊字符的转义
prometheus学习4Grafana监控mysql&blackbox了解
Serverless + domain name can also build a personal blog? Really, and soon
Redis源码:Redis源码怎么查看、Redis源码查看顺序、Redis外部数据结构到Redis内部数据结构查看源码顺序
一张图了解JVM八大原子操作
随机推荐
Redis source code: how to view the Redis source code, the order of viewing the Redis source code, the sequence of the source code from the external data structure of Redis to the internal data structu
常见激活函数及其导数
Strongly recommend an easy-to-use API interface
关于Excel实现分组求和最全文档
测试用例很难?有手就行
你是如何做好Unity项目性能优化的
2022年中国软饮料市场洞察
联想集团:2022/23财年第一季度业绩
Trill keyword search goods - API
opencv实现数据增强(图片+标签)平移,翻转,缩放,旋转
Tf中的平方,多次方,开方计算
js判断图片是否存在
LeetCode刷题系列 -- 46. 全排列
1051 复数乘法 (15 分)
【Pytorch】nn.ReLU(inplace=True)
Service的两种状态形式
1003 我要通过 (20 分)
prometheus学习5altermanager
[Recommender System]: Overview of Collaborative Filtering and Content-Based Filtering
Pico neo3在Unity中的交互操作