当前位置：网站首页>Contrastive Learning Model Cheat Sheet (1)

Contrastive Learning Model Cheat Sheet (1)

2022-08-07 00:15:00 【Andy Dennis】

前言

Contrast learning with the fires of the past few years,I make a record here,哎,It's all because I didn't study well before.Didn't get it right~



模型

Moco

CVPR 2020 Best nomination.



θ k = m θ k − 1   +   ( 1 − m ) θ q \theta_k = m \theta_{k-1} ~+ ~(1-m)\theta_q θk​=mθk−1​ + (1−m)θq​

论文: Momentum Contrast for Unsupervised Visual Representation Learning
代码: https://github.com/facebookresearch/moco

One of He Kaiming's masterpieces.
Refined thesis MoCo 论文逐段精读【论文精读】.To understand contrastive learning,Some common proxy tasks need to be known first.如instance discrimination, Introduced in this video(4:28时刻)

infoNCE
l o s s = − l o g e x p ( q ⋅ k + / τ ) ∑ i = 0 K e x p ( q ⋅ k + / τ ) loss = -log\frac{exp(q\cdot k_+ /\tau)}{\sum_{i=0}^Kexp(q\cdot k_+ /\tau)} loss=−log∑i=0K​exp(q⋅k+​/τ)exp(q⋅k+​/τ)​
其中K是负样本个数(i=0是正样本,i=1…K是负样本), τ \tau τ是温度系数

原网站

版权声明
本文为[Andy Dennis]所创，转载请带上原文链接，感谢
https://yzsam.com/2022/219/202208070010420846.html

边栏推荐

猜你喜欢

随机推荐