当前位置:网站首页>Pytorch: a pit about the implementation of gradreverselayer
Pytorch: a pit about the implementation of gradreverselayer
2022-04-23 12:54:00 【Qing San】
I've been working on one recently domain adaptation Project , Yes gradreverselayer:`class GradReverse(Function):
def forward(ctx, x, alpha, **kwargs:None):
ctx.alpha = alpha
return x.view_as(x)
def backward(ctx, grad_output):
output = grad_output * -ctx.alpha
return output, None
There seems to be no problem , But keep reporting the wrong :
I haven't done it for a long time , So I made a stupid decision , I added one :
src_domain_loss=torch.tensor(src_domain_loss, requires_grad=True)
As a result, the error disappeared , I happily used this silly * The code took two months , Results the experimental results have been unsatisfactory , Until today, check the training gradient , It is found that the gradient is always zero !!!
Speechless , It must be the pot of this code , But without it, an error will be reported , I searched every corner of the Internet , No solution has been found , And paid people to consult ( cry ).
result , A flash of light , I found something missing in the code of gradient flip layer :
class GradReverse(Function):
@staticmethod
def forward(ctx, x, alpha, **kwargs:None):
ctx.alpha = alpha
return x.view_as(x)
@staticmethod
def backward(ctx, grad_output):
output = grad_output * -ctx.alpha
return output, None`
Have you found it? ?@staticmethod!!! I always thought this was a comment , Delete... Directly , Great speechless event , cause sb. to die of anger !!!
In fact, this is declaring a static method !
I hope you will take a warning .( cry )
版权声明
本文为[Qing San]所创,转载请带上原文链接,感谢
https://yzsam.com/2022/04/202204230616251505.html
边栏推荐
- Flash project cross domain interception and DBM database learning [Baotou cultural and creative website development]
- [csnote] ER diagram
- 标签与路径
- NBIOT的AT指令
- Remote sensing image classification and recognition system based on convolutional neural network
- leetcode-791. Custom string sorting
- Bert base Chinese Download (SMART)
- SSL证书退款说明
- 精度、速度完美平衡,最新图像分割SOTA模型重磅发布!!!
- Process virtual address space partition
猜你喜欢
How to prevent the website from being hacked and tampered with
5 free audio material websites, recommended collection
Unable to create servlet under SRC subfile of idea
4.DRF 权限&访问频率&过滤&排序
实现一个盒子在父盒子中水平垂直居中的几种“姿势”
World Book Day: I'd like to recommend these books
The accuracy and speed are perfectly balanced, and the latest image segmentation SOTA model is released!!!
SSL certificate refund instructions
Homomorphic encryption technology learning
基于卷积神经网络的遥感影像分类识别系统
随机推荐
Analysis of InnoDB execution process in MySQL
STM32控制步进电机(ULN2003+28byj)
Ad20 supplementary note 3 - shortcut key + continuous update
leetcode-791. 自定义字符串排序
洛谷P3236 [HNOI2014]画框 题解
0基础可以考CPDA数据分析师证书吗
Kubernets Getting started tutoriel
Unlock openharmony technology day! The annual event is about to open!
NPDP|产品经理如何做到不会被程序员排斥?
洛谷P5540 [BalkanOI2011] timeismoney | 最小乘积生成树 题解
The quill editor image zooms, multiple rich text boxes are used on one page, and the quill editor upload image address is the server address
flask项目跨域拦截处理以及dbm数据库学习【包头文创网站开发】
有趣的IDEA插件推荐,给你的开发工作增添色彩
Uni app native app local packaging integrated Aurora push (jg-jpush) detailed tutorial
Realize several "Postures" in which a box is horizontally and vertically centered in the parent box
Get the punch in record of nailing attendance machine
甲辰篇 創世紀《「內元宇宙」聯載》
mysql中 innoDB执行过程分析
8 websites that should be known for product development to enhance work experience
【每日一题】棋盘问题