当前位置:网站首页>Pytorch: a pit about the implementation of gradreverselayer
Pytorch: a pit about the implementation of gradreverselayer
2022-04-23 12:54:00 【Qing San】
I've been working on one recently domain adaptation Project , Yes gradreverselayer:`class GradReverse(Function):
def forward(ctx, x, alpha, **kwargs:None):
ctx.alpha = alpha
return x.view_as(x)
def backward(ctx, grad_output):
output = grad_output * -ctx.alpha
return output, None
There seems to be no problem , But keep reporting the wrong :
I haven't done it for a long time , So I made a stupid decision , I added one :
src_domain_loss=torch.tensor(src_domain_loss, requires_grad=True)
As a result, the error disappeared , I happily used this silly * The code took two months , Results the experimental results have been unsatisfactory , Until today, check the training gradient , It is found that the gradient is always zero !!!
Speechless , It must be the pot of this code , But without it, an error will be reported , I searched every corner of the Internet , No solution has been found , And paid people to consult ( cry ).
result , A flash of light , I found something missing in the code of gradient flip layer :
class GradReverse(Function):
@staticmethod
def forward(ctx, x, alpha, **kwargs:None):
ctx.alpha = alpha
return x.view_as(x)
@staticmethod
def backward(ctx, grad_output):
output = grad_output * -ctx.alpha
return output, None`
Have you found it? ?@staticmethod!!! I always thought this was a comment , Delete... Directly , Great speechless event , cause sb. to die of anger !!!
In fact, this is declaring a static method !
I hope you will take a warning .( cry )
版权声明
本文为[Qing San]所创,转载请带上原文链接,感谢
https://yzsam.com/2022/04/202204230616251505.html
边栏推荐
- Stm32cubeprogrammer basic instructions
- SSM框架系列——Junit单元测试优化day2-3
- Hard core parsing promise object (do you know these seven common APIs and seven key questions?)
- 如何防止网站被黑客入侵篡改
- flask项目跨域拦截处理以及dbm数据库学习【包头文创网站开发】
- Uni app native app local packaging integrated Aurora push (jg-jpush) detailed tutorial
- [daily question] chessboard question
- leetcode-791. 自定义字符串排序
- 标签与路径
- SSL证书退款说明
猜你喜欢
Record the problems encountered in using v-print
免费试用一个月的服务器,并附上教程
Process virtual address space partition
The accuracy and speed are perfectly balanced, and the latest image segmentation SOTA model is released!!!
Kubernetes 入門教程
Remote access to raspberry pie at home (Part 1)
教你快速开发一个 狼人杀微信小程序(附源码)
mysql8安装
Redis deployment of cloud native kubesphere
Object. The disorder of key value array after keys
随机推荐
有趣的IDEA插件推荐,给你的开发工作增添色彩
No idle servers? Import OVF image to quickly experience smartx super fusion community version
Ad20 supplementary note 3 - shortcut key + continuous update
只是不断地建构平台,不断地收拢流量,并不能够做好产业互联网
leetcode-791. Custom string sorting
SSM框架系列——Junit单元测试优化day2-3
Teach you to quickly develop a werewolf killing wechat applet (with source code)
Servlet监听器&过滤器介绍
Trier les principales utilisations de l'Agent IP réseau
如何防止网站被黑客入侵篡改
Calculate the past date and days online, and calculate the number of live days
Kubernets Getting started tutoriel
STM32工程移植:不同型号芯片工程之间的移植:ZE到C8
NPDP|产品经理如何做到不会被程序员排斥?
实现一个盒子在父盒子中水平垂直居中的几种“姿势”
Web17——EL与JSTL的使用
Analysis of InnoDB execution process in MySQL
8 websites that should be known for product development to enhance work experience
Uni app native app local packaging integrated Aurora push (jg-jpush) detailed tutorial
leetcode:437. Path sum III [DFS selected or not selected?]