当前位置：网站首页>NLLLoss+log_ SoftMax=CE_ Loss

NLLLoss+log_ SoftMax=CE_ Loss

2022-04-23 09:17:00 【365JHWZGo】

List of articles

subject

NLLLoss+log_SoftMax=CE_Loss

Preface

It hasn't been updated for several days , alas , A little busy recently , Please forgive me ！ Today, I finally stole some time to watch B Station video discovery NLLLoss+log_SoftMax=CE_Loss, ha-ha , I've seen it again , So let's take a closer look today ！

NLLLoss

chinese

Negative log likelihood loss

function

torch.nn.NLLLoss(
	weight=None, 
	size_average=None, 
	ignore_index=- 100, 
	reduce=None, 
	reduction='mean'
)

Parameters,

Parameters	type	meaning
weight	tensor(optional)	Manually readjust the weight proportion of each class
size_average	bool(optional) Have been abandoned	Loss is a batch The average value of each loss element in
ignore_index	int(optional)	Ignoring a target value does not affect the gradient of the input
reduce	bool(optional) Have been abandoned	According to each mini-batch The average size of the loss is divided equally or summarized
reduction	string(optional)	reuduction Will be applied to the output

reduction Parameters	explain
none	Expand all
mean	Add up / Number
sum	hold none The results add up

Function input and output

	Input	The goal is	Output
type	tensor	tensor	tensor
dimension	（ $N, C$ ）/（ $C$ ）	（ $N$ ）/（）	（ $N$ ）/ $N, d_1, d_2, ..., d_K)$

Code

import torch 
#  Imitate the output after passing through the model 
preds = torch.tensor([[1.5,2.5,3.0]])
#  Real label 
target = torch.tensor([1])

nllloss = torch.nn.NLLLoss()
print('nllloss:',nllloss(preds,target))

And CE_Loss The difference and connection

difference ：

CE_Loss It's going through first log_softmax after NLLLoss Loss of steps
NLLLoss It just corresponds to index Of target The value becomes the opposite number

contact ：

There is a certain correlation , after softmax Most of the subsequent values are [0,1] Between , It's going on log After taking the logarithm, the value is distributed in ( $-\infty$ ,0] Between , and loss It can't be negative , So we need to NLLLoss To make it positive

import torch 
#  Imitate the output after passing through the model 
preds = torch.tensor([[1.5,2.5,3.0]])
#  Real label 
target = torch.tensor([1])

cross_entropy_loss = torch.nn.CrossEntropyLoss()
log_softmax = torch.nn.LogSoftmax(dim=1)
nllloss = torch.nn.NLLLoss()

cs_loss = cross_entropy_loss(preds,target)
nls_loss = nllloss(log_softmax(preds),target)
print(f' The cross entropy loss function is :{
      cs_loss}\n Go first log_softmax after nll The loss function is :{
      nls_loss}')