torch.nn.Embedding

function ：

torch.nn.Embedding(num_embeddings, embedding_dim, padding_idx=None, max_norm=None, norm_type=2, scale_grad_by_freq=False, sparse=False)

Parameter interpretation ：

1. num_embeddings: The size of the query table
2. embedding_dim: The dimension of each query vector

Approximate explanation of function ： A simple lookup table with a fixed dictionary and size . This module is often used to save word embedding and retrieve them with subscripts . The input to the module is a list of subscripts , The output is the corresponding word embedded . It's equivalent to randomly generating a tensor, Think of it as a query table , Its size by [num_embeddings, embedding_dim]. among num_embeddings Is the size of the query table ,embedding_dim Is the dimension of each query vector .

Popular speaking , This is a matrix class , It initializes a random matrix , The size of the long-term Dictionary of the matrix , Width is used to represent the attribute vector of each element in the dictionary , The dimension of the vector depends on the complexity of the element you want to represent . After class instantiation, you can find the vector corresponding to the element according to the subscript of the element in the dictionary . Input subscript 0, Output is embeds In matrix 0 That's ok .

Specific examples ：

1. Create a query matrix and use it to do Embedding:

embedding = nn.Embedding(5,3) # Define a with 5 Word , Dimension for 3 Query matrix 
print(embedding.weight) # Show the details of the matrix 
test = torch.LongTensor([[0, 2, 0, 1],
						[1, 3, 4, 4]]) # The test The matrix is used to be embed, Its size by [2,4]
#  The first act [0, 2, 0, 1],  Indicates to get the information in the query matrix ID by 0, 2, 0, 1 The query vector of 
#  It can be later test Output medium and embed Compare the output of 
test = embedding(test)
print(test.size()) # Output embed after test Of size,  by [2, 4, 3], To increase the 3, Because the dimension of the query is 3
print(test) #  Output embed After test The content of 

 Output ：
Parameter containing:
tensor([[-1.8056,  0.1836, -1.4376],
        [ 0.8409,  0.1034, -1.3735],
        [-1.3317,  0.8350, -1.7235],
        [ 1.5251, -0.2363, -0.6729],
        [ 0.4148, -0.0923,  0.2459]], requires_grad=True)
torch.Size([2, 4, 3])
tensor([[[-1.8056,  0.1836, -1.4376],
         [-1.3317,  0.8350, -1.7235],
         [-1.8056,  0.1836, -1.4376],
         [ 0.8409,  0.1034, -1.3735]],

        [[ 0.8409,  0.1034, -1.3735],
         [ 1.5251, -0.2363, -0.6729],
         [ 0.4148, -0.0923,  0.2459],
         [ 0.4148, -0.0923,  0.2459]]], grad_fn=<EmbeddingBackward>)			

It can be seen that ,embedding It is equivalent to using the index of a matrix to query the characteristics of the elements in the original embedded matrix .

2. Find a specific in the query matrix ID（ word ） The query vector of （ The word vector ）：

#  Visit a ID, That is to say N A query vector of words 
print(embedding(torch.LongTensor([3]))) #  This means that the query No 3 The word vector of a word 

 Output ：
tensor([[-1.6016, -0.8350, -0.7878]], grad_fn=<EmbeddingBackward>)

3. Output hello The word word embedding

import numpy as np
import torch
import torch.nn as nn
import torch.nn.functional as F
from torch.autograd import Variable

word_to_ix = {
    'hello':0, 'world':1}
embeds = nn.Embedding(2,5)
hello_idx = torch.LongTensor([word_to_ix['hello']])
hello_idx = Variable(hello_idx)
hello_embed = embeds(hello_idx)
print(hello_embed)

Output results ：

Variable containing:
 0.4606  0.6847 -1.9592  0.9434  0.2316
[torch.FloatTensor of size 1x5]