tf. keras. layers. Embedding function

The function prototype

tf.keras.layers.Embedding(input_dim,
    					  output_dim,
    					  embeddings_initializer='uniform',
    					  embeddings_regularizer=None,
    					  activity_regularizer=None,
    					  embeddings_constraint=None,
    					  mask_zero=False,
    					  input_length=None,
    					  **kwargs
)

Function description

The embedding layer is mainly responsible for converting a feature into a vector . The embedded layer is generally placed on the first layer , It is often used in the processing of natural language sequences .

As shown in the figure above , Each word corresponds to a label , such as “late” Corresponding 3、“yeah” Corresponding 8, In this way, the word sequence can be transformed into a vector , It is convenient for data processing . The Embedding The function of layer is to map each label value in the vector into a 3 Dimension vector , In this way, a three-dimensional vector can be used to represent a word .

Embedding Function realizes the function of embedded layer . Parameters input_dim Indicates the size of the vocabulary , For example, the word sequence to be processed has 100 That's ok , Every line has 50 Word , So there's a total of 5000 Word , Assume that this 5000 Different words in a word are 2000 individual , Then the vocabulary of the input data is 2000.

Parameters output_dim Represents the vector dimension of each word mapping , If you need to use 20 A dimensional vector represents a word , that output_dim for 20. There is also a common parameter input_length, This parameter is used to specify the length of the input word sequence , If the word sequence length is 30 individual , Then the value of this parameter should be set to 30. If no parameters are set input_length, Then the length of the input sequence can be changed .

Be careful ,Embedding The layer input is a two-dimensional tensor , Shape is (batch_size, input_length), The output shape is (batch_size, input_length, output_dim), It's a three-dimensional tensor .

The usage function

model = tf.keras.models.Sequential([
    #  Embedded layer , The vocabulary is 256
    tf.keras.layers.Embedding(256, 125),
    # LSTM layer 
    tf.keras.layers.LSTM(125, return_sequences=True),
    # LSTM layer 
    tf.keras.layers.LSTM(125, return_sequences=True),
    #  Fully connected in the time dimension Dense layer 
    tf.keras.layers.TimeDistributed(tf.keras.layers.Dense(256, activation="softmax"))
])

Model: "sequential"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding (Embedding)       (None, None, 125)         32000     
                                                                 
 lstm (LSTM)                 (None, None, 125)         125500    
                                                                 
 lstm_1 (LSTM)               (None, None, 125)         125500    
                                                                 
 time_distributed (TimeDistr  (None, None, 256)        32256     
 ibuted)                                                         
                                                                 
=================================================================
Total params: 315,256
Trainable params: 315,256
Non-trainable params: 0
_________________________________________________________________