How to use nn.Embedding (pytorch)

TOP MENU

Python

library
・pip

・MeCab

numpy
・digitize

・mgrid

・pad

・polyfit

・prod

・shape

matplotlib
・figure

・pcolormesh

・scatter

pytorch
・BCELoss, MSELoss

・device

・Embedding

・TensorDataset, Dataloader

・RNN, LSTM

scikit-learn
・SVC

scipy
・interpolate

tkinter
・postscript

・image display

・frame, grid

other
・linear interpolation

OpenAI gym
・CartPole-v0

Release date:2023/5/27　　　　　　　　　

・In Japanese
＜premise knowledge＞
・perceptron

■Description of Embedding function

Vectorize the words that make up a sentence. This is called word embedding. For example, when embedding a word in the sentence "Today is sunny", each word is broken down and given an ID as shown below, and expressed as one hot.

The embedded layer becomes a perceptron operation as follows. For example, for the word "sunny", 1 is input to the third node of the input layer, and the perceptron operation is performed and output.

■Concrete example of embedding function

The initial values of the perceptron parameters are random values. The parameter values are updated by learning.

import torch
import torch.nn as nn

vocab_size = 3
embed_dim = 4

embed = nn.Embedding(vocab_size, embed_dim)
emb = embed(torch.tensor([2]))     # The value to be entered is up to vocab_size (0 to 2)
print(emb)

⇒ tensor([[-0.4139, 0.4187, 1.4098, 0.1499]], grad_fn=＜EmbeddingBackward＞)

emb = embed(torch.tensor([2,1]))     # You can enter multiple numbers
print(emb)

⇒ tensor([[ 0.5731, 0.6799, 0.0471, 0.9301],
                [ 0.9365, -1.6096, 2.3542, 0.2670]], grad_fn=＜EmbeddingBackward＞)

List of related articles