LSTM(Long Short Term Memory) and RNN are both neural networks that infer future values from current data for time-series, meaningful data.
LSTM can predict longer time series data than RNN. The comparison between RNN and LSTM is below.
■Overview of LSTM
LSTM has a cell Ct that stores past information, and the output is determined by this cell and output gate.
Cells are also determined by the forget gate layer and input gate layer, and the forget gate layer allows you to decide how much past value is erased.
＜forget gate＞
The forget gate is a sigmoid function that retains past memories when it reaches 1 and erases past memories when it goes to 0.
Therefore, if the current value has a strong correlation, it will be retained, and if the correlation with the past value is weak, it will be deleted.
＜input gate＞
Update the cell value on the input gate.
The forget gate only attenuates the value of the cell, so it does not reflect the current value.
Although the formula may seem complicated at first glance, it is easier to understand if you visualize the weighted moving average formula as shown below.
＜output gate＞
The final output is determined by multiplying the cell Ct by the output gate value.
■Example of implementing LSTM in python
The future movement is predicted from the data representing the damped vibration of the spring as shown below. Since LSTM allows long-term prediction, set the prediction period to be long.
＜simulation result＞ Use pytorch's LSTM function here.
The results are as follows. The middle layer has 128 nodes and 50,000 epochs.
We also include the results obtained by RNN for the same data. In this case, the prediction accuracy of LSTM was slightly better than RNN using tanh, but worse than RNN using relu.
It is possible that LSTM will give better results if you change the number of middle layers, etc., but this may not necessarily mean that LSTM is better.
＜Note＞
In the above program, you can run RNN by making the following changes.
TOP MENU
