LSTM 预测连续值

如何解决LSTM 预测连续值

您好，我是 LSTM 模型的新手，我需要一些帮助。 我有一个数据集，其中我的 X 向量中有一些单词，我想使用 LSTM 嵌入层来预测我的 y 向量，该向量包含 0-1 之间的值。我看过很多例子，其中 y 向量由类组成，我们得到了一个分类问题，但在这个回归问题中我找不到解决方案。
我在下面使用此代码。首先我导入一些库，然后我尝试使用一个简单的 LSTM 深度网络来预测 y。此外，如您所见，我使用 keras 嵌入层来预测 y 值

import numpy as np
from tensorflow.keras.callbacks import EarlyStopping
from tensorflow.keras.preprocessing.text import Tokenizer
from tensorflow.keras.preprocessing.sequence import pad_sequences
from tensorflow.keras.layers import Input,Embedding,LSTM,SpatialDropout1D,Dense,Dropout
from tensorflow.keras.models import Model,Sequential```
you can see the X and y vectors

X_train

array(['sinus surgery yesterday recover binging game throne start beginning hope not long mend','respect game throne good fight scene week barry like far generally amazing episode television bill hader direction win emmi offend wanton consumerism game throne oreos going eat nonstop remove face earth face end humanity human game throne wait night battle actually lie plan realistic far type game throne viewer happen happen end big winner game throne viewer sibling write article night game throne episode social medium absolute core game throne experience point watch thing time woman quickly accumulate power rule man mistake dead uncontrollable kid breathe fire game throne realistic family nerd weirdo end run world game throne basically story silicon valley game throne completely wipe trump trend story list winter feel good english major let offer insight subtle foreshadowing game throne threat winter come battle winterfell stoner guide game throne minute series end start episode feel like new weekend family role play game throne ramsey wife cersei kid joffrey finally way people internet impossible open page scroll timeline game throne spoiler minute east coast release finally kick online habit come popular news list game throne subject rise petty politic trump tantrum remain air good nation winner game throne officially','wow people sign petition remake game throne season competent writer petition',...,'star war rian johnson game throne creator work new trilogy honestly not mind bit think work fantastic new trilogy','watch game throne documentary tonight','game throne trash glad tell people read book'],dtype=object)

y_train

array([0.23024851,0.47096001,0.2237812,0.07742384,0.05732593,0.06153279])```

tokenizer = Tokenizer()
tokenizer.fit_on_texts(X_train)

max_length = max([len(s) for s in X_train])
vocab_size = len(tokenizer.word_index) + 1
X_train_tokens = tokenizer.texts_to_sequences(X_train)
X_train_pad = pad_sequences(X_train_tokens,maxlen= max_length,padding= 'post')
embedding_dim = 100

model = Sequential()
model.add(Embedding(vocab_size,embedding_dim,input_length = max_length))
model.add(SpatialDropout1D(0.25))
model.add(LSTM(80,dropout=0.5))
model.add(Dropout(0.2))
model.add(Dense(1,activation='sigmoid'))
model.compile(loss='mse',optimizer='rmsprop')
print(model.summary())

model.fit(  X_train_pad,y_train,validation_split=0.2,batch_size= 64,epochs= 20)


Epoch 1/20
5031/5031 [==============================] - 612s 120ms/step - loss: 0.0062 - val_loss: 0.0055
Epoch 2/20
5031/5031 [==============================] - 603s 120ms/step - loss: 0.0056 - val_loss: 0.0057
Epoch 3/20
5031/5031 [==============================] - 601s 119ms/step - loss: 0.0056 - val_loss: 0.0060
Epoch 4/20
5031/5031 [==============================] - 603s 120ms/step - loss: 0.0056 - val_loss: 0.0055
Epoch 5/20
5031/5031 [==============================] - 601s 119ms/step - loss: 0.0056 - val_loss: 0.0057
Epoch 6/20
5031/5031 [==============================] - 602s 120ms/step - loss: 0.0055 - val_loss: 0.0060
Epoch 7/20
5031/5031 [==============================] - 600s 119ms/step - loss: 0.0056 - val_loss: 0.0060
Epoch 8/20
5031/5031 [==============================] - 603s 120ms/step - loss: 0.0056 - val_loss: 0.0058
Epoch 9/20
5031/5031 [==============================] - 601s 120ms/step - loss: 0.0056 - val_loss: 0.0055
Epoch 10/20
5031/5031 [==============================] - 607s 121ms/step - loss: 0.0055 - val_loss: 0.0057
Epoch 11/20
5031/5031 [==============================] - 604s 120ms/step - loss: 0.0056 - val_loss: 0.0056
Epoch 12/20
5031/5031 [==============================] - 600s 119ms/step - loss: 0.0055 - val_loss: 0.0059
Epoch 13/20
5031/5031 [==============================] - 600s 119ms/step - loss: 0.0056 - val_loss: 0.0059
Epoch 14/20
5031/5031 [==============================] - 605s 120ms/step - loss: 0.0056 - val_loss: 0.0062
Epoch 15/20
5031/5031 [==============================] - 605s 120ms/step - loss: 0.0056 - val_loss: 0.0055
Epoch 16/20
5031/5031 [==============================] - 602s 120ms/step - loss: 0.0055 - val_loss: 0.0056
Epoch 17/20
5031/5031 [==============================] - 604s 120ms/step - loss: 0.0055 - val_loss: 0.0059
Epoch 18/20
5031/5031 [==============================] - 601s 119ms/step - loss: 0.0054 - val_loss: 0.0058
Epoch 19/20
5031/5031 [==============================] - 599s 119ms/step - loss: 0.0056 - val_loss: 0.0057
Epoch 20/20
5031/5031 [==============================] - 599s 119ms/step - loss: 0.0056 - val_loss: 0.0054

如您所见，验证损失非常小，当我尝试预测某些值时，我的值与预测值相同。 我该如何解决？是否可以通过词嵌入来预测连续向量，还是只能预测类别？

解决方法

我已阅读您显示的整个代码，可以指出您在这里遗漏了一行：

model.add(Dense(1,activation='sigmoid')).

这里我们需要 activation = 'linear'，因为您正在进行回归，或者不定义激活参数，因为默认情况下是线性的。