如何解决我们如何解释下面的验证损失?
我在 CNN+LSTM 编码器和解码器模型中训练了不同数量的层。 我面临的问题对我来说很奇怪。验证损失在 3.*** 左右波动。从下面的损失图中我们可以看出。我有3 CNN layer+1 layer BLSTM at encoder and 1 LSTM at decoder
3 layer CNN+2 layers of BLSTM at encoder and 1 layer LSTM at encoder
我也尝试过从 0.1 到 0.000001 的重量衰减。但我仍然得到这种类型的损失图。请注意,模型的准确性在验证和训练集上都在增加。验证损失怎么可能仍然在 3 左右,但准确度却在增加?有人能解释一下吗?
谢谢 ` 类编码器(nn.Module): def init(self,height,width,enc_hid_dim,dec_hid_dim,dropout): super().init() self.height=高度 self.enc_hid_dim=enc_hid_dim self.width=width
self.layer0 = nn.Sequential(
nn.Conv2d(1,8,kernel_size=(3,3),stride =(1,1),padding=1),nn.ReLU(),nn.Batchnorm2d(8),nn.MaxPool2d(2,2),)
self.layer1 = nn.Sequential(
nn.Conv2d(8,32,nn.Batchnorm2d(32),)
self.layer2 = nn.Sequential(
nn.Conv2d(32,64,nn.Batchnorm2d(64),2)
)
self.rnn = nn.LSTM(self.height//8*64,self.enc_hid_dim,bidirectional=True)
self.fc = nn.Linear(enc_hid_dim * 2,dec_hid_dim)
self.dropout = nn.Dropout(dropout)
self.cnn_dropout = nn.Dropout(p=0.2)
def forward(self,src,in_data_len,train):
batch_size = src.shape[0]
out = self.layer0(src)
out = self.layer1(out)
out = self.layer2(out)
out = self.dropout(out) # torch.Size([batch,channel,h,w])
out = out.permute(3,2,1) # (width,batch,channels)
out.contiguous()
out = out.reshape(-1,batch_size,self.height//8*64) #(w,(height,channels))
width = out.shape[0]
src_len = in_data_len.numpy()*(width/self.width)
src_len = src_len + 0.999 # in case of 0 length value from float to int
src_len = src_len.astype('int')
out = pack_padded_sequence(out,src_len.tolist(),batch_first=False)
outputs,hidden_out = self.rnn(out)
hidden=hidden_out[0]
cell=hidden_out[1]
# output: t,b,f*2 hidden: 2,f
outputs,output_len = pad_packed_sequence(outputs,batch_first=False)
hidden = torch.tanh(self.fc(torch.cat((hidden[-2,:,:],hidden[-1,:]),dim = 1)))
cell = torch.tanh(self.fc(torch.cat((cell[-2,cell[-1,dim = 1)))
return outputs,hidden,cell,output_len
类解码器(nn.Module): def init(self,output_dim,emb_dim,dropout,attention): super().init()
self.output_dim = output_dim
self.attention = attention
self.embedding = nn.Embedding(output_dim,emb_dim)
self.rnn = nn.LSTM((enc_hid_dim * 2) + emb_dim,dec_hid_dim)
self.fc_out = nn.Linear((enc_hid_dim * 2) + dec_hid_dim + emb_dim,output_dim)
self.dropout_layer = nn.Dropout(dropout)
def forward(self,input,encoder_outputs,train):
input=torch.topk(input,1)[1]
embedded = self.embedding(input)
if train:
embedded=self.dropout_layer(embedded)
embedded = embedded.permute(1,2)
#embedded = [1,batch size,emb dim]
a = self.attention(hidden,encoder_outputs)
#a = [batch size,src len]
a = a.unsqueeze(1)
#a = [batch size,1,src len]
encoder_outputs = encoder_outputs.permute(1,2)
#encoder_outputs = [batch size,src len,enc hid dim * 2]
weighted = torch.bmm(a,encoder_outputs)
weighted = weighted.permute(1,2)
#weighted = [1,enc hid dim * 2]
rnn_input = torch.cat((embedded,weighted),dim = 2)
output,hidden_out = self.rnn(rnn_input (hidden.unsqueeze(0),cell.unsqueeze(0)))
hidden=hidden_out[0]
cell=hidden_out[1]
assert (output == hidden).all()
embedded = embedded.squeeze(0)
output = output.squeeze(0)
weighted = weighted.squeeze(0)
prediction = self.fc_out(torch.cat((output,weighted,embedded),dim = 1))
return prediction,hidden.squeeze(0),cell.squeeze(0)
`
版权声明:本文内容由互联网用户自发贡献,该文观点与技术仅代表作者本人。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容, 请发送邮件至 dio@foxmail.com 举报,一经查实,本站将立刻删除。