如何解决PyTorch Optimizer.step不更新权重
我认为问题可能出在loss和loss.backward()步骤上,因为当我看一看loss.grad_fn链时,它并没有显示出我所期望的S型和线性层。
import torch
import torch.nn as nn
import torch.optim as optim
class Neuron(nn.Module):
def __init__(self):
super(Neuron,self).__init__()
# an affine operation: y = Wx + b
self.linear = nn.Linear(3,1,bias=False)
self.sig = nn.Sigmoid()
def forward(self,x):
linear_output = self.linear(x)
f = self.sig(linear_output)
return f
my_neuron = Neuron()
# data from above,appended a 1 to each row to match my implementation
x = torch.tensor([[1.2,1],[0.2,1.4,[0.5,0.5,[-1.5,-1.3,-1.4,[-0.7,-0.5,1]])
y = torch.tensor([0,1 ]) # targets
criterion = nn.CrossEntropyLoss()
optimizer = optim.SGD(my_neuron.parameters(),lr=0.01)
n = 10
for epoch in range(n): # loop over dataset n times
optimizer.zero_grad() # zero the gradient buffers
# run each data point through the network
output = torch.empty((6,6),dtype=torch.float)
idx = 0
for pt in x:
output[idx,:] = my_neuron(pt)
idx += 1
print(my_neuron.linear.weight[0])
loss = criterion(output,y)
loss.backward()
optimizer.step() # does the update step
print(loss.grad_fn)
print(loss.grad_fn.next_functions[0][0])
print(loss.grad_fn.next_functions[0][0].next_functions[0][0])
print(loss.item())
在这里,我输出了线性层的权重和参数,仅供比较。我还输出了loss.grad_fn链,您可以在其中看到线性层不存在。
my_neuron.linear.weight: tensor([-0.0055,0.1782,0.2472],grad_fn=<SelectBackward>)
my_neuron.parameters: tensor([-0.0055,grad_fn=<SelectBackward>)
my_neuron.linear.weight: tensor([-0.0055,grad_fn=<SelectBackward>)
<NllLossBackward object at 0x7fcfeae411d0>
<LogsoftmaxBackward object at 0x7fcfeafd36a0>
<copySlices object at 0x7fcfeae411d0>
版权声明:本文内容由互联网用户自发贡献,该文观点与技术仅代表作者本人。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容, 请发送邮件至 dio@foxmail.com 举报,一经查实,本站将立刻删除。