错误：DQN 期望模型每个动作都有一个维度

如何解决错误：DQN 期望模型每个动作都有一个维度

我正在构建一个为其定义模型的 RL 代理：

def build_model(height,width,channels,actions):
    model = Sequential()
    model.add(Conv2D(32,(8,8),strides=(4,4),activation='relu',input_shape=(3,height,channels)))
    model.add(Conv2D(64,(4,strides=(2,2),activation='relu'))
    model.add(Dense(512,activation='relu'))
    model.add(Dense(256,activation='relu'))
    model.add(Dense(actions,activation='linear'))
    return model

然后创建一个模型：

env = gym.make('SpaceInvaders-v0')
height,channels = env.observation_space.shape
actions = env.action_space.n
model = build_model(height,actions)

然后定义代理：

def build_agent(model,actions):
    policy = LinearannealedPolicy(epsgreedyQPolicy(),attr='eps',value_max=1,value_min=.1,value_test=-.2,nb_steps=10000)
    memory = SequentialMemory(limit=2000,window_length=3)
    dqn = DQNAgent(model=model,memory=memory,policy=policy,dueling_type='avg',nb_actions=actions,nb_steps_warmup=1000)

抱怨我尝试构建代理：

dqn = build_agent(model,actions)

我收到此错误：

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-100-320eef5d391d> in <module>
----> 1 dqn = build_agent(model,actions)

<ipython-input-95-97801a497208> in build_agent(model,actions)
      2     policy = LinearannealedPolicy(epsgreedyQPolicy(),nb_steps=10000)
      3     memory = SequentialMemory(limit=2000,window_length=3)
----> 4     dqn = DQNAgent(model=model,nb_steps_warmup=1000)

~/Documents/RL/pRL/lib/python3.8/site-packages/rl/agents/dqn.py in __init__(self,model,policy,test_policy,enable_double_dqn,enable_dueling_network,dueling_type,*args,**kwargs)
    105         # Validate (important) input.
    106         if list(model.output.shape) != list((None,self.nb_actions)):
--> 107             raise ValueError(f'Model output "{model.output}" has invalid shape. DQN expects a model that has one dimension for each action,in this case {self.nb_actions}.')
    108 
    109         # Parameters.

ValueError: Model output "Tensor("dense_30/BiasAdd:0",shape=(None,3,24,18,6),dtype=float32)" has invalid shape. DQN expects a model that has one dimension for each action,in this case 6.

我检查了 model.output.shape 和 self.nb_actions 的值：

print(model.output_shape)
# (None,6)
print(actions) # This is self.nb_action
# 6

我不知道“model.output”的形状应该是什么。在这个错误中，它看起来应该是 '(None,6)' 但我不知道我必须做出什么改变才能得到这个形状。

解决方法

将 model.output.shape 转换为元组，然后比较 if tuple(model.output.shape) != (None,4)