时间步与预期的time_step_spec不匹配-PyEnvironment

如何解决时间步与预期的time_step_spec不匹配-PyEnvironment

我目前正在为python的强化学习项目编写PyEnvironment。但是，当我尝试使用utils validate_py_environment（env，episodes = 5）验证环境时，出现以下错误：

Traceback (most recent call last):
  File "C:/Users/Cornelius/PycharmProjects/test/Reinforcment Learning.py",line 116,in <module>
    utils.validate_py_environment(env,episodes=5)
  File "C:\Users\Cornelius\PycharmProjects\test\venv\lib\site-packages\tf_agents\environments\utils.py",line 67,in validate_py_environment
    (time_step,time_step_spec))
ValueError: Given `time_step`: timestep(step_type=array(0),reward=array(0.,dtype=float32),discount=array(1.,observation=array([5.000000e+02,3.113270e+05,1.185080e+04,1.185619e+04,1.185153e+04,1.185234e+04,4.620000e-01,0.000000e+00,1.494143e+06,1.185152e+04,1.750795e+06,1.185572e+04,1.185993e+04,1.186221e+04,1.187471e+04,1.187774e+04,1.189487e+04,1.190100e+04],dtype=float32)) does not match expected `time_step_spec`: timestep(step_type=ArraySpec(shape=(),dtype=dtype('int32'),name='step_type'),reward=ArraySpec(shape=(),dtype=dtype('float32'),name='reward'),discount=BoundedArraySpec(shape=(),name='discount',minimum=0.0,maximum=1.0),observation=BoundedArraySpec(shape=(1,31),name='Positions',maximum=100000.0))

以下是相关代码：

class environment(py_environment.PyEnvironment):

    def __init__(self,data):
        self._action_spec = array_spec.BoundedArraySpec(shape=(),dtype=np.int32,minimum=0,maximum=2,name='Trade')
        self._observation_spec = array_spec.BoundedArraySpec(shape=(1,dtype=np.float32,maximum=100000,name='Positions')
        self.data = data
        self.number = 0
        # Initialize with first values
        self._state = self.data.take([self.number])
        self._episode_ended = False
        self.overall_profit = 0
        self.lastValue = None

    def _reset(self):
        # reset with first values
        # self._step = 0
        self._episode_ended = False
        self.overall_profit = 0
        self.number = 0
        self._state = self.data.iloc[int(self.number)]

        return ts.restart(np.array(self._state,dtype=np.float32))

我只发布了init并进行了重置，因为除重置外没有其他内容了。