使用fit_generator的训练模型不显示val_loss和val_acc并在第一个时期中断

如何解决使用fit_generator的训练模型不显示val_loss和val_acc并在第一个时期中断

我实现了一个数据生成器，将训练数据分成256个小批量，以避免内存错误。它在训练数据上运行，但在每个时期结束时未显示验证损失和验证准确性。我还将数据生成器应用于验证数据并定义了验证步骤。我不知道代码没有显示验证损失和准确性的确切原因是什么？ 这是代码：

early_stopping_cb=tf.keras.callbacks.EarlyStopping(patience=3,restore_best_weights=True)
batch_size=256
epoch_steps=math.ceil(len(utt)/ batch_size)
val_steps=math.ceil(len(val_prev)/ batch_size)

hist = model.fit_generator(generate_data(utt_minus_one,utt,y_train,batch_size),steps_per_epoch=epoch_steps,epochs=3,callbacks = [early_stopping_cb],validation_data=generate_data(val_prev,val_curr,y_val,validation_steps=val_steps,class_weight=custom_weight_dict,verbose=1)

这是生成器的代码：

#method to use generator to split data into mini batches of 256 each loaded at run time
def generate_data(X1,X2,Y,batch_size):
  p_input=[]
  c_input=[]
  target=[]
  batch_count=0
  for i in range(len(X1)):
    p_input.append(X1[i])
    c_input.append(X2[i])
    target.append(Y[i])
    batch_count+=1
    if batch_count>batch_size:
      prev_X=np.array(p_input,dtype=np.int64)
      cur_X=np.array(c_input,dtype=np.int64)
      cur_y=np.array(target,dtype=np.int32)
      yield ([prev_X,cur_X],cur_y ) 
      p_input=[]
      c_input=[]
      target=[]
      batch_count=0
  return

这是第一个纪元的轨迹，也给出了错误：

Epoch 1/3
346/348 [============================>.] - ETA: 4s - batch: 172.5000 - size: 257.0000 - loss: 0.8972 - accuracy: 0.8424WARNING:tensorflow:Your dataset iterator ran out of data; interrupting training. Make sure that your iterator can generate at least `steps_per_epoch * epochs` batches (in this case,1044 batches). You may need touse the repeat() function when building your dataset.
WARNING:tensorflow:Early stopping conditioned on metric `val_loss` which is not available. Available metrics are: loss,accuracy
346/348 [============================>.] - 858s 2s/step - batch: 172.5000 - size: 257.0000 - loss: 0.8972 - accuracy: 0.8424

任何人都可以帮助您解决这些问题吗？

解决方法

在for循环上需要每个时期有一个while循环，以分成小批。因此，如果每个时期有348个批次，则总共3 * 348 = 1044个批次。

#method to use generator to split data into mini batches of 256 each loaded at run time
def generate_data(X1,X2,Y,batch_size):
  count=0
  p_input=[]
  c_input=[]
  target=[]
  batch_count=0
  while True:
    for i in range(len(X1)):
      p_input.append(X1[i])
      c_input.append(X2[i])
      target.append(Y[i])
      batch_count+=1
      if batch_count>batch_size:
        count=count+1
        prev_X=np.array(p_input,dtype=np.int64)
        cur_X=np.array(c_input,dtype=np.int64)
        cur_y=np.array(target,dtype=np.int32)
        yield ([prev_X,cur_X],cur_y ) 
        p_input=[]
        c_input=[]
        target=[]
        batch_count=0
    print(count)
  return

并跟踪第一个时期：

Epoch 1/3
335/347 [===========================>..] - ETA: 30s - batch: 167.0000 - size: 257.0000 - loss: 1.2734 - accuracy: 0.8105346
347/347 [==============================] - ETA: 0s - batch: 173.0000 - size: 257.0000 - loss: 1.2635 - accuracy: 0.8113WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training_v1.py:2048: Model.state_updates (from tensorflow.python.keras.engine.training) is deprecated and will be removed in a future version.
Instructions for updating:
This property should not be used in TensorFlow 2.0,as updates are applied automatically.
86
347/347 [==============================] - 964s 3s/step - batch: 173.0000 - size: 257.0000 - loss: 1.2635 - accuracy: 0.8113 - val_loss: 0.5700 - val_accuracy: 0.8367