如何使用图像数据生成器和数据帧流使用CSV文件实施K折交叉验证

如何解决如何使用图像数据生成器和数据帧流使用CSV文件实施K折交叉验证

请显示或解释一个虚拟的示例代码片段，以演示针对Keras的Flow_from_Dataframe，Training_Generator和Valid_Generator对象进行K折交叉验证。这是我当前拥有的代码（没有k-fold只是简单的拟合）：

ImageDataGen对象以执行所有扩充

IMG_SIZE = (150,150)
core_idg = ImageDataGenerator(samplewise_center=True,samplewise_std_normalization=True,horizontal_flip = True,vertical_flip = False,height_shift_range= 0.05,width_shift_range=0.1,rotation_range=5,shear_range = 0.1,fill_mode = 'reflect',zoom_range=0.15)

将主数据帧拆分为train_dataframe和valid_dataframe

train_df,valid_df = train_test_split(main_DF,test_size = 0.10,random_state = 2018,stratify = df_large['BINARY'].map(lambda x: x))

使用之前创建的ImageDatagen对象的flow_from_dataframe方法创建train_gen和valid_gen。

“ IMAGE_NAMES”和“ BINARY”是由图像名称和标签0或1组成的列。

all_labels = [ "0","1" ]

train_gen = core_idg.flow_from_dataframe(dataframe=train_df,directory="./DataFolder/",x_col = 'IMAGE_NAMES',y_col = 'BINARY',class_mode = 'categorical',classes = all_labels,target_size = IMG_SIZE,color_mode = 'rgb',batch_size = 64)

valid_gen = core_idg.flow_from_dataframe(dataframe=valid_df,batch_size = 256)

test_X,test_Y = next(core_idg.flow_from_dataframe(dataframe=valid_df,y_col = 'BIN_STR',batch_size = 256))

#fitting
hist = model.fit_generator(train_gen,validation_data = (test_X,test_Y),epochs = 30,callbacks = call_list)

现在如何将其转换为K折交叉验证？根据我的说法，core_idg必须在K-Fold循环之外创建一次，而不是使用train_df和valid_df，我们应该使用index的K-Fold方法进行拆分。那么我提到的代码片段如何可以转换？

解决方法

像这样的事情对我有用，在K形折叠循环内创建数据框

IMG_SIZE = (150,150)
core_idg = ImageDataGenerator(samplewise_center=True,samplewise_std_normalization=True,horizontal_flip = True,vertical_flip = False,height_shift_range= 0.05,width_shift_range=0.1,rotation_range=5,shear_range = 0.1,fill_mode = 'reflect',zoom_range=0.15)

# Training with K-fold cross validation
kf = KFold(n_splits=k_folds,random_state=None,shuffle=True)
X= np.array(df_large["IMAGE_NAMES"])
i = 1
for train_index,test_index in kf.split(X):
    trainData = X[train_index]
    testData = X[test_index]
    ## create train,valid dataframe and thus train_gen,valid_gen for each fold-loop
    train_df = df_large.loc[df_large["IMAGE_NAMES"].isin(list(trainData))]
    valid_df = df_large.loc[df_large["IMAGE_NAMES"].isin(list(testData))]
    #create model object
    model= build_model()
    all_labels = [ "0","1" ]
    train_gen = core_idg.flow_from_dataframe(dataframe=train_df,directory="./DataFolder/",x_col = 'IMAGE_NAMES',y_col = 'BINARY',class_mode = 'categorical',classes = all_labels,target_size = IMG_SIZE,color_mode = 'rgb',batch_size = 64)
    valid_gen = core_idg.flow_from_dataframe(dataframe=valid_df,batch_size = 256)
    hist = img_classify.fit_generator(
            train_gen,steps_per_epoch= len(trainData),epochs= n_epochs,validation_data=valid_gen,callbacks = callback_list
)

如果有任何改善建议，请发表评论。