如何解决在 GoogleColab 中使用 autokeras 时达到磁盘限制
我想避免在 GoogleColab 中使用 autokeras 时达到磁盘限制时的错误终止。
我目前正在使用并安装一个 200GB 的 GoogleDrive 计划,并将 autokeras 训练历史记录存储在驱动器中,以减少 Colab 中使用的空间量。
我想知道是否有一种方法可以在不消耗 GoogleColab 磁盘空间的情况下使用 autokeras 进行训练。
错误:
OSError: [Errno 28] No space left on device
代码:
import shutil
import autokeras as ak
import keras as ks
from keras.utils import np_utils
from sklearn.datasets import fetch_openml
import pandas as pd
import numpy as np
import matplotlib
import matplotlib.pyplot as plt
from PIL import Image
import glob
import os
from keras.preprocessing.image import load_img,img_to_array
import sys
import pickle
import requests
from tqdm import tqdm
from io import StringIO
#my_module
from module import sep_dataset,line_send,pre_treatment,model_evaluation,set_datarange
train_folder_path = '/content/drive/MyDrive/data/subJ'
max_model = 30
def make_model():
input_node = ak.Imageinput()
output_node = ak.ImageBlock(normalize=False,augment=False,)(input_node)
output_node = ak.ClassificationHead()(output_node)
model = ak.AutoModel(inputs=input_node,outputs=output_node,overwrite=True,max_trials=max_model,directory=train_folder_path,)
return model
t1,v1 = sep_dataset.main(train_folder_path,0)
t2,v2 = sep_dataset.main(train_folder_path,1)
t3,v3 = sep_dataset.main(train_folder_path,2)
train = []
val = []
for i in range(len(t1)):
train.append([t1[i]]+[t2[i]]+[t3[i]])
val.append([v1[i]]+[v2[i]]+[v3[i]])
r,c,v,p = set_datarange.main(train_folder_path)
for count in range(p,len(val)):
for i in range(v,3):
X_train,Y_train = pre_treatment.main(train[count][i],len(train[count][i]))
X_test,Y_test = pre_treatment.main(val[count][i],len(val[count][i]))
model = make_model()
model.fit(X_train,Y_train,validation_data=(X_test,Y_test))
his = model.export_model()
save_path = os.path.join(train_folder_path.rsplit(os.sep,1)[0],'result_info','model.png')
model_evaluation.main(X_test,Y_test,model,val,train_folder_path,count,i)
v = 0
版权声明:本文内容由互联网用户自发贡献,该文观点与技术仅代表作者本人。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容, 请发送邮件至 dio@foxmail.com 举报,一经查实,本站将立刻删除。