如何解决BrokenPipeError:[Errno 32]管道损坏当使用multiprocessing.Process并调用process.start时,出现此错误怎么解决呢?
我正在尝试使我的代码通过多处理工作,以缩短我的代码的执行时间。但是在使用multiprocessing.Process(...)函数创建了一个进程之后,我调用了start()函数以启动我创建的该进程,但是我收到以下错误消息:
brokenPipeError:[Errno 32]管道损坏
请注意,我使用KFold()以4种不同的方式分离数据集,以对模型执行交叉验证。
这是我的代码:
def cross_validation():
w = initialization_of_parameters(7)
x,y = get_dataset_celiac(7)
x,y = shuffle(x,y)
x_train,x_test,y_train,y_test = [],[],[]
kf = KFold(n_splits=4)
for train_index,test_index in kf.split(x):
x_train.append(x[train_index])
x_test.append(x[test_index])
y_train.append(y[train_index])
y_test.append(y[test_index])
p1 = Process(target=train,args=(w,10,0.01,0.5,x_train[0],y_train[0],x_test[0],y_test[0],))
p2 = Process(target=train,x_train[1],y_train[1],x_test[1],y_test[1],))
p3 = Process(target=train,x_train[2],y_train[2],x_test[2],y_test[2],))
p4 = Process(target=train,x_train[3],y_train[3],x_test[3],y_test[3],))
p1.start()
p2.start()
p3.start()
p4.start()
p1.join()
p2.join()
p3.join()
p4.join()
def main():
cross_validation()
if __name__ == "__main__":
main()
这是我的错误:
---------------------------------------------------------------------------
brokenPipeError Traceback (most recent call last)
<ipython-input-1-7b3adfb29bf6> in <module>
170
171 if __name__ == "__main__":
--> 172 main()
<ipython-input-1-7b3adfb29bf6> in main()
167
168 def main():
--> 169 cross_validation()
170
171 if __name__ == "__main__":
<ipython-input-1-7b3adfb29bf6> in cross_validation()
156 p3 = Process(target=train,))
157 p4 = Process(target=train,))
--> 158 p1.start()
159 p2.start()
160 p3.start()
~\anaconda3\lib\multiprocessing\process.py in start(self)
110 'daemonic processes are not allowed to have children'
111 _cleanup()
--> 112 self._popen = self._Popen(self)
113 self._sentinel = self._popen.sentinel
114 # Avoid a refcycle if the target function holds an indirect
~\anaconda3\lib\multiprocessing\context.py in _Popen(process_obj)
221 @staticmethod
222 def _Popen(process_obj):
--> 223 return _default_context.get_context().Process._Popen(process_obj)
224
225 class DefaultContext(BaseContext):
~\anaconda3\lib\multiprocessing\context.py in _Popen(process_obj)
320 def _Popen(process_obj):
321 from .popen_spawn_win32 import Popen
--> 322 return Popen(process_obj)
323
324 class SpawnContext(BaseContext):
~\anaconda3\lib\multiprocessing\popen_spawn_win32.py in __init__(self,process_obj)
87 try:
88 reduction.dump(prep_data,to_child)
---> 89 reduction.dump(process_obj,to_child)
90 finally:
91 set_spawning_popen(None)
~\anaconda3\lib\multiprocessing\reduction.py in dump(obj,file,protocol)
58 def dump(obj,protocol=None):
59 '''Replacement for pickle.dump() using ForkingPickler.'''
---> 60 ForkingPickler(file,protocol).dump(obj)
61
62 #
brokenPipeError: [Errno 32] broken pipe
请注意,我正在使用jupyter笔记本,并且我的计算机具有4核,如果有帮助的话,可以使用8GB的ram。如果缺少任何信息,请告诉我。
请问有人有解决这个问题的想法吗?
版权声明:本文内容由互联网用户自发贡献,该文观点与技术仅代表作者本人。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容, 请发送邮件至 dio@foxmail.com 举报,一经查实,本站将立刻删除。