如何解决无法在 pathos.multiprocessing
我想并行化多个代理的模拟。因为我希望我的结果是一个类的实例,为了避免序列化问题,我使用 pathos.multiprocessing 而不是 multiprocessing。
我是这样做的:
import pathos.multiprocessing as mp
sim_agent(T): #simulate single agent for T periods
ag = agent()
for t in range(T):
ag.step()
return ag
def simulate_parallel(N,T):
if __name__ == '__main__':
pool = mp.Processpool()
results = pool.amap(sim_agent,[T]*N)
agents = results.get()
return agents
我可以运行一次simulate_parallel。当我再次执行时,出现错误:
--> 142 agents = results.get()
~/anaconda3/anaconda3/lib/python3.7/site-packages/multiprocess/pool.py in get(self,timeout)
655 return self._value
656 else:
--> 657 raise self._value
658
659 def _set(self,i,obj):
~/anaconda3/anaconda3/lib/python3.7/site-packages/multiprocess/pool.py in _handle_tasks(taskqueue,put,outqueue,pool,cache)
429 break
430 try:
--> 431 put(task)
432 except Exception as e:
433 job,idx = task[:2]
~/anaconda3/anaconda3/lib/python3.7/site-packages/multiprocess/connection.py in send(self,obj)
207 self._check_closed()
208 self._check_writable()
--> 209 self._send_bytes(_ForkingPickler.dumps(obj))
210
211 def recv_bytes(self,maxlength=None):
~/anaconda3/anaconda3/lib/python3.7/site-packages/multiprocess/connection.py in _send_bytes(self,buf)
394 n = len(buf)
395 # For wire compatibility with 3.2 and lower
--> 396 header = struct.pack("!i",n)
397 if n > 16384:
398 # The payload is large so Nagle's algorithm won't be triggered
error: 'i' format requires -2147483648 <= number <= 2147483647
当我尝试在其他函数中使用 pool.amap() 时,我也遇到同样的错误。即使我使 N 和 T 变小,它仍然是相同的行为:我可以运行 simulate_parallel 一次,而不能运行第二次。为什么会这样?谢谢!
版权声明:本文内容由互联网用户自发贡献,该文观点与技术仅代表作者本人。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容, 请发送邮件至 dio@foxmail.com 举报,一经查实,本站将立刻删除。