微信公众号搜"智元新知"关注
微信扫一扫可直接关注哦!

类对象的Pathos多处理失败?

如何解决类对象的Pathos多处理失败?

我将Pathos.Multiprocessing用于以下代码之一,并且可以正常工作:

class A:

    def B(self,text):
        obj = some_class()
        return obj._some_func2(raw_text,obj.some_func)

    def C(self,obj.some_func)

    def D(self,text):
        obj = some_class()
        return obj._some_func2(text,lambda t: obj.some_func(obj.some_func(t)))

    def E(self,lambda t: obj.some_func(obj.some_func(t)))

    def F(self,file_name,operation):
        num_workers = cpu_count() - 2
        pool = Pool(num_workers)
        with open(file_name) as f:
            results = pool.map(operation,f)

        print(results)

但是,当我有一个init函数时,我尝试在那里初始化对象,然后尝试在该类的所有函数中使用它,如下所示:

class A:

    def __init__(self):

        self.obj = some_class()

    def B(self,f)

        print(results)

我收到此错误

python3.6/lib/python3.6/site-packages/pathos/parallel.py:234: in map
return list(self.imap(f,*args))
python3.6/lib/python3.6/site-packages/pathos/parallel.py:247: in imap
return (subproc() for subproc in list(builtins.map(submit,*args)))
python3.6/lib/python3.6/site-packages/pathos/parallel.py:243: in submit
return _pool.submit(f,argz,globals=globals())
python3.6/lib/python3.6/site-packages/pp/_pp.py:500: in submit
sargs = pickle.dumps(args,self.__pickle_proto)
python3.6/lib/python3.6/site-packages/dill/_dill.py:265: in dumps
dump(obj,file,protocol,byref,fmode,recurse,**kwds)#,strictio)
python3.6/lib/python3.6/site-packages/dill/_dill.py:259: in dump
Pickler(file,**_kwds).dump(obj)
python3.6/lib/python3.6/site-packages/dill/_dill.py:446: in dump
StockPickler.dump(self,obj)
python3.6/lib/python3.6/pickle.py:409: in dump
self.save(obj)
python3.6/lib/python3.6/pickle.py:476: in save
f(self,obj) # Call unbound method with explicit self
python3.6/lib/python3.6/pickle.py:736: in save_tuple
save(element)
python3.6/lib/python3.6/pickle.py:521: in save
self.save_reduce(obj=obj,*rv)
python3.6/lib/python3.6/pickle.py:634: in save_reduce
save(state)
python3.6/lib/python3.6/pickle.py:476: in save
f(self,obj) # Call unbound method with explicit self
python3.6/lib/python3.6/site-packages/dill/_dill.py:933: in save_module_dict
StockPickler.save_dict(pickler,obj)
python3.6/lib/python3.6/pickle.py:821: in save_dict
self._batch_setitems(obj.items())
python3.6/lib/python3.6/pickle.py:852: in _batch_setitems
save(v)
python3.6/lib/python3.6/pickle.py:521: in save
self.save_reduce(obj=obj,obj)
python3.6/lib/python3.6/pickle.py:821: in save_dict
self._batch_setitems(obj.items())
python3.6/lib/python3.6/pickle.py:847: in _batch_setitems
save(v)
python3.6/lib/python3.6/pickle.py:521: in save
self.save_reduce(obj=obj,obj)
python3.6/lib/python3.6/pickle.py:821: in save_dict
self._batch_setitems(obj.items())
python3.6/lib/python3.6/pickle.py:847: in _batch_setitems
save(v)
python3.6/lib/python3.6/pickle.py:496: in save
rv = reduce(self.proto)
E   TypeError: 'bool' object is not callable

现在,我可以看到正在调用莳萝,而且还可以看到pickle.py。 不能完全使用莳萝的Pathos引起了这里的问题,或者Pathos是否也限制了我们在类内构造代码的方式,因为使用init初始化变量会破坏Pathos?

为了高效的多处理功能是否必须是顶级的?

版权声明:本文内容由互联网用户自发贡献,该文观点与技术仅代表作者本人。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容, 请发送邮件至 dio@foxmail.com 举报,一经查实,本站将立刻删除。