我在
Windows上运行
Python并行处理.这是我的代码:
from joblib import Parallel,delayed def f(x): return sqrt(x) if __name__ == '__main__': a = Parallel(n_jobs=2)(delayed(f)(i) for i in range(10))
这是错误消息:
Process PoolWorker-2: Process PoolWorker-1: Traceback (most recent call last): File "C:\Users\yoyo__000.BIGBLACK\AppData\Local\Enthought\Canopy\App\appdata\canopy-1.5.4.3105.win-x86_64\lib\multiprocessing\process.py",line 258,in _bootstrap self.run() File "C:\Users\yoyo__000.BIGBLACK\AppData\Local\Enthought\Canopy\App\appdata\canopy-1.5.4.3105.win-x86_64\lib\multiprocessing\process.py",line 114,in run self._target(*self._args,**self._kwargs) File "C:\Users\yoyo__000.BIGBLACK\AppData\Local\Enthought\Canopy\App\appdata\canopy-1.5.4.3105.win-x86_64\lib\multiprocessing\pool.py",line 102,in worker task = get() File "C:\Users\yoyo__000.BIGBLACK\AppData\Local\Enthought\Canopy\User\lib\site-packages\joblib\pool.py",line 363,in get return recv() AttributeError: 'module' object has no attribute 'f'
解决方法
根据
this site,问题是Windows特定的:
Yes: under linux we are forking,thus their is no need to pickle the
function,and it works fine. Under windows,the function needs to be
pickleable,ie it needs to be imported from another file. This is
actually good practice: making modules pushes for reuse.
我已经尝试过你的代码,它在Linux下运行完美.
在Windows下,如果从脚本运行,它运行正常,如python script_with_your_code.py.但是在交互式python会话中运行时失败了.当我将f函数保存在单独的模块中并将其导入到我的交互式会话中时,它对我有用.
不工作:
互动环节:
>>> from math import sqrt >>> from joblib import Parallel,delayed >>> def f(x): ... return sqrt(x) >>> if __name__ == '__main__': ... a = Parallel(n_jobs=2)(delayed(f)(i) for i in range(10)) ... Process PoolWorker-1: Traceback (most recent call last): File "C:\Python27\lib\multiprocessing\process.py",in _bootstrap self.run() File "C:\Python27\lib\multiprocessing\process.py",in run self._target(*self._args,**self._kwargs) File "C:\Python27\lib\multiprocessing\pool.py",in worker task = get() File "C:\Python27\lib\site-packages\joblib\pool.py",line 359,in get return recv() AttributeError: 'module' object has no attribute 'f'
工作方式:
fun.py
from math import sqrt def f(x): return sqrt(x)
互动环节:
>>> from joblib import Parallel,delayed >>> from fun import f >>> if __name__ == '__main__': ... a = Parallel(n_jobs=2)(delayed(f)(i) for i in range(10)) ... >>> a [0.0,1.0,1.4142135623730951,1.7320508075688772,2.0,2.23606797749979,2.449489742783178,2.6457513110645907,2.8284271247461903,3.0]
版权声明:本文内容由互联网用户自发贡献,该文观点与技术仅代表作者本人。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容, 请发送邮件至 dio@foxmail.com 举报,一经查实,本站将立刻删除。