如何解决Daskdistributed.core - 错误 - “元组”对象不支持项目分配
我在我的项目中使用了 Dask 和 cython,我在向客户端注册后调用了 cython 代码,并将从 cython 代码中获得的结果收集到我的 dask-python 代码中。当我使用processes = True创建一个集群时,它工作正常。但是,一旦我写了 processes=False,它就会给我以下错误:
distributed.core - ERROR - 'tuple' object does not support item assignment
Traceback (most recent call last):
File "/home/user/anaconda3/envs/Dask/lib/python3.7/site-packages/distributed/core.py",line 555,in handle_stream
msgs = await comm.read()
File "/home/user/anaconda3/envs/Dask/lib/python3.7/site-packages/distributed/comm/inproc.py",line 199,in read
msg = nested_deserialize(msg)
File "/home/user/anaconda3/envs/Dask/lib/python3.7/site-packages/distributed/protocol/serialize.py",line 549,in nested_deserialize
return replace_inner(x)
File "/home/user/anaconda3/envs/Dask/lib/python3.7/site-packages/distributed/protocol/serialize.py",line 541,in replace_inner
x[k] = replace_inner(v)
File "/home/user/anaconda3/envs/Dask/lib/python3.7/site-packages/distributed/protocol/serialize.py",line 534,in replace_inner
x[k] = deserialize(v.header,v.frames)
File "/home/user/anaconda3/envs/Dask/lib/python3.7/site-packages/distributed/protocol/serialize.py",line 388,in deserialize
return loads(header,frames)
File "/home/user/anaconda3/envs/Dask/lib/python3.7/site-packages/distributed/protocol/serialize.py",line 83,in pickle_loads
buffers[i] = buf.cast(mv.format)
TypeError: 'tuple' object does not support item assignment
distributed.worker - ERROR - 'tuple' object does not support item assignment
我使用的以下代码片段:
from dask.distributed import Client,LocalCluster,wait
cluster=LocalCluster(processes=False)
client=Client(cluster)
client.register_worker_callbacks(init_pyx) ## init_pyx is a function by which worker register to cython code
df = dd.read_csv(path_to_csv_file)
processed_df=df.map_partitions(lambda part: handle_partition(part),meta=meta) #handle partition is function which uses cython code to preprocess each partition of dataframe,meta is metadata related to preproceesed part
client.close()
另外,早些时候我遇到了与内存视图相关的问题,但是当我重新安装 dask 时,内存视图错误消失了。现在,与“TypeError: 'tuple' object does not support item assignment”相关的新错误即将到来。
我也是 Dask 的新手,如果有人知道如何解决它,请在这方面帮助我。 提前致谢。
版权声明:本文内容由互联网用户自发贡献,该文观点与技术仅代表作者本人。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容, 请发送邮件至 dio@foxmail.com 举报,一经查实,本站将立刻删除。