使用 dill 将一个类伪装成 python python-pickleable

如何解决使用 dill 将一个类伪装成 python python-pickleable

我正在使用 pytorch，而 pytorch 又依赖于 python 本地 pickle 模块进行多处理。

我有一个不可pickle的类，因为它包含一些lambda（我不想改变它）。由于我无法更改 pytorch 的多处理实现（以支持 dill），因此我希望我的类是 python 原生的pickleable。我想知道我是否可以更改我的类，以便从外部可以腌制，但在内部仅指 dill？

class NonPickleableClass:
    #...
    def __getstate__(self):
        # HACK: make it pickleable with dill :D
        return dill.dumps(self.__dict__)

    def __setstate__(self,state):
        self.__dict__.update(dill.loads(state))

虽然它在我的单元测试中运行良好，但它在 pytorch 中超出递归限制而崩溃（不知道确切原因？）。

  File "/home/AD/muellelu/xconda3/envs/hand_seg/lib/python3.9/pickle.py",line 955,in _batch_appends
    save(x)
  File "/home/AD/muellelu/xconda3/envs/hand_seg/lib/python3.9/pickle.py",line 692,in save_reduce
    save(args)
  File "/home/AD/muellelu/xconda3/envs/hand_seg/lib/python3.9/pickle.py",line 691,in save_reduce
    save(func)
  File "/home/AD/muellelu/xconda3/envs/hand_seg/lib/python3.9/pickle.py",line 560,in save
    f(self,obj)  # Call unbound method with explicit self
  File "/home/AD/muellelu/xconda3/envs/hand_seg/lib/python3.9/pickle.py",line 536,in save
    self.framer.commit_frame()
  File "/home/AD/muellelu/xconda3/envs/hand_seg/lib/python3.9/site-packages/dill/_dill.py",line 1444,in save_function
    pickler.save_reduce(_create_function,(obj.__code__,File "/home/AD/muellelu/xconda3/envs/hand_seg/lib/python3.9/pickle.py",line 901,in save_tuple
    save(element)
  File "/home/AD/muellelu/xconda3/envs/hand_seg/lib/python3.9/pickle.py",line 220,in commit_frame
    if f.tell() >= self._FRAME_SIZE_TARGET or force:
  File "/home/AD/muellelu/xconda3/envs/hand_seg/lib/python3.9/pickle.py",line 941,in save_module_dict
    StockPickler.save_dict(pickler,obj)
  File "/home/AD/muellelu/xconda3/envs/hand_seg/lib/python3.9/pickle.py",line 971,in save_dict
    self._batch_setitems(obj.items())
  File "/home/AD/muellelu/xconda3/envs/hand_seg/lib/python3.9/pickle.py",line 1002,in _batch_setitems
    save(v)
  File "/home/AD/muellelu/xconda3/envs/hand_seg/lib/python3.9/pickle.py",line 931,in save_list
    self._batch_appends(obj)
  File "/home/AD/muellelu/xconda3/envs/hand_seg/lib/python3.9/pickle.py",obj)  # Call unbound method with explicit self
  File "/home/AD/muellelu/xconda3/envs/hand_seg/lib/python3.9/site-packages/dill/_dill.py",RecursionError: maximum recursion depth exceeded while calling a Python object
  File "/home/AD/muellelu/xconda3/envs/hand_seg/lib/python3.9/pickle.py",in save
    self.framer.commit_frame()
  File "/home/AD/muellelu/xconda3/envs/hand_seg/lib/python3.9/pickle.py",in commit_frame
    if f.tell() >= self._FRAME_SIZE_TARGET or force:
RecursionError: maximum recursion depth exceeded while calling a Python object
RecursionError: maximum recursion depth exceeded while calling a Python object