在 multiprocessing.Manager().dict()

如何解决在 multiprocessing.Manager().dict()

长话短说，我试图在主线程中打印 multiprocessing.Manager().dict() 的内容，但出现了一个奇怪的错误。

我仅在池数大于 20（是的，正好是 20）并且仅在 macOS 上（在 Linux 上完美运行）时遇到此错误。

规格：

  cpu: Intel(R) Core(TM) i9-9880H cpu @ 2.30GHz
macOS: 11.2.3

附上最小错误代码：

#!/usr/bin/env python3

from contextlib import suppress
import multiprocessing as mp
import time


def run():
    D[mp.current_process().name] = 'some val'
    time.sleep(0.5)


if __name__ == '__main__':
    mp.set_start_method('fork')
    D,rets = mp.Manager().dict(),[]
    with mp.Pool(25) as p:
        for _ in range(33):
            rets.append(p.apply_async(run))
        while rets:
            for ret in rets[:]:
                with suppress(mp.TimeoutError):
                    ret.get(timeout=0)
                    rets.remove(ret)
                    print(len(D))

错误：

multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
  File "/usr/local/Cellar/python@3.9/3.9.2_4/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/managers.py",line 801,in _callmethod
    conn = self._tls.connection
AttributeError: 'ForkAwareLocal' object has no attribute 'connection'

During handling of the above exception,another exception occurred:

Traceback (most recent call last):
  File "/usr/local/Cellar/python@3.9/3.9.2_4/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/pool.py",line 125,in worker
    result = (True,func(*args,**kwds))
  File "/Users/???",line 9,in run
    D[mp.current_process().name] = 'some val'
  File "<string>",line 2,in __setitem__
  File "/usr/local/Cellar/python@3.9/3.9.2_4/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/managers.py",line 805,in _callmethod
    self._connect()
  File "/usr/local/Cellar/python@3.9/3.9.2_4/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/managers.py",line 792,in _connect
    conn = self._Client(self._token.address,authkey=self._authkey)
  File "/usr/local/Cellar/python@3.9/3.9.2_4/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/connection.py",line 507,in Client
    c = SocketClient(address)
  File "/usr/local/Cellar/python@3.9/3.9.2_4/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/connection.py",line 635,in SocketClient
    s.connect(address)
ConnectionRefusedError: [Errno 61] Connection refused
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/???",line 22,in <module>
    ret.get(timeout=0)
  File "/usr/local/Cellar/python@3.9/3.9.2_4/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/pool.py",line 771,in get
    raise self._value
ConnectionRefusedError: [Errno 61] Connection refused